Propagating Explicit Congestion Notification across IP Tunnel Headers Separated by a Shim Independent
United Kingdom ietf@bobbriscoe.net https://bobbriscoe.net/
tsv tsvwg Congestion Control and Management Congestion Notification Information Security Tunnelling Encapsulation & Decapsulation Protocol ECN Layering RFC 6040 on "Tunnelling of Explicit Congestion Notification" made the rules for propagation of Explicit Congestion Notification (ECN) consistent for all forms of IP-in-IP tunnel. This specification updates RFC 6040 to clarify that its scope includes tunnels where two IP headers are separated by at least one shim header that is not sufficient on its own for wide-area packet forwarding. It surveys widely deployed IP tunnelling protocols that use such shim headers and updates the specifications of those that do not mention ECN propagation (including RFCs 2661, 3931, 2784, 4380 and 7450, which specify L2TPv2, L2TPv3, Generic Routing Encapsulation (GRE), Teredo, and Automatic Multicast Tunneling (AMT), respectively). This specification also updates RFC 6040 with configuration requirements needed to make any legacy tunnel ingress safe.
Status of This Memo This is an Internet Standards Track document. This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841. Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at .
Copyright Notice Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents () in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English.
Table of Contents
  • .  Introduction
  • .  Terminology
  • .  Scope of RFC 6040
    • .  Feasibility of ECN Propagation between Tunnel Headers
    • .  Desirability of ECN Propagation between Tunnel Headers
  • .  Making a Non-ECN Tunnel Ingress Safe by Configuration
  • .  ECN Propagation and Fragmentation/Reassembly
  • .  IP-in-IP Tunnels with Tightly Coupled Shim Headers
    • .  Specific Updates to Protocols under IETF Change Control
      • .  L2TP (v2 and v3) ECN Extension
      • .  GRE
      • .  Teredo
      • .  AMT
  • .  IANA Considerations
  • .  Security Considerations
  • .  References
    • .  Normative References
    • .  Informative References
  • Acknowledgements
  • Author's Address
Introduction on "Tunnelling of Explicit Congestion Notification" made the rules for propagation of Explicit Congestion Notification (ECN) consistent for all forms of IP-in-IP tunnel. A common pattern for many tunnelling protocols is to encapsulate an inner IP header (v4 or v6) with one or more shim headers then an outer IP header (v4 or v6). Some of these shim headers are designed as generic encapsulations, so they do not necessarily directly encapsulate an inner IP header. Instead, they can encapsulate headers such as link-layer (L2) protocols that, in turn, often encapsulate IP. Thus, the abbreviation 'IP-shim-(L2)-IP' can be used for tunnels that are in scope of this document. To clear up confusion, this specification clarifies that the scope of includes any IP-in-IP tunnel, including those with one or more shim headers and other encapsulations between the IP headers. Where necessary, it updates the specifications of the relevant encapsulation protocols with the specific text necessary to comply with . This specification also updates to state how operators ought to configure a legacy tunnel ingress to avoid unsafe system configurations.
Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. This specification uses the terminology defined in .
Scope of RFC 6040 In , its scope is defined as:
...ECN field processing at encapsulation and decapsulation for any IP-in-IP tunnelling, whether IPsec or non-IPsec tunnels. It applies irrespective of whether IPv4 or IPv6 is used for either the inner or outer headers.
There are two problems with the above scoping statement: Problem 1: It was intended to include cases where one or more shim headers sit between the IP headers. Many tunnelling implementers have interpreted the scope of as it was intended, but it is ambiguous. Therefore, this specification updates by adding the following scoping text after the sentences quoted above:
It applies in cases where an outer IP header encapsulates an inner IP header either directly or indirectly by encapsulating other headers that in turn encapsulate (or might encapsulate) an inner IP header.
Problem 2: Like many IETF specifications, is written as a specification that implementations can choose to claim compliance with. This means it does not cover two important situations:
  1. Cases where it is infeasible for an implementation to access an inner IP header when adding or removing an outer IP header
  2. Cases where implementations choose not to propagate ECN between IP headers
However, the ECN field is a non-optional part of the IP header (v4 and v6), so any implementation that creates an outer IP header has to give the ECN field some value. There is only one safe value a tunnel ingress can use if it does not know whether the egress supports propagation of the ECN field; it has to clear the ECN field in any outer IP header to 0b00. However, an RFC has no jurisdiction over implementations that choose not to comply or cannot comply with the RFC, including all implementations that predated it. Therefore, it would have been unreasonable to add such a requirement to . Nonetheless, to ensure safe propagation of the ECN field over tunnels, it is reasonable to add requirements on operators to ensure they configure their tunnels safely (where possible). Before resolving 'Problem 2' by stating these configuration requirements (in ), the factors that determine whether propagating ECN is feasible or desirable will be briefly introduced.
Feasibility of ECN Propagation between Tunnel Headers In many cases, one or more shim headers and an outer IP header are always added to (or removed from) an inner IP packet as part of the same procedure. We call these tightly coupled shim headers. Processing a shim and outer header together is often necessary because a shim is not sufficient for packet forwarding in its own right; not unless complemented by an outer header. In these cases, it will often be feasible for an implementation to propagate the ECN field between the IP headers. In some cases, a tunnel adds an outer IP header and a tightly coupled shim header to an inner header that is not an IP header, but that, in turn, encapsulates an IP header (or might encapsulate an IP header). For instance, an inner Ethernet (or other link-layer) header might encapsulate an inner IP header as its payload. We call this a tightly coupled shim over an encapsulating header. Digging to arbitrary depths to find an inner IP header within an encapsulation is strictly a layering violation, so it cannot be a required behaviour. Nonetheless, some tunnel endpoints already look within a Layer 2 (L2) header for an IP header, for instance, to map the Diffserv codepoint between an encapsulated IP header and an outer IP header . In such cases at least, it should be feasible to also (independently) propagate the ECN field between the same IP headers. Thus, access to the ECN field within an encapsulating header can be a useful and benign optimization. The guidelines in give the conditions for this layering violation to be benign.
Desirability of ECN Propagation between Tunnel Headers Developers and network operators are encouraged to implement and deploy tunnel endpoints compliant with (as updated by the present specification) in order to provide the benefits of wider ECN deployment . Nonetheless, propagation of ECN between IP headers, whether separated by shim headers or not, has to be optional to implement and to use, because:
  • legacy implementations of tunnels without any ECN support already exist;
  • a network might be designed so that there is usually no bottleneck within the tunnel; and
  • if the tunnel endpoints would have to search within an L2 header to find an encapsulated IP header, it might not be worth the potential performance hit.
Making a Non-ECN Tunnel Ingress Safe by Configuration Even when no specific attempt has been made to implement propagation of the ECN field at a tunnel ingress, it ought to be possible for the operator to render a tunnel ingress safe by configuration. The main safety concern is to disable (clear to zero) the ECN capability in the outer IP header at the ingress if the egress of the tunnel does not implement ECN logic to propagate any ECN markings into the packet forwarded beyond the tunnel. Otherwise, the non-ECN egress could discard any ECN marking introduced within the tunnel, which would break all the ECN-based control loops that regulate the traffic load over the tunnel. Therefore, this specification updates by inserting the following text at the end of the section:
Whether or not an ingress implementation claims compliance with , , or , when the outer tunnel header is IP (v4 or v6), if possible, the ingress MUST be configured to zero the outer ECN field in all of the following cases:
  • if it is known that the tunnel egress does not support any of the RFCs that define propagation of the ECN field (, , or the full functionality mode of );
  • if the behaviour of the egress is not known or an egress with unknown behaviour might be dynamically paired with the ingress (one way for an operator of a tunnel ingress to determine the behaviour of an otherwise unknown egress is described in );
  • if an IP header might be encapsulated within a non-IP header that the tunnel ingress is encapsulating, but the ingress does not inspect within the encapsulation.
For the avoidance of doubt, the above only concerns the outer IP header. The ingress MUST NOT alter the ECN field of the arriving IP header that will become the inner IP header. In order that the network operator can comply with the above safety rules, an implementation of a tunnel ingress:
  • MUST NOT treat the former Type of Service (ToS) octet (IPv4) or the former Traffic Class octet (IPv6) as a single 8-bit field. This is because the resulting linkage of ECN and Diffserv field propagation between inner and outer headers is not consistent with the definition of the 6-bit Diffserv field in and .
  • SHOULD be able to be configured to zero the ECN field of the outer header.
These last two rules apply even if an implementation of a tunnel ingress does not claim to support , , or the full functionality mode of
For instance, if a tunnel ingress with no ECN-specific logic had a configuration capability to refer to the last 2 bits of the old ToS Byte of the outer (e.g., with a 0x3 mask) and set them to zero, while also being able to allow the DSCP to be re-mapped independently, that would be sufficient to satisfy both implementation requirements above. There might be concern that the above "MUST NOT" makes compliant implementations non-compliant at a stroke. However, by definition, it solely applies to equipment that provides Diffserv configuration. Any such Diffserv equipment that is configuring treatment of the former ToS octet (IPv4) or the former Traffic Class octet (IPv6) as a single 8-bit field must have always been non-compliant with the definition of the 6-bit Diffserv field in and . If a tunnel ingress does not have any ECN logic, copying the ECN field as a side effect of copying the DSCP is a seriously unsafe bug that risks breaking the feedback loops that regulate load on a tunnel, because it omits to check the ECN capability of the tunnel egress. Zeroing the outer ECN field of all packets in all circumstances would be safe, but it would not be sufficient to claim compliance with because it would not meet the aim of introducing ECN support to tunnels (see ).
ECN Propagation and Fragmentation/Reassembly The following requirements update , which omitted handling of the ECN field during fragmentation or reassembly. These changes might alter how many ECN-marked packets are propagated by a tunnel that fragments packets, but this would not raise any backward compatibility issues. If a tunnel ingress fragments a packet, it MUST set the outer ECN field of all the fragments to the same value as it would have set if it had not fragmented the packet. specifies ECN requirements for reassembly of sets of 'outer fragments' into packets (in 'outer fragmentation', the fragmentation is visible in the outer header so that the tunnel egress can reassemble the fragments ). Additionally, the following requirements apply at a tunnel egress:
  • During reassembly of outer fragments, the packet MUST be discarded if the ECN fields of the outer headers being reassembled into a single packet consist of a mixture of Not ECN-Capable Transport (Not-ECT) and other ECN codepoints.
  • If there is mix of ECT(0) and ECT(1) outer fragments, then the reassembled packet MUST be set to ECT(1).
Reasoning: originally defined ECT(0) and ECT(1) as equivalent, but has been updated by to make ECT(1) available for congestion marking differences. The rule is independent of the current experimental use of ECT(1) for Low Latency, Low Loss, and Scalable throughput (L4S) . The rule is compatible with Pre-Congestion Notification (PCN) , which uses 2 levels of congestion severity, with the ranking of severity from highest to lowest being Congestion Experienced (CE), ECT(1), ECT(0). The decapsulation rules in take a similar approach.
IP-in-IP Tunnels with Tightly Coupled Shim Headers Below is a list of specifications of encapsulations with tightly coupled shim header(s) in rough chronological order. This list is confined to Standards Track or widely deployed protocols. So, for the avoidance of doubt, the updated scope of is defined in and is not limited to this list.
  • Point-to-Point Tunneling Protocol (PPTP)
  • Layer Two Tunneling Protocol (L2TP), specifically L2TPv2 and L2TPv3 , which not only includes all the L2-specific specializations of L2TP, but also derivatives such as the Keyed IPv6 Tunnel
  • Generic Routing Encapsulation (GRE) and Network Virtualization using GRE (NVGRE)
  • GPRS Tunnelling Protocol (GTP), specifically GTPv1 , GTP v1 User Plane , and GTP v2 Control Plane
  • Teredo
  • Control And Provisioning of Wireless Access Points (CAPWAP)
  • Locator/Identifier Separation Protocol (LISP)
  • Automatic Multicast Tunneling (AMT)
  • Virtual eXtensible Local Area Network (VXLAN) and Generic Protocol Extensions for VXLAN (VXLAN-GPE)
  • The Network Service Header (NSH) for Service Function Chaining (SFC)
  • Geneve
  • Direct tunnelling of an IP packet within a UDP/IP datagram (see )
  • TCP Encapsulation of Internet Key Exchange Protocol (IKE) and IPsec Packets (see )
Some of the listed protocols enable encapsulation of a variety of network layer protocols as inner and/or outer. This specification applies to the cases where there is an inner and outer IP header as described in . Otherwise, gives guidance on how to design propagation of ECN into other protocols that might encapsulate IP. Where protocols in the above list need to be updated to specify ECN propagation and are under IETF change control, update text is given in the following subsections. For those not under IETF control, it is RECOMMENDED that implementations of encapsulation and decapsulation comply with . It is also RECOMMENDED that their specifications are updated to add a requirement to comply with (as updated by the present document). PPTP is not under the change control of the IETF, but it has been documented in an Informational RFC . However, there is no need for the present specification to update PPTP because L2TP has been developed as a standardized replacement. NVGRE is not under the change control of the IETF, but it has been documented in an Informational RFC . NVGRE is a specific use case of GRE (it re-purposes the key field from the initial specification of GRE as a Virtual Subnet ID). Therefore, the text that updates GRE in below is also intended to update NVGRE. Although the definition of the various GTP shim headers is under the control of the Third Generation Partnership Project (3GPP), it is hard to determine whether the 3GPP or the IETF controls standardization of the process of adding both a GTP and an IP header to an inner IP header. Nonetheless, the present specification is provided so that the 3GPP can refer to it from any of its own specifications of GTP and IP header processing. The specification of CAPWAP already specifies ECN propagation and ECN capability negotiation. Without modification, the CAPWAP specification already interworks with the backward-compatible updates to in . LISP made the ECN propagation procedures in mandatory from the start. has since been updated by , but the changes are backwards compatible, so there is still no need for LISP tunnel endpoints to negotiate their ECN capabilities. VXLAN is not under the change control of the IETF, but it has been documented in an Informational RFC. It is RECOMMENDED that VXLAN implementations comply with when the VXLAN header is inserted between (or removed from between) IP headers. The authors of any future update of the VXLAN spec are also encouraged to add a requirement to comply with as updated by the present specification. In contrast, VXLAN-GPE is being documented under IETF change control and it does require compliance with . The Network Service Header (NSH) has been defined as a shim-based encapsulation to identify the Service Function Path (SFP) in the Service Function Chaining (SFC) architecture . A proposal has been made for the processing of ECN when handling transport encapsulation . The specification of Geneve already refers to for ECN encapsulation. already explains that a tunnel that encapsulates an IP header within a UDP/IP datagram needs to follow when propagating the ECN field between inner and outer IP headers. of the present specification updates to clarify that its scope includes cases with a shim header between the IP headers. So it indirectly updates the scope of to include cases with a shim header as well as a UDP header between the IP headers. The requirements in update , and hence also indirectly update the UDP usage guidelines in to add the important but previously unstated requirement that, if the UDP tunnel egress does not, or might not, support ECN propagation, a UDP tunnel ingress has to clear the outer IP ECN field to 0b00, e.g., by configuration. already recommends the compatibility mode of in this case because there is not a one-to-one mapping between inner and outer packets when TCP encapsulates IKE or IPsec.
Specific Updates to Protocols under IETF Change Control
L2TP (v2 and v3) ECN Extension The L2TP terminology used here is defined in and . L2TPv3 is used as a shim header between any packet-switched network (PSN) header (e.g., IPv4, IPv6, and MPLS) and many types of L2 headers. The L2TPv3 shim header encapsulates an L2-specific sub-layer, then an L2 header that is likely to contain an inner IP header (v4 or v6). Then this whole stack of headers can be encapsulated within an optional outer UDP header and an outer PSN header that is typically IP (v4 or v6). L2TPv2 is used as a shim header between any PSN header and a PPP header, which is in turn likely to encapsulate an IP header. Even though these shims are rather fat (particularly in the case of L2TPv3), they still fit the definition of a tightly coupled shim header over an encapsulating header () because all the headers encapsulating the L2 header are added (or removed) together. L2TPv2 and L2TPv3 are therefore within the scope of , as updated by . Implementation of the ECN extension to L2TPv2 and L2TPv3 defined in is RECOMMENDED in order to provide the benefits of ECN whenever a node within an L2TP tunnel becomes the bottleneck for an end-to-end traffic flow.
Safe Configuration of a "Non-ECN" Ingress LCCE The following text is appended to both and as an update to the base L2TPv2 and L2TPv3 specifications:
The operator of an LCCE that does not support the ECN extension in of RFC 9601 MUST follow the configuration requirements in of RFC 9601 to ensure it clears the outer IP ECN field to 0b00 when the outer PSN header is IP (v4 or v6).
In particular, for an L2TP Control Connection Endpoint (LCCE) implementation that does not support the ECN extension, this means that configuration of how it propagates the ECN field between inner and outer IP headers MUST be independent of any configuration of the Diffserv extension of L2TP .
ECN Extension for L2TP (v2 or v3) When the outer PSN header and the payload inside the L2 header are both IP (v4 or v6), an LCCE will propagate the ECN field at ingress and egress by following the rules in . Before encapsulating any data packets, requires an ingress LCCE to check that the egress LCCE supports ECN propagation as defined in or one of its compatible predecessors ( or the full functionality mode of ). If the egress supports ECN propagation, the ingress LCCE can use the normal mode of encapsulation (copying the ECN field from inner to outer). Otherwise, the ingress LCCE has to use compatibility mode (clearing the outer IP ECN field to 0b00). An LCCE can determine the remote LCCE's support for ECN either statically (by configuration) or by dynamic discovery during setup of each control connection between the LCCEs using the ECN Capability Attribute-Value Pair (AVP) defined in . Where the outer PSN header is some protocol other than IP that supports ECN, the appropriate ECN propagation specification will need to be followed, e.g., for MPLS. Where no specification exists for ECN propagation by a particular PSN, gives general guidance on how to design ECN propagation into a protocol that encapsulates IP.
ECN Capability AVP for Negotiation between LCCEs The ECN Capability AVP defined here has Attribute Type 103. The AVP has the following format:
ECN Capability AVP for L2TP (v2 or v3) 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |M|H|0|0|0|0| Length | Vendor ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 103 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
This AVP MAY be present in the Start-Control-Connection-Request (SCCRQ) and Start-Control-Connection-Reply (SCCRP) message types. This AVP MAY be hidden (the H-bit is set to 0 or 1) and is optional (the M-bit is not set). The length (before hiding) of this AVP is 6 octets. The Vendor ID is the IETF Vendor ID of 0. When an LCCE sends an ECN Capability AVP, it indicates that it supports ECN propagation. When no ECN Capability AVP is present, it indicates that the sender does not support ECN propagation. If an LCCE initiating a control connection supports ECN propagation, it will send an SCCRQ containing an ECN Capability AVP. If the tunnel terminator supports ECN, it will return an SCCRP that also includes an ECN Capability AVP. Then, for any sessions created by that control connection, both ends of the tunnel can use the normal mode of ; i.e., they can copy the IP ECN field from inner to outer when encapsulating data packets. On the other hand, if the tunnel terminator does not support ECN, it will ignore the ECN Capability AVP and send an SCCRP to the tunnel initiator without an ECN Capability AVP. The tunnel initiator interprets the absence of the ECN Capability flag in the SCCRP as an indication that the tunnel terminator is incapable of supporting ECN. When encapsulating data packets for any sessions created by that control connection, the tunnel initiator will then use the compatibility mode of to clear the ECN field of the outer IP header to 0b00. If the tunnel terminator does not support this ECN extension, the network operator is still expected to configure it to comply with the safety provisions set out in when it acts as an ingress LCCE. If ECN support by the ingress and egress LCCEs is configured statically, as allowed in , they both ignore the presence or absence of any ECN capability AVP.
GRE The GRE terminology used here is defined in . GRE is often used as a tightly coupled shim header between IP headers. Sometimes, the GRE shim header encapsulates an L2 header, which might in turn encapsulate an IP header. Therefore, GRE is within the scope of as updated by . Implementation of support for as updated by the present specification is RECOMMENDED for GRE tunnel endpoints in order to provide the benefits of ECN whenever a node within a GRE tunnel becomes the bottleneck for an end-to-end IP traffic flow tunnelled over GRE using IP as the delivery protocol (outer header). GRE itself does not support dynamic setup and configuration of tunnels. However, control plane protocols, such as Next Hop Resolution Protocol (NHRP) , Mobile IPv4 (MIP4) , Mobile IPv6 (MIP6) , Proxy Mobile IP (PMIP) , and IKEv2 , are sometimes used to set up GRE tunnels dynamically. When these control protocols set up IP-in-IP or IPsec tunnels, it is likely that the resulting tunnels will propagate the ECN field as defined in or one of its compatible predecessors ( or the full functionality mode of ). However, if they use a GRE encapsulation, this presumption is less sound. Therefore, if the outer delivery protocol is IP (v4 or v6), the operator is obliged to follow the safe configuration requirements in . updates the base GRE specification with this requirement to emphasize its importance. Where the delivery protocol is some protocol other than IP that supports ECN, the appropriate ECN propagation specification will need to be followed, e.g., for MPLS. Where no specification exists for ECN propagation by a particular PSN, gives more general guidance on how to propagate ECN to and from protocols that encapsulate IP.
Safe Configuration of a "Non-ECN" GRE Ingress The following text is appended to as an update to the base GRE specification:
The operator of a GRE tunnel ingress MUST follow the configuration requirements in of RFC 9601 when the outer delivery protocol is IP (v4 or v6).
Teredo Teredo provides a way to tunnel IPv6 over an IPv4 network with a UDP-based shim header between the two. For Teredo tunnel endpoints to provide the benefits of ECN, the Teredo specification would have to be updated to include negotiation of the ECN capability between Teredo tunnel endpoints. Otherwise, it would be unsafe for a Teredo tunnel ingress to copy the ECN field to the IPv6 outer. Those implementations known to the authors at the time of writing do not support propagation of ECN, but they do safely zero the ECN field in the outer IPv6 header. However, the specification does not mention anything about this. To make existing Teredo deployments safe, it would be possible to add ECN capability negotiation to those that are subject to remote OS update. However, for those implementations not subject to remote OS update, it will not be feasible to require them to be configured correctly because Teredo tunnel endpoints are generally deployed on hosts. Therefore, until ECN support is added to the specification of Teredo, the only feasible further safety precaution available here is to update the specification of Teredo implementations with the following text as a new section:
5.1.3. Safe "Non-ECN" Teredo Encapsulation A Teredo tunnel ingress implementation that does not support ECN propagation as defined in or one of its compatible predecessors ( or the full functionality mode of ) MUST zero the ECN field in the outer IPv6 header.
AMT AMT is a tightly coupled shim header that encapsulates an IP packet and is encapsulated within a UDP/IP datagram. Therefore, AMT is within the scope of as updated by . Implementation of support for as updated by the present specification is RECOMMENDED for AMT tunnel endpoints in order to provide the benefits of ECN whenever a node within an AMT tunnel becomes the bottleneck for an IP traffic flow tunnelled over AMT. To comply with , an AMT relay and gateway will follow the rules for propagation of the ECN field at ingress and egress, respectively, as described in . Before encapsulating any data packets, requires an ingress AMT relay to check that the egress AMT gateway supports ECN propagation as defined in or one of its compatible predecessors ( or the full functionality mode of ). If the egress gateway supports ECN, the ingress relay can use the normal mode of encapsulation (copying the IP ECN field from inner to outer). Otherwise, the ingress relay has to use compatibility mode, which means it has to clear the outer ECN field to zero . An AMT tunnel is created dynamically (not manually), so the relay will need to determine the remote gateway's support for ECN using the ECN capability declaration defined in .
Safe Configuration of a "Non-ECN" Ingress AMT Relay The following text is appended to as an update to the AMT specification:
The operator of an AMT relay that does not support or one of its compatible predecessors ( or the full functionality mode of ) MUST follow the configuration requirements in of RFC 9601 to ensure it clears the outer IP ECN field to zero.
ECN Capability Declaration of an AMT Gateway
Updated AMT Request Message Format 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | V=0 |Type=3 | Reserved |E|P| Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Request Nonce | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Bit 14 of the AMT Request Message counting from 0 (or bit 7 of the Reserved field counting from 1) is defined here as the AMT Gateway ECN Capability flag (E) as shown in . The definitions of all other fields in the AMT Request Message are unchanged from . When the E flag is set to 1, it indicates that the sender of the message supports ECN propagation. When it is cleared to zero, it indicates the sender of the message does not support ECN propagation. An AMT gateway "that supports ECN propagation" means one that propagates the ECN field to the forwarded data packet based on the combination of arriving inner and outer ECN fields as defined in . The other bits of the Reserved field remain reserved. They will continue to be cleared to zero when sent and ignored when either received or forwarded as specified in . An AMT gateway that does not support MUST NOT set the E flag of its Request Message to 1. An AMT gateway that supports ECN propagation MUST set the E flag of its Relay Discovery Message to 1. The action of the corresponding AMT relay that receives a Request message with the E flag set to 1 depends on whether the relay itself supports ECN propagation:
  • If the relay supports ECN propagation, it will store the ECN capability of the gateway along with its address. Then, whenever it tunnels datagrams towards this gateway, it MUST use the normal mode of to propagate the ECN field when encapsulating datagrams (i.e., it copies the IP ECN field from inner to outer header).
  • If the discovered AMT relay does not support ECN propagation, it will ignore the E flag in the Reserved field as per . If the AMT relay does not support ECN propagation, the network operator is still expected to configure it to comply with the safety provisions set out in .
IANA Considerations IANA has assigned the following AVP in the L2TP "Control Message Attribute Value Pairs" registry:
Attribute Type Description Reference
103 ECN Capability RFC 9601
Security Considerations The Security Considerations in and apply equally to the scope defined for the present specification.
References Normative References Key words for use in RFCs to Indicate Requirement Levels In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers This document defines the IP header field, called the DS (for differentiated services) field. [STANDARDS-TRACK] Layer Two Tunneling Protocol "L2TP" This document describes the Layer Two Tunneling Protocol (L2TP). [STANDARDS-TRACK] Generic Routing Encapsulation (GRE) This document specifies a protocol for encapsulation of an arbitrary network layer protocol over another arbitrary network layer protocol. [STANDARDS-TRACK] The Addition of Explicit Congestion Notification (ECN) to IP This memo specifies the incorporation of ECN (Explicit Congestion Notification) to TCP and IP, including ECN's use of two bits in the IP header. [STANDARDS-TRACK] Layer Two Tunneling Protocol - Version 3 (L2TPv3) This document describes "version 3" of the Layer Two Tunneling Protocol (L2TPv3). L2TPv3 defines the base control protocol and encapsulation for tunneling multiple Layer 2 connections between two IP nodes. Additional documents detail the specifics for each data link type being emulated. [STANDARDS-TRACK] Security Architecture for the Internet Protocol This document describes an updated version of the "Security Architecture for IP", which is designed to provide security services for traffic at the IP layer. This document obsoletes RFC 2401 (November 1998). [STANDARDS-TRACK] Teredo: Tunneling IPv6 over UDP through Network Address Translations (NATs) We propose here a service that enables nodes located behind one or more IPv4 Network Address Translations (NATs) to obtain IPv6 connectivity by tunneling packets over UDP; we call this the Teredo service. Running the service requires the help of "Teredo servers" and "Teredo relays". The Teredo servers are stateless, and only have to manage a small fraction of the traffic between Teredo clients; the Teredo relays act as IPv6 routers between the Teredo service and the "native" IPv6 Internet. The relays can also provide interoperability with hosts using other transition mechanisms such as "6to4". [STANDARDS-TRACK] Explicit Congestion Marking in MPLS RFC 3270 defines how to support the Diffserv architecture in MPLS networks, including how to encode Diffserv Code Points (DSCPs) in an MPLS header. DSCPs may be encoded in the EXP field, while other uses of that field are not precluded. RFC 3270 makes no statement about how Explicit Congestion Notification (ECN) marking might be encoded in the MPLS header. This document defines how an operator might define some of the EXP codepoints for explicit congestion notification, without precluding other uses. [STANDARDS-TRACK] Tunnelling of Explicit Congestion Notification This document redefines how the explicit congestion notification (ECN) field of the IP header should be constructed on entry to and exit from any IP-in-IP tunnel. On encapsulation, it updates RFC 3168 to bring all IP-in-IP tunnels (v4 or v6) into line with RFC 4301 IPsec ECN processing. On decapsulation, it updates both RFC 3168 and RFC 4301 to add new behaviours for previously unused combinations of inner and outer headers. The new rules ensure the ECN field is correctly propagated across a tunnel whether it is used to signal one or two severity levels of congestion; whereas before, only one severity level was supported. Tunnel endpoints can be updated in any order without affecting pre-existing uses of the ECN field, thus ensuring backward compatibility. Nonetheless, operators wanting to support two severity levels (e.g., for pre-congestion notification -- PCN) can require compliance with this new specification. A thorough analysis of the reasoning for these changes and the implications is included. In the unlikely event that the new rules do not meet a specific need, RFC 4774 gives guidance on designing alternate ECN semantics, and this document extends that to include tunnelling issues. [STANDARDS-TRACK] Encoding Three Pre-Congestion Notification (PCN) States in the IP Header Using a Single Diffserv Codepoint (DSCP) The objective of Pre-Congestion Notification (PCN) is to protect the quality of service (QoS) of inelastic flows within a Diffserv domain. The overall rate of PCN-traffic is metered on every link in the PCN- domain, and PCN-packets are appropriately marked when certain configured rates are exceeded. Egress nodes pass information about these PCN-marks to Decision Points that then decide whether to admit or block new flow requests or to terminate some already admitted flows during serious pre-congestion. This document specifies how PCN-marks are to be encoded into the IP header by reusing the Explicit Congestion Notification (ECN) codepoints within a PCN-domain. The PCN wire protocol for non-IP protocol headers will need to be defined elsewhere. Nonetheless, this document clarifies the PCN encoding for MPLS in an informational appendix. The encoding for IP provides for up to three different PCN marking states using a single Diffserv codepoint (DSCP): not-marked (NM), threshold-marked (ThM), and excess-traffic-marked (ETM). Hence, it is called the 3-in-1 PCN encoding. This document obsoletes RFC 5696. [STANDARDS-TRACK] Automatic Multicast Tunneling This document describes Automatic Multicast Tunneling (AMT), a protocol for delivering multicast traffic from sources in a multicast-enabled network to receivers that lack multicast connectivity to the source network. The protocol uses UDP encapsulation and unicast replication to provide this functionality. The AMT protocol is specifically designed to support rapid deployment by requiring minimal changes to existing network infrastructure. Guidelines for Adding Congestion Notification to Protocols that Encapsulate IP Informative References A Test for IP-ECN Propagation by a Remote Tunnel Endpoint Independent Technical Report, TR-BB-2023-003 General Packet Radio Service (GPRS); GPRS Tunnelling Protocol (GTP) across the Gn and Gp interface 3GPP General Packet Radio System (GPRS) Tunnelling Protocol User Plane (GTPv1-U) 3GPP 3GPP Evolved Packet System (EPS); Evolved General Packet Radio Service (GPRS) Tunnelling Protocol for Control plane (GTPv2-C); Stage 3 3GPP IP Tunnels in the Internet Architecture Independent Consultant Cisco This document discusses the role of IP tunnels in the Internet architecture. An IP tunnel transits IP datagrams as payloads in non- link layer protocols. This document explains the relationship of IP tunnels to existing protocol layers and the challenges in supporting IP tunneling, based on the equivalence of tunnels to links. The implications of this document updates RFC 4459 and its MTU and fragmentation recommendations for IP tunnels. Work in Progress Generic Protocol Extension for VXLAN (VXLAN-GPE) Cisco Systems Arrcus Intel This document describes extending Virtual eXtensible Local Area Network (VXLAN), via changes to the VXLAN header, with four new capabilities: support for multi-protocol encapsulation, support for operations, administration and maintenance (OAM) signaling, support for ingress-replicated BUM Traffic (i.e. Broadcast, Unknown unicast, or Multicast), and explicit versioning. New protocol capabilities can be introduced via shim headers. Work in Progress Generic Routing Encapsulation (GRE) This document specifies a protocol for performing encapsulation of an arbitrary network layer protocol over another arbitrary network layer protocol. This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind. NBMA Next Hop Resolution Protocol (NHRP) This document describes the NBMA Next Hop Resolution Protocol (NHRP). NHRP can be used by a source station (host or router) connected to a Non-Broadcast, Multi-Access (NBMA) subnetwork to determine the internetworking layer address and NBMA subnetwork addresses of the "NBMA next hop" towards a destination station. [STANDARDS-TRACK] Point-to-Point Tunneling Protocol (PPTP) This document specifies a protocol which allows the Point to Point Protocol (PPP) to be tunneled through an IP network. This memo provides information for the Internet community. Differentiated Services and Tunnels This document considers the interaction of Differentiated Services (diffserv) with IP tunnels of various forms. This memo provides information for the Internet community. New Terminology and Clarifications for Diffserv This memo captures Diffserv working group agreements concerning new and improved terminology, and provides minor technical clarifications. It is intended to update RFC 2474, RFC 2475 and RFC 2597. When RFCs 2474 and 2597 advance on the standards track, and RFC 2475 is updated, it is intended that the revisions in this memo will be incorporated, and that this memo will be obsoleted by the new RFCs. This memo provides information for the Internet community. Layer Two Tunneling Protocol (L2TP) Differentiated Services Extension Control And Provisioning of Wireless Access Points (CAPWAP) Protocol Specification This specification defines the Control And Provisioning of Wireless Access Points (CAPWAP) Protocol, meeting the objectives defined by the CAPWAP Working Group in RFC 4564. The CAPWAP protocol is designed to be flexible, allowing it to be used for a variety of wireless technologies. This document describes the base CAPWAP protocol, while separate binding extensions will enable its use with additional wireless technologies. [STANDARDS-TRACK] Generic Routing Encapsulation (GRE) Key Option for Proxy Mobile IPv6 This specification defines a new mobility option for allowing the mobile access gateway and the local mobility anchor to negotiate Generic Routing Encapsulation (GRE) encapsulation mode and exchange the downlink and uplink GRE keys that are used for marking the downlink and uplink traffic that belong to a specific mobility session. In addition, the same mobility option can be used to negotiate the GRE encapsulation mode without exchanging the GRE keys. [STANDARDS-TRACK] IP Mobility Support for IPv4, Revised This document specifies protocol enhancements that allow transparent routing of IP datagrams to mobile nodes in the Internet. Each mobile node is always identified by its home address, regardless of its current point of attachment to the Internet. While situated away from its home, a mobile node is also associated with a care-of address, which provides information about its current point of attachment to the Internet. The protocol provides for registering the care-of address with a home agent. The home agent sends datagrams destined for the mobile node through a tunnel to the care-of address. After arriving at the end of the tunnel, each datagram is then delivered to the mobile node. [STANDARDS-TRACK] Mobility Support in IPv6 This document specifies Mobile IPv6, a protocol that allows nodes to remain reachable while moving around in the IPv6 Internet. Each mobile node is always identified by its home address, regardless of its current point of attachment to the Internet. While situated away from its home, a mobile node is also associated with a care-of address, which provides information about the mobile node's current location. IPv6 packets addressed to a mobile node's home address are transparently routed to its care-of address. The protocol enables IPv6 nodes to cache the binding of a mobile node's home address with its care-of address, and to then send any packets destined for the mobile node directly to it at this care-of address. To support this operation, Mobile IPv6 defines a new IPv6 protocol and a new destination option. All IPv6 nodes, whether mobile or stationary, can communicate with mobile nodes. This document obsoletes RFC 3775. [STANDARDS-TRACK] A Comparison of IPv6-over-IPv4 Tunnel Mechanisms This document provides an overview of various ways to tunnel IPv6 packets over IPv4 networks. It covers mechanisms in current use, touches on several mechanisms that are now only of historic interest, and discusses some newer tunnel mechanisms that are not widely used at the time of publication. The goal of the document is helping people with an IPv6-in-IPv4 tunneling need to select the mechanisms that may apply to them. Internet Key Exchange Protocol Version 2 (IKEv2) This document describes version 2 of the Internet Key Exchange (IKE) protocol. IKE is a component of IPsec used for performing mutual authentication and establishing and maintaining Security Associations (SAs). This document obsoletes RFC 5996, and includes all of the errata for it. It advances IKEv2 to be an Internet Standard. Virtual eXtensible Local Area Network (VXLAN): A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks This document describes Virtual eXtensible Local Area Network (VXLAN), which is used to address the need for overlay networks within virtualized data centers accommodating multiple tenants. The scheme and the related protocols can be used in networks for cloud service providers and enterprise data centers. This memo documents the deployed VXLAN protocol for the benefit of the Internet community. NVGRE: Network Virtualization Using Generic Routing Encapsulation This document describes the usage of the Generic Routing Encapsulation (GRE) header for Network Virtualization (NVGRE) in multi-tenant data centers. Network Virtualization decouples virtual networks and addresses from physical network infrastructure, providing isolation and concurrency between multiple virtual networks on the same physical network infrastructure. This document also introduces a Network Virtualization framework to illustrate the use cases, but the focus is on specifying the data-plane aspect of NVGRE. Service Function Chaining (SFC) Architecture This document describes an architecture for the specification, creation, and ongoing maintenance of Service Function Chains (SFCs) in a network. It includes architectural concepts, principles, and components used in the construction of composite services through deployment of SFCs, with a focus on those to be standardized in the IETF. This document does not propose solutions, protocols, or extensions to existing protocols. UDP Usage Guidelines The User Datagram Protocol (UDP) provides a minimal message-passing transport that has no inherent congestion control mechanisms. This document provides guidelines on the use of UDP for the designers of applications, tunnels, and other protocols that use UDP. Congestion control guidelines are a primary focus, but the document also provides guidance on other topics, including message sizes, reliability, checksums, middlebox traversal, the use of Explicit Congestion Notification (ECN), Differentiated Services Code Points (DSCPs), and ports. Because congestion control is critical to the stable operation of the Internet, applications and other protocols that choose to use UDP as an Internet transport must employ mechanisms to prevent congestion collapse and to establish some degree of fairness with concurrent traffic. They may also need to implement additional mechanisms, depending on how they use UDP. Some guidance is also applicable to the design of other protocols (e.g., protocols layered directly on IP or via IP-based tunnels), especially when these protocols do not themselves provide congestion control. This document obsoletes RFC 5405 and adds guidelines for multicast UDP usage. The Benefits of Using Explicit Congestion Notification (ECN) The goal of this document is to describe the potential benefits of applications using a transport that enables Explicit Congestion Notification (ECN). The document outlines the principal gains in terms of increased throughput, reduced delay, and other benefits when ECN is used over a network path that includes equipment that supports Congestion Experienced (CE) marking. It also discusses challenges for successful deployment of ECN. It does not propose new algorithms to use ECN nor does it describe the details of implementation of ECN in endpoint devices (Internet hosts), routers, or other network devices. Keyed IPv6 Tunnel This document describes a tunnel encapsulation for Ethernet over IPv6 with a mandatory 64-bit cookie for connecting Layer 2 (L2) Ethernet attachment circuits identified by IPv6 addresses. The encapsulation is based on the Layer 2 Tunneling Protocol Version 3 (L2TPv3) over IP and does not use the L2TPv3 control plane. Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. Network Service Header (NSH) This document describes a Network Service Header (NSH) imposed on packets or frames to realize Service Function Paths (SFPs). The NSH also provides a mechanism for metadata exchange along the instantiated service paths. The NSH is the Service Function Chaining (SFC) encapsulation required to support the SFC architecture (defined in RFC 7665). Relaxing Restrictions on Explicit Congestion Notification (ECN) Experimentation This memo updates RFC 3168, which specifies Explicit Congestion Notification (ECN) as an alternative to packet drops for indicating network congestion to endpoints. It relaxes restrictions in RFC 3168 that hinder experimentation towards benefits beyond just removal of loss. This memo summarizes the anticipated areas of experimentation and updates RFC 3168 to enable experimentation in these areas. An Experimental RFC in the IETF document stream is required to take advantage of any of these enabling updates. In addition, this memo makes related updates to the ECN specifications for RTP in RFC 6679 and for the Datagram Congestion Control Protocol (DCCP) in RFCs 4341, 4342, and 5622. This memo also records the conclusion of the ECN nonce experiment in RFC 3540 and provides the rationale for reclassification of RFC 3540 from Experimental to Historic; this reclassification enables new experimental use of the ECT(1) codepoint. Geneve: Generic Network Virtualization Encapsulation Network virtualization involves the cooperation of devices with a wide variety of capabilities such as software and hardware tunnel endpoints, transit fabrics, and centralized control clusters. As a result of their role in tying together different elements of the system, the requirements on tunnels are influenced by all of these components. Therefore, flexibility is the most important aspect of a tunneling protocol if it is to keep pace with the evolution of technology. This document describes Geneve, an encapsulation protocol designed to recognize and accommodate these changing capabilities and needs. The Locator/ID Separation Protocol (LISP) This document describes the data plane protocol for the Locator/ID Separation Protocol (LISP). LISP defines two namespaces: Endpoint Identifiers (EIDs), which identify end hosts; and Routing Locators (RLOCs), which identify network attachment points. With this, LISP effectively separates control from data and allows routers to create overlay networks. LISP-capable routers exchange encapsulated packets according to EID-to-RLOC mappings stored in a local Map-Cache. LISP requires no change to either host protocol stacks or underlay routers and offers Traffic Engineering (TE), multihoming, and mobility, among other features. This document obsoletes RFC 6830. TCP Encapsulation of Internet Key Exchange Protocol (IKE) and IPsec Packets This document describes a method to transport Internet Key Exchange Protocol (IKE) and IPsec packets over a TCP connection for traversing network middleboxes that may block IKE negotiation over UDP. This method, referred to as "TCP encapsulation", involves sending both IKE packets for Security Association (SA) establishment and Encapsulating Security Payload (ESP) packets over a TCP connection. This method is intended to be used as a fallback option when IKE cannot be negotiated over UDP. TCP encapsulation for IKE and IPsec was defined in RFC 8229. This document clarifies the specification for TCP encapsulation by including additional clarifications obtained during implementation and deployment of this method. This documents obsoletes RFC 8229. The Explicit Congestion Notification (ECN) Protocol for Low Latency, Low Loss, and Scalable Throughput (L4S) This specification defines the protocol to be used for a new network service called Low Latency, Low Loss, and Scalable throughput (L4S). L4S uses an Explicit Congestion Notification (ECN) scheme at the IP layer that is similar to the original (or 'Classic') ECN approach, except as specified within. L4S uses 'Scalable' congestion control, which induces much more frequent control signals from the network, and it responds to them with much more fine-grained adjustments so that very low (typically sub-millisecond on average) and consistently low queuing delay becomes possible for L4S traffic without compromising link utilization. Thus, even capacity-seeking (TCP-like) traffic can have high bandwidth and very low delay at the same time, even during periods of high traffic load. The L4S identifier defined in this document distinguishes L4S from 'Classic' (e.g., TCP-Reno-friendly) traffic. Then, network bottlenecks can be incrementally modified to distinguish and isolate existing traffic that still follows the Classic behaviour, to prevent it from degrading the low queuing delay and low loss of L4S traffic. This Experimental specification defines the rules that L4S transports and network elements need to follow, with the intention that L4S flows neither harm each other's performance nor that of Classic traffic. It also suggests open questions to be investigated during experimentation. Examples of new Active Queue Management (AQM) marking algorithms and new transports (whether TCP-like or real time) are specified separately. Explicit Congestion Notification (ECN) and Congestion Feedback Using the Network Service Header (NSH) and IPFIX Independent Huawei Technologies Malis Consulting Huawei Technologies Work in Progress
Acknowledgements Thanks to for initial discussions on the need for ECN propagation in L2TP and its applicability. Thanks also to , , , , , , , , , , , and for helpful advice and comments. helped to identify a number of tunnelling protocols to include within the scope of this document. was part-funded by the Research Council of Norway through the TimeIn project for early drafts, and he was funded by Apple Inc. for later draft versions (from -17). The views expressed here are solely those of the authors.
Author's Address Independent
United Kingdom ietf@bobbriscoe.net https://bobbriscoe.net/