Ericsson

Magyar Tudosok krt. 11. Budapest Hungary 1117 balazs.a.varga@ericsson.com

Ericsson

Magyar Tudosok krt. 11. Budapest Hungary 1117 janos.farkas@ericsson.com

LabN Consulting, L.L.C.

lberger@labn.net

Malis Consulting

agmalis@gmail.com

Futurewei Technologies

sb@stewartbryant.com

jouni.nospam@gmail.com

DetNet This document specifies the Deterministic Networking (DetNet) data plane when operating over an MPLS Packet Switched Network. It leverages existing pseudowire (PW) encapsulations and MPLS Traffic Engineering (MPLS-TE) encapsulations and mechanisms. This document builds on the DetNet architecture and data plane framework.

Status of This Memo This is an Internet Standards Track document. This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841. Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at .

Copyright Notice Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents () in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents

• .  Introduction
• .  Terminology
  • .  Terms Used in This Document
  • .  Abbreviations
  • .  Requirements Language
• .  Overview of the DetNet MPLS Data Plane
  • .  Layers of DetNet Data Plane
  • .  DetNet MPLS Data Plane Scenarios
• .  MPLS-Based DetNet Data Plane Solution
  • .  DetNet over MPLS Encapsulation Components
  • .  MPLS Data Plane Encapsulation
    • .  DetNet Control Word and DetNet Sequence Number
    • .  S-Labels
    • .  F-Labels
  • .  OAM Indication
  • .  Flow Aggregation
    • .  Aggregation via LSP Hierarchy
    • .  Aggregating DetNet Flows as a New DetNet Flow
  • .  Service Sub-Layer Considerations
    • .  Edge Node Processing
    • .  Relay Node Processing
  • .  Forwarding Sub-Layer Considerations
    • .  Class of Service
    • .  Quality of Service
• .  Management and Control Information Summary
  • .  Service Sub-Layer Information Summary
    • .  Service Aggregation Information Summary
  • .  Forwarding Sub-Layer Information Summary
• .  Security Considerations
• .  IANA Considerations
• .  References
  • .  Normative References
  • .  Informative References
• Acknowledgements
• Contributors
• Authors' Addresses

Introduction Deterministic Networking (DetNet) is a service that can be offered by a network to DetNet flows. DetNet provides a capability for the delivery of data flows with extremely low packet loss rates and bounded end-to-end delivery latency. General background and concepts of DetNet can be found in the DetNet architecture . The purpose of this document is to describe the use of the MPLS data plane to establish and support DetNet flows. The DetNet architecture models the DetNet-related data plane functions as being decomposed into two sub-layers: a service sub-layer and a forwarding sub-layer. The service sub-layer is used to provide DetNet service functions, such as protection and reordering. At the DetNet data plane, a new set of functions (Packet Replication, Elimination and Ordering Functions (PREOF)) provide the tasks specific to the service sub-layer. The forwarding sub-layer is used to provide forwarding assurance (low loss, assured latency, and limited out-of-order delivery). The use of the functionalities of the DetNet service sub-layer and the DetNet forwarding sub-layer require careful design and control by the Controller Plane in addition to the DetNet-specific use of MPLS encapsulation as specified by this document. This document specifies the DetNet data plane operation and the on-wire encapsulation of DetNet flows over an MPLS-based Packet Switched Network (PSN) using the service reference model. MPLS encapsulation already provides a solid foundation of building blocks to enable the DetNet service and forwarding sub-layer functions. MPLS-encapsulated DetNet can be carried over a variety of different network technologies that can provide the level of service required for DetNet. However, the specific details of how DetNet MPLS is carried over different network technologies are out of scope for this document. MPLS-encapsulated DetNet flows can carry different types of traffic. The details of the types of traffic that are carried in DetNet are also out of scope for this document. An example of IP using DetNet MPLS sub-networks can be found in . DetNet MPLS may use an associated controller and Operations, Administration, and Maintenance (OAM) functions that are defined outside of this document. Background information common to all data planes for DetNet can be found in the DetNet data plane framework .

Terminology

Terms Used in This Document This document uses the terminology established in the DetNet architecture and the DetNet data plane framework . The reader is assumed to be familiar with these documents, any terminology defined therein, and basic MPLS-related terminologies in . The following terminology is introduced in this document:

F-Label

A DetNet "forwarding" label that identifies the Label Switched Path (LSP) used to forward a DetNet flow across an MPLS PSN, i.e., a hop-by-hop label used between Label Switching Routers (LSRs).

S-Label

A DetNet "service" label that is used between DetNet nodes that implement the DetNet service sub-layer functions. An S-Label is used to identify a DetNet flow at the DetNet service sub-layer at a receiving DetNet node.

A-Label

A special case of an S-Label, whose aggregation properties are known only at the aggregation and deaggregation end points.

d-CW

A DetNet Control Word (d-CW) that is used for sequencing information of a DetNet flow at the DetNet service sub-layer.

Abbreviations The following abbreviations are used in this document:

CoS

Class of Service

CW

Control Word

DetNet

Deterministic Networking

LSR

Label Switching Router

MPLS

Multiprotocol Label Switching

MPLS-TE

Multiprotocol Label Switching Traffic Engineering

MPLS-TP

Multiprotocol Label Switching Transport Profile

OAM

Operations, Administration, and Maintenance

PE

Provider Edge

PEF

Packet Elimination Function

PRF

Packet Replication Function

PREOF

Packet Replication, Elimination and Ordering Functions

POF

Packet Ordering Function

PSN

Packet Switched Network

PW

Pseudowire

QoS

Quality of Service

S-PE

Switching Provider Edge

T-PE

Terminating Provider Edge

TSN

Time-Sensitive Networking

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Overview of the DetNet MPLS Data Plane

Layers of DetNet Data Plane MPLS provides a wide range of capabilities that can be utilized by DetNet. A straight-forward approach utilizing MPLS for a DetNet service sub-layer is based on the existing pseudowire (PW) encapsulations and utilizes existing MPLS-TE encapsulations and mechanisms. Background on PWs can be found in , , and . Background on MPLS-TE can be found in and .

DetNet Adaptation to MPLS Data Plane DetNet MPLS . Bottom of Stack . (inner label) +------------+ | Service | d-CW, S-Label (A-Label) +------------+ | Forwarding | F-Label(s) +------------+ Top of Stack . (outer label) .

The DetNet MPLS data plane representation is illustrated in . The service sub-layer includes a DetNet Control Word (d-CW) and an identifying service label (S-Label). The DetNet Control Word (d-CW) conforms to the Generic PW MPLS Control Word (PWMCW) defined in . An aggregation label (A-Label) is a special case of S-Label used for aggregation. A node operating on a received DetNet flow at the DetNet service sub-layer uses the local context associated with a received S-Label, i.e., a received F-Label, to determine which local DetNet operation(s) are applied to that packet. An S-Label may be taken from the platform label space , making it unique and enabling DetNet flow identification regardless of which input interface or LSP the packet arrives on. It is important to note that S-Label values are driven by the receiver, not the sender. The DetNet forwarding sub-layer is supported by zero or more forwarding labels (F-Labels). MPLS-TE encapsulations and mechanisms can be utilized to provide a forwarding sub-layer that is responsible for providing resource allocation and explicit routes.

DetNet MPLS Data Plane Scenarios

A DetNet MPLS Network DetNet MPLS Relay Transit Relay DetNet MPLS End System Node Node Node End System (T-PE) (S-PE) (LSR) (S-PE) (T-PE) +----------+ +----------+ | Appl. |<------------ End-to-End Service ----------->| Appl. | +----------+ +---------+ +---------+ +----------+ | Service |<--| Service |-- DetNet flow --| Service |-->| Service | +----------+ +---------+ +----------+ +---------+ +----------+ |Forwarding| |Fwd| |Fwd| |Forwarding| |Fwd| |Fwd| |Forwarding| +-------.--+ +-.-+ +-.-+ +----.---.-+ +-.-+ +-.-+ +---.------+ : Link : / ,-----. \ : Link : / ,-----. \ +........+ +-[ Sub- ]-+ +......+ +-[ Sub- ]-+ [Network] [Network] `-----' `-----' |<- LSP -->| |<-------- LSP -----------| |<--- LSP -->| |<----------------- DetNet MPLS --------------------->|

illustrates a hypothetical DetNet MPLS-only network composed of DetNet-aware MPLS-enabled end systems operating over a DetNet-aware MPLS network. In this figure, the relay nodes are PE devices that define the MPLS LSP boundaries, and the transit nodes are LSRs. DetNet end systems and relay nodes understand the particular needs of DetNet flows and provide both DetNet service and forwarding sub-layer functions. In the case of MPLS, DetNet service-aware nodes add, remove, and process d-CWs, S-Labels, and F-Labels as needed. DetNet MPLS nodes provide functionality analogous to T-PEs when they sit at the edge of an MPLS domain and S-PEs when they are in the middle of an MPLS domain; see . In a DetNet MPLS network, transit nodes may be DetNet-service-aware or DetNet-unaware MPLS Label Switching Routers (LSRs). In this latter case, such LSRs would be unaware of the special requirements of the DetNet service sub-layer but would still provide traffic engineering functions and the QoS capabilities needed to ensure that the (TE) LSPs meet the service requirements of the carried DetNet flows. The application of DetNet using MPLS supports a number of control and management plane types. These types support certain MPLS data plane deployments. For example, RSVP-TE, MPLS-TP, or MPLS Segment Routing (when extended to support resource allocation) are all valid MPLS deployment cases. illustrates how an end-to-end MPLS-based DetNet service is provided in more detail. In this figure, the Customer Edge (CE1 and CE2) are able to send and receive MPLS-encapsulated DetNet flows, and R1, R2, and R3 are relay nodes in the middle of a DetNet network. The 'X' in the end systems and relay nodes represents potential DetNet compound flow packet replication and elimination points. In this example, service protection is supported utilizing at least two DetNet member flows and TE LSPs. For a unidirectional flow, R1 supports PRF, and R3 supports PEF and POF. Note that the relay nodes may change the underlying forwarding sub-layer, for example, tunneling MPLS over IEEE 802.1 TSN or simply over interconnected network links.

MPLS-Based Native DetNet DetNet DetNet DetNet Service Transit Transit Service DetNet MPLS | |<-Tnl->| |<-Tnl->| | MPLS End | V 1 V V 2 V | End System | +--------+ +--------+ +--------+ | System +---+ | | R1 |=======| R2 |=======| R3 | | +---+ | X...DFa...|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|.DFa..|.X | |CE1|========| \ | | X | | / |======|CE2| | | | | \_.|..DF2..|._/ \__.|..DF4..|._/ | | | | +---+ | |=======| |=======| | +---+ ^ +--------+ +--------+ +--------+ ^ | Relay Node Relay Node Relay Node | | (S-PE) (S-PE) (S-PE) | | | |<---------------------- DetNet MPLS --------------------->| | | |<--------------- End-to-End DetNet Service -------------->| -------------------------- Data Flow ------------------------->

X

- Optional service protection (none, PRF, PREOF, PEF/POF)

DFx

- DetNet member flow x over a TE LSP

MPLS-Based DetNet Data Plane Solution

DetNet over MPLS Encapsulation Components To carry DetNet over MPLS, the following is required:

1. A method of identifying the MPLS payload type.
2. A method of identifying the DetNet flow(s) to the processing element.
3. A method of distinguishing DetNet OAM packets from DetNet data packets.
4. A method of carrying the DetNet sequence number.
5. A suitable LSP to deliver the packet to the egress PE.
6. A method of carrying queuing and forwarding indication.

In this design, an MPLS service label (the S-Label) is similar to a pseudowire (PW) label and is used to identify both the DetNet flow identity and the MPLS payload type satisfying (1) and (2) in the list above. OAM traffic discrimination happens through the use of the Associated Channel method described in . The DetNet sequence number is carried in the DetNet Control Word, which also carries the Data/OAM discriminator. To simplify implementation and to maximize interoperability, two sequence number sizes are supported: a 16-bit sequence number and a 28-bit sequence number. The 16-bit sequence number is needed to support some types of legacy clients. The 28-bit sequence number is used in situations where it is necessary to ensure that, in high-speed networks, the sequence number space does not wrap whilst packets are in flight. The LSP used to forward the DetNet packet is not restricted regarding any method used for establishing that LSP (for example, MPLS-LDP, MPLS-TE, MPLS-TP , MPLS Segment Routing , etc.). The F-Label(s) and the S-Label may be used alone or together to indicate the required queue processing as well as the forwarding parameters. Note that the possible use of Penultimate Hop Popping (PHP) means that the S-Label may be the only label received at the terminating DetNet service.

MPLS Data Plane Encapsulation illustrates a DetNet data plane MPLS encapsulation. The MPLS-based encapsulation of the DetNet flows is well suited for the scenarios described in . Furthermore, an end-to-end DetNet service, i.e., native DetNet deployment (see ), is also possible if DetNet end systems are capable of initiating and terminating MPLS-encapsulated packets. The MPLS-based DetNet data plane encapsulation consists of:

• A DetNet Control Word (d-CW) containing sequencing information for packet replication and duplicate elimination purposes, and the OAM indicator.
• A DetNet service label (S-Label) that identifies a DetNet flow at the receiving DetNet service sub-layer processing node.
• Zero or more DetNet MPLS forwarding label(s) (F-Label) used to direct the packet along the Label Switched Path (LSP) to the next DetNet service sub-layer processing node along the path. When PHP is in use, there may be no F-Label in the protocol stack on the final hop.
• The necessary data-link encapsulation is then applied prior to transmission over the physical media.

Encapsulation of a DetNet App-Flow in an MPLS PSN DetNet MPLS-based encapsulation +---------------------------------+ | | | DetNet App-Flow | | Payload Packet | | | +---------------------------------+ <--\ | DetNet Control Word | | +---------------------------------+ +--> DetNet data plane | S-Label | | MPLS encapsulation +---------------------------------+ | | [ F-Label(s) ] | | +---------------------------------+ <--/ | Data-Link | +---------------------------------+ | Physical | +---------------------------------+

DetNet Control Word and DetNet Sequence Number A DetNet Control Word (d-CW) conforms to the Generic PW MPLS Control Word (PWMCW) defined in . The d-CW formatted as shown in MUST be present in all DetNet packets containing App-flow data. This format of the d-CW was created in order to (1) allow larger sequence number space to avoid sequence number rollover frequency in some applications and (2) allow sequence numbering systems that include the value zero as a valid sequence number, which simplifies implementation.

DetNet Control Word 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0| Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

(bits 0 to 3)

Per , MUST be set to zero (0).

Sequence Number (bits 4 to 31)

An unsigned value implementing the DetNet sequence number. The sequence number space is a circular one with no restriction on the initial value.

A separate sequence number space MUST be maintained by the node that adds the d-CW for each DetNet App-flow, i.e., DetNet service. The following Sequence Number field lengths MUST be supported:

• 0 bits
• 16 bits
• 28 bits

The sequence number length MUST be provisioned on a per-DetNet-service basis via configuration, i.e., via the Controller Plane described in . A 0-bit Sequence Number field length indicates that there is no DetNet sequence number used for the flow. When the length is zero, the Sequence Number field MUST be set to zero (0) on all packets sent for the flow. When the Sequence Number field length is 16 or 28 bits for a flow, the sequence number MUST be incremented by one for each new App-flow packet sent. When the field length is 16 bits, d-CW bits 4 to 15 MUST be set to zero (0). The values carried in this field can wrap, and it is important to note that zero (0) is a valid field value. For example, where the sequence number size is 16 bits, the sequence will contain: 65535, 0, 1, where zero (0) is an ordinary sequence number. It is important to note that this document differs from , where a sequence number of zero (0) is used to indicate that the sequence number check algorithm is not used. The sequence number is optionally used during receive processing, as described below in Sections and .

S-Labels A DetNet flow at the DetNet service sub-layer is identified by an S-Label. MPLS-aware DetNet end systems and edge nodes, which are by definition MPLS ingress and egress nodes, MUST add (push) and remove (pop) a DetNet service-specific d-CW and S-Label. Relay nodes MAY swap S-Label values when processing a DetNet flow, i.e., incoming and outgoing S-Labels of a DetNet flow can be different. S-Label values MUST be provisioned per DetNet service via configuration, i.e., via the Controller Plane described in . Note that S-Labels provide identification at the downstream DetNet service sub-layer receiver, not the sender. As such, S-Labels MUST be allocated by the entity that controls the service sub-layer receiving a node's label space and MAY be allocated from the platform label space . Because S-Labels are local to each node, rather than being a global identifier within a domain, they must be advertised to their upstream DetNet service-aware peer nodes (i.e., a DetNet MPLS end system or a DetNet relay or edge node) and interpreted in the context of their received F-Label(s). In some PREOF topologies, the node performing replication will be sending to multiple nodes performing PEF or POF and may need to send different S-Label values for the different member flows for the same DetNet service. An S-Label will normally be at the bottom of the label stack once the last F-Label is removed, immediately preceding the d-CW. To support OAM at the service sub-layer level, an OAM Associated Channel Header (ACH) together with a Generic Associated Channel Label (GAL) MAY be used in place of a d-CW. Similarly, an Entropy Label Indicator (ELI) and Entropy Label (EL) MAY be carried below the S-Label in the label stack in networks where DetNet flows would otherwise receive ECMP treatment. When ELs are used, the same EL value SHOULD be used for all of the packets sent using a specific S-Label to force the flow to follow the same path. However, as outlined in , the use of ECMP for DetNet flows is NOT RECOMMENDED. ECMP MAY be used for non-DetNet flows within a DetNet domain. When receiving a DetNet MPLS packet, an implementation MUST identify the DetNet service associated with the incoming packet based on the S-Label. When a node is using platform labels for S-Labels, no additional information is needed, as the S-Label uniquely identifies the DetNet service. In the case where platform labels are not used, zero or more F-Labels proceeding the S-Label MUST be used together with the S-Label to uniquely identify the DetNet service associated with a received packet. The incoming interface MAY also be used together with any present F-Label(s) and the S-Label to uniquely identify an incoming DetNet service, for example, in the case where PHP is used. Note that the choice to use the platform label space for an S-Label or an S-Label plus one or more F-Labels to identify DetNet services is a local implementation choice, with one caveat. When one or more F-Labels, or the incoming interface, is needed together with an S-Label to uniquely identify a service, the Controller Plane must ensure that incoming DetNet MPLS packets arrive with the needed information (F-Label(s) and/or the incoming interface) and provision the needed information. The provisioned information MUST then be used to identify incoming DetNet service based on the combination of S-Label and F-Label(s) or the incoming interface. The use of platform labels for S-Labels matches other pseudowire encapsulations for consistency, but there is no hard requirement in this regard. Implementation details of PREOF are out of scope for this document. defines equivalent replication and elimination-specific aspects, which can be applied to PRF and PEF.

Packet Replication Function Processing The Packet Replication Function (PRF) MAY be supported by an implementation for outgoing DetNet flows. The use of the PRF for a particular DetNet service MUST be provisioned via configuration, i.e., via the Controller Plane described in . When replication is configured, the same App-flow data will be sent over multiple outgoing DetNet member flows using forwarding sub-layer LSPs. An S-Label value MUST be configured per outgoing member flow. The same d-CW field value MUST be used on all outgoing member flows for each replicated MPLS packet.

Packet Elimination Function Processing Implementations MAY support the Packet Elimination Function (PEF) for received DetNet MPLS flows. When supported, use of the PEF for a particular DetNet service MUST be provisioned via configuration, i.e., via the Controller Plane described in . After a DetNet service is identified for a received DetNet MPLS packet, as described above, if PEF is configured for that DetNet service, duplicate (replicated) instances of a particular sequence number MUST be discarded. The specific mechanisms used for an implementation to identify which received packets are duplicates and which are new is an implementation choice. Note that, per , the Sequence Number field length may be 16 or 28 bits, and the field value can wrap. PEF MUST NOT be used with DetNet flows configured with a d-CW Sequence Number field length of 0 bits. An implementation MAY constrain the maximum number of sequence numbers that are tracked on either a platform-wide or per-flow basis. Some implementations MAY support the provisioning of the maximum number of sequence numbers that are tracked on either a platform-wide or per-flow basis.

Packet Ordering Function Processing A function that is related to in-order delivery is the Packet Ordering Function (POF). Implementations MAY support POF. When supported, use of the POF for a particular DetNet service MUST be provisioned via configuration, i.e., via the Controller Plane described by . Implementations MAY require that PEF and POF be used in combination. There is no requirement related to the order of execution of the PEF and POF in an implementation. After a DetNet service is identified for a received DetNet MPLS packet, as described above, if POF is configured for that DetNet service, packets MUST be processed in the order indicated in the received d-CW Sequence Number field, which may not be in the order the packets are received. As defined in , the Sequence Number field length may be 16 or 28 bits, the sequence number is incremented by one (1) for each new MPLS packet sent for a particular DetNet service, and the field value can wrap. The specific mechanisms used for an implementation to identify the order of received packets is an implementation choice. An implementation MAY constrain the maximum number of out-of-order packets that can be processed on either a platform-wide or per-flow basis. The number of out-of-order packets that can be processed also impacts the latency of a flow. The latency impact on the system resources needed to support a specific DetNet flow will need to be evaluated by the Controller Plane based on that flow's traffic specification. An example traffic specification that can be used with MPLS-TE can be found in . DetNet implementations can use flow-specific requirements (e.g., maximum number of out-of-order packets and maximum latency of the flow) for configuration of POF-related buffers. POF implementation details are out of scope for this document, and POF configuration parameters are implementation specific. The Controller Plane identifies and sets the POF configuration parameters.

F-Labels F-Labels support the DetNet forwarding sub-layer. F-Labels are used to provide LSP-based connectivity between DetNet service sub-layer processing nodes.

Service Sub-Layer-Related Processing DetNet MPLS end systems, edge nodes, and relay nodes may operate at the DetNet service sub-layer with understanding of DetNet services and their requirements. As mentioned earlier, when operating at this layer, such nodes can push, pop, or swap (pop then push) S-Labels. In all cases, the F-Label(s) used for a DetNet service are always replaced, and the following procedures apply. When sending a DetNet flow, zero or more F-Labels MAY be pushed on top of an S-Label by the node pushing an S-Label. The F-Label(s) to be pushed when sending a particular DetNet service MUST be provisioned per outgoing S-Label via configuration, i.e., via the Controller Plane discussed in . F-Label(s) can also provide context for an S-Label. To allow for the omission of F-Label(s), an implementation SHOULD also allow an outgoing interface to be configured per S-Label. Note that when PRF is supported, the same App-flow data will be sent over multiple outgoing DetNet member flows using forwarding sub-layer LSPs. This means that an implementation may be sending different sets of F-Labels per DetNet member flow, each with a different S-Label. When a single set of F-Labels is provisioned for a particular outgoing S-Label, that set of F-Labels MUST be pushed after the S-Label is pushed. The outgoing packet is then forwarded, as described below in . When a single outgoing interface is provisioned, the outgoing packet is then forwarded, as described below in . When multiple sets of outgoing F-Labels or interfaces are provisioned for a particular DetNet service (i.e., for PRF), a copy of the outgoing packet, including the pushed member flow-specific S-Label, MUST be made per F-Label set and outgoing interface. Each set of provisioned F-Labels are then pushed onto a copy of the packet. Each copy is then forwarded, as described below in . As described in the previous section, when receiving a DetNet MPLS flow, an implementation identifies the DetNet service associated with the incoming packet based on the S-Label. When a node is using platform labels for S-Labels, any F-Labels can be popped, and the S-Label uniquely identifies the DetNet service. In the case where platform labels are not used, incoming F-Label(s) and, optionally, the incoming interface MUST also be provisioned for a DetNet service.

Common F-Label Processing All DetNet-aware MPLS nodes process F-Labels as needed to meet the service requirements of the DetNet flow or flows carried in the LSPs represented by the F-Labels. This includes normal push, pop, and swap operations. Such processing is essentially the same type of processing provided for TE LSPs, although the specific service parameters, or traffic specification, can differ. When the DetNet service parameters of the DetNet flow or flows carried in an LSP represented by an F-Label can be met by an existing TE mechanism, the forwarding sub-layer processing node MAY be a DetNet-unaware, i.e., standard, MPLS LSR. Such TE LSPs may provide LSP forwarding service as defined in, but not limited, to the following: , , , , , , and . More specifically, as mentioned above, the DetNet forwarding sub-layer provides explicit routes and allocated resources, and F-Labels are used to map to each. Explicit routes are supported based on the topmost (outermost) F-Label that is pushed or swapped and the LSP that corresponds to this label. This topmost (outgoing) label MUST be associated with a provisioned outgoing interface and, for non-point-to-point outgoing interfaces, a next-hop LSR. Note that this information MUST be provisioned via configuration or the Controller Plane. In the previously mentioned special case where there are no added F-Labels and the outgoing interface is not a point-to-point interface, the outgoing interface MUST also be associated with a next-hop LSR. Resources may be allocated in a hierarchical fashion per each LSP that is represented by each F-Label. Each LSP MAY be provisioned with a service parameter that dictates the specific traffic treatment to be received by the traffic carried over that LSP. Implementations of this document MUST ensure that traffic carried over each LSP represented by one or more F-Labels receives the traffic treatment provisioned for that LSP. Typical mechanisms used to provide different treatment to different flows include the allocation of system resources (such as queues and buffers) and provisioning of related parameters (such as shaping and policing) that may be found in implementations of the Resource ReSerVation Protocol (RSVP) and RSVP-TE . Support can also be provided via an underlying network technology, such as IEEE 802.1 TSN . The specific mechanisms selected by a DetNet node to ensure DetNet service delivery requirements are met for supported DetNet flows is outside the scope of this document. Packets that are marked in a way that do not correspond to allocated resources, e.g., lack a provisioned F-Label, can disrupt the QoS offered to properly reserved DetNet flows by using resources allocated to the reserved flows. Therefore, the network nodes of a DetNet network:

• MUST defend the DetNet QoS by discarding or remarking (to an allocated DetNet flow or noncompeting non-DetNet flow) packets received that are not associated with a completed resource allocation.
• MUST NOT use a DetNet allocated resource, e.g., a queue or shaper reserved for DetNet flows, for any packet that does match the corresponding DetNet flow.
• MUST ensure a QoS flow does not exceed its allocated resources or provisioned service level.
• MUST ensure a CoS flow or service class does not impact the service delivered to other flows. This requirement is similar to the requirement for MPLS LSRs that CoS LSPs do not impact the resources allocated to TE LSPs, e.g., via .

Subsequent sections provide additional considerations related to CoS (), QoS (), and aggregation ().

OAM Indication OAM follows the procedures set out in with the restriction that only Virtual Circuit Connectivity Verification (VCCV) type 1 is supported. As shown in Figure 3 of , when the first nibble of the d-CW is 0x0, the payload following the d-CW is normal user data. However, when the first nibble of the d-CW is 0x1, the payload that follows the d-CW is an OAM payload with the OAM type indicated by the value in the d-CW Channel Type field. The reader is referred to for a more detailed description of the Associated Channel mechanism and to the DetNet work on OAM for more information about DetNet OAM. Additional considerations on DetNet-specific OAM are subjects for further study.

Flow Aggregation The ability to aggregate individual flows and their associated resource control into a larger aggregate is an important technique for improving scaling of control in the data, management, and control planes. The DetNet data plane allows for the aggregation of DetNet flows to improved scaling. There are two methods of supporting flow aggregation covered in this section. The resource control and management aspects of aggregation (including the configuration of queuing, shaping, and policing) are the responsibility of the DetNet Controller Plane and are out of scope for this document. It is also the responsibility of the Controller Plane to ensure that consistent aggregation methods are used.

Aggregation via LSP Hierarchy DetNet flows forwarded via MPLS can leverage MPLS-TE's existing support for hierarchical LSPs (H-LSPs); see . H-LSPs are typically used to aggregate control and resources; they may also be used to provide OAM or protection for the aggregated LSPs. Arbitrary levels of aggregation naturally fall out of the definition for hierarchy and the MPLS label stack . DetNet nodes that support aggregation (LSP hierarchy) map one or more LSPs (labels) into and from an H-LSP. Both carried LSPs and H-LSPs may or may not use the Traffic Class (TC) field, i.e., L-LSPs (Label-Only-Inferred-PSC LSPs) or E-LSPs (EXP-Inferred-PSC LSPs , which were renamed to "Explicitly TC-encoded-PSC LSPs" in ). Such nodes will need to ensure that individual LSPs and H-LSPs receive the traffic treatment required to ensure the required DetNet service is preserved. Additional details of the traffic control capabilities needed at a DetNet-aware node may be covered in the new service definitions mentioned above or in separate future documents. Controller Plane mechanisms will also need to ensure that the service required on the aggregate flow are provided, which may include the discarding or remarking mentioned in the previous sections.

Aggregating DetNet Flows as a New DetNet Flow An aggregate can be built by layering DetNet flows, including both their S-Label and (when present) F-Labels, as shown below:

DetNet Aggregation as a New DetNet Flow +---------------------------------+ | | | DetNet Flow | | Payload Packet | | | +---------------------------------+ <--\ | DetNet Control Word | | +=================================+ | | S-Label | | +---------------------------------+ | | [ F-Label(s) ] | +----DetNet data plane +---------------------------------+ | | DetNet Control Word | | +=================================+ | | A-Label | | +---------------------------------+ | | F-Label(s) | <--/ +---------------------------------+ | Data-Link | +---------------------------------+ | Physical | +---------------------------------+

Both the aggregation label, which is referred to as an A-Label, and the individual flow's S-Label have their MPLS S bit set indicating the bottom of stack, and the d-CW allows the PREOF to work. An A-Label is a special case of an S-Label, whose properties are known only at the aggregation and deaggregation end points. It is a property of the A-Label that what follows is a d-CW followed by an MPLS label stack. A relay node processing the A-Label would not know the underlying payload type, and the A-Label would be processed as a normal S-Label. This would only be known to a node that was a peer of the node imposing the S-Label. However, there is no real need for it to know the payload type during aggregation processing. As in the previous section, nodes supporting this type of aggregation will need to ensure that individual and aggregated flows receive the traffic treatment required to ensure the required DetNet service is preserved. Also, it is the Controller Plane's responsibility to ensure that the service required on the aggregate flow is properly provisioned.

Service Sub-Layer Considerations The internal procedures for edge and relay nodes related to PREOF are implementation specific. The order of a packet elimination or replication is out of scope for this specification. It is important that the DetNet layer is configured such that a DetNet node never receives its own replicated packets. If it were to receive such packets, the replication function would make the loop more destructive of bandwidth than a conventional unicast loop. Ultimately, the TTL in the S-Label will cause the packet to die during a transient loop, but given the sensitivity of applications to packet latency, the impact on the DetNet application would be severe. To avoid the problem of a transient forwarding loop, changes to an LSP supporting DetNet MUST be loop-free.

Edge Node Processing A DetNet edge node operates in the DetNet forwarding sub-layer and service sub-layer. An edge node is responsible for matching incoming packets to the service they require and encapsulating them accordingly. An edge node may perform PRF, PEF, and/or POF. Details on encapsulation can be found in ; details on PRF can be found in ; details on PEF can be found in ; and details on POF can be found in .

Relay Node Processing A DetNet relay node operates in the DetNet forwarding sub-layer and service sub-layer. For DetNet using MPLS, forwarding-related processing is performed on the F-Label. This processing is done within an extended forwarder function. Whether an incoming DetNet flow receives DetNet-specific processing depends on how the forwarding is programmed. Some relay nodes may be DetNet service aware for certain DetNet services, while, for other DetNet services, these nodes may perform as unmodified LSRs that only understand how to switch MPLS-TE LSPs, i.e., as a transit node; see . Again, this is entirely up to how the forwarding has been programmed. During the elimination and replication process, the sequence number of an incoming DetNet packet MUST be preserved and carried in the corresponding outgoing DetNet packet. For example, a relay node that performs both PEF and PRF first performs PEF on incoming packets to create a compound flow. It then performs PRF and copies the App-flow data and the d-CW into packets for each outgoing DetNet member flow. The internal design of a relay node is out of scope for this document. However, the reader's attention is drawn to the need to make any PREOF state available to the packet processor(s) dealing with packets to which PREOF must be applied and to maintain that state in such a way that it is available to the packet processor operation on the next packet in the DetNet flow (which may be a duplicate, a late packet, or the next packet in sequence).

Forwarding Sub-Layer Considerations

Class of Service Class of Service (CoS) and Quality of Service (QoS) are terms that are often used interchangeably and confused with each other. In the context of DetNet, CoS is used to refer to mechanisms that provide traffic-forwarding treatment based on non-flow-specific traffic classification, and QoS is used to refer to mechanisms that provide traffic-forwarding treatment based on DetNet flow-specific traffic classification. Examples of existing network-level CoS mechanisms include Differentiated Services (Diffserv), which is enabled by the IP header Differentiated Services Code Point (DSCP) field and MPLS label Traffic Class field and at Layer 2 by IEEE 802.1p Priority Code Point (PCP). CoS for DetNet flows carried in PWs and MPLS is provided using the existing MPLS Differentiated Services (Diffserv) architecture . Both E-LSP and L-LSP MPLS Diffserv modes MAY be used to support DetNet flows. The Traffic Class field (formerly the EXP field) of an MPLS label follows the definition of and . The Uniform, Pipe, and Short Pipe Diffserv tunneling and TTL processing models are described in and and MAY be used for MPLS LSPs supporting DetNet flows. MPLS Explicit Congestion Notification (ECN) MAY also be used, as defined in ECN and updated by .

Quality of Service In addition to explicit routes and packet replication and elimination (described in above), DetNet provides zero congestion loss and bounded latency and jitter. As described in , there are different mechanisms that may be used separately or in combination to deliver a zero congestion loss service. This includes QoS mechanisms at the MPLS layer, which may be combined with the mechanisms defined by the underlying network layer, such as IEEE 802.1 TSN. QoS mechanisms for flow-specific traffic treatment typically include a guarantee/agreement for the service and allocation of resources to support the service. Example QoS mechanisms include discrete resource allocation, admission control, flow identification and isolation, and sometimes path control, traffic protection, shaping, policing, and remarking. Example protocols that support QoS control include the Resource ReSerVation Protocol (RSVP) and RSVP-TE . The existing MPLS mechanisms defined to support CoS can also be used to reserve resources for specific traffic classes. A baseline set of QoS capabilities for DetNet flows carried in PWs and MPLS can be provided by MPLS-TE . TE LSPs can also support explicit routes (path pinning). Current service definitions for packet TE LSPs can be found in "Specification of the Controlled-Load Network Element Service" , "Specification of Guaranteed Quality of Service" , and "Ethernet Traffic Parameters" . Additional service definitions are expected in future documents to support the full range of DetNet services. In all cases, the existing label-based marking mechanisms defined for TE LSPs and even E-LSPs are used to support the identification of flows requiring DetNet QoS.

Management and Control Information Summary The specific information needed for the processing of each DetNet service depends on the DetNet node type and the functions being provided on the node. This section summarizes this information based on the DetNet sub-layer and if the DetNet traffic is being sent or received. All DetNet node types are DetNet forwarding sub-layer aware, while all but transit nodes are service sub-layer aware. This is shown in . Much like other MPLS labels, there are a number of alternatives available for DetNet S-Label and F-Label advertisement to an upstream peer node. These include distributed signaling protocols (such as RSVP-TE), centralized label distribution via a controller that manages both the sender and the receiver using the Network Configuration Protocol (NETCONF) and YANG, BGP, the Path Computation Element Communication Protocol (PCEP), etc., and hybrid combinations of the two. The details of the Controller Plane solution required for the label distribution and the management of the label number space are out of scope for this document. Particular DetNet considerations and requirements are discussed in . Conformance language is not used in the summary, since it applies to future mechanisms, such as those that may be provided in signaling and YANG models, e.g., .

Service Sub-Layer Information Summary The following summarizes the information that is needed (on a per-service basis) on nodes that are service sub-layer aware and transmit DetNet MPLS traffic:

• App-flow identification information, e.g., IP information as defined in . Note that this information is not needed on DetNet relay nodes.
• The sequence number length to be used for the service. Valid values include 0, 16, and 28 bits. 0 bits cannot be used when PEF or POF is configured for the service.
• If PRF is to be provided for the service.
• The outgoing S-Label for each of the service's outgoing DetNet (member) flows.
• The forwarding sub-layer information associated with the output of the service sub-layer. Note that when PRF is provisioned, this information is per DetNet member flow. Logically, the forwarding sub-layer information is a pointer to further details of transmission of DetNet flows at the forwarding sub-layer.

The following summarizes the information that is needed (on a per-service basis) on nodes that are service sub-layer aware and receive DetNet MPLS traffic:

• The forwarding sub-layer information associated with the input of the service sub-layer. Note that when PEF is provisioned, this information is per DetNet member flow. Logically, the forwarding sub-layer information is a pointer to further details of the reception of DetNet flows at the forwarding sub-layer or A-Label.
• The incoming S-Label for the service.
• If PEF or POF is to be provided for the service.
• The sequence number length to be used for the service. Valid values included 0, 16, and 28 bits. 0 bits cannot be used when PEF or POF are configured for the service.
• App-flow identification information, e.g., IP information as defined in . Note that this information is not needed on DetNet relay nodes.

Service Aggregation Information Summary Nodes performing aggregation using A-Labels, per , require the additional information summarized in this section. The following summarizes the additional information that is needed on a node that sends aggregated flows using A-Labels:

• The S-Labels or F-Labels that are to be carried over each aggregated service.
• The A-Label associated with each aggregated service.
• The other S-Label information summarized above.

On the receiving node, the A-Label provides the forwarding context of an incoming interface or an F-Label and is used in subsequent service or forwarding sub-layer receive processing, as appropriate. The related additional configuration that may be provided is discussed elsewhere in this section.

Forwarding Sub-Layer Information Summary The following summarizes the information that is needed (on a per-forwarding-sub-layer-flow basis) on nodes that are forwarding sub-layer aware and send DetNet MPLS traffic:

• The outgoing F-Label stack to be pushed. The stack may include H-LSP labels.
• The traffic parameters associated with a specific label in the stack. Note that there may be multiple sets of traffic parameters associated with specific labels in the stack, e.g., when H-LSPs are used.
• Outgoing interface and, for unicast traffic, the next-hop information.
• Sub-network-specific parameters on a technology-specific basis. For example, see .

The following summarizes the information that is needed (on a per-forwarding-sub-layer-flow basis) on nodes that are forwarding sub-layer aware and receive DetNet MPLS traffic:

• The incoming interface. For some implementations and deployment scenarios, this information may not be needed.
• The incoming F-Label stack to be popped. The stack may include H-LSP labels.
• How the incoming forwarding sub-layer flow is to be handled, i.e., forwarded as a transit node or provided to the service sub-layer.

It is the responsibility of the DetNet Controller Plane to properly provision both flow identification information and the flow-specific resources needed to provide the traffic treatment needed to meet each flow's service requirements. This applies for aggregated and individual flows.

Security Considerations Detailed security considerations for DetNet are cataloged in , and more general security considerations are described in . This section exclusively considers security considerations that are specific to the DetNet MPLS data plane. The considerations raised related to MPLS networks in general in are equally applicable to the DetNet MPLS data plane. Security aspects that are unique to DetNet are those whose aim is to protect the support of specific QoS aspects of DetNet, which are primarily to deliver data flows with extremely low packet loss rates and bounded end-to-end delivery latency. Achieving such loss rates and bounded latency may not be possible in the face of a highly capable adversary, such as the one envisioned by the Internet Threat Model of BCP 72 that can arbitrarily drop or delay any or all traffic. In order to present meaningful security considerations, we consider a somewhat weaker attacker who does not control the physical links of the DetNet domain but may have the ability to control a network node within the boundary of the DetNet domain. An additional consideration for the DetNet data plane is to maintain integrity of data and delivery of the associated DetNet service traversing the DetNet network. Application flows can be protected through whatever means are provided by the underlying technology. For example, encryption may be used, such as that provided by IPsec for IP flows and/or by an underlying sub-network using MACsec for IP over Ethernet (Layer 2) flows. MPLS doesn't provide any native security services to account for confidentiality and integrity. From a data plane perspective, this document does not add or modify any application header information. At the management and control level, DetNet flows are identified on a per-flow basis, which may provide Controller Plane attackers with additional information about the data flows (when compared to Controller Planes that do not include per-flow identification). This is an inherent property of DetNet that has security implications that should be considered when determining if DetNet is a suitable technology for any given use case. To provide uninterrupted availability of the DetNet service, provisions can be made against DoS attacks and delay attacks. To protect against DoS attacks, excess traffic due to malicious or malfunctioning devices is prevented or mitigated through the use of existing mechanisms, for example, by policing and shaping incoming traffic. To prevent DetNet packets from having their delay manipulated by an external entity, precautions need to be taken to ensure that all devices on an LSP are those intended to be there by the network operator and that they are well behaved. In addition to physical security, technical methods, such as authentication and authorization of network equipment and the instrumentation and monitoring of the LSP packet delay, may be used. If a delay attack is suspected, traffic may be moved to an alternate path, for example, through changing the LSP or management of the PREOF configuration.

IANA Considerations This document has no IANA actions.

References Normative References In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. This memo specifies the network element behavior required to deliver Controlled-Load service in the Internet. [STANDARDS-TRACK] This memo describes the network element behavior required to deliver a guaranteed service (guaranteed delay and bandwidth) in the Internet. [STANDARDS-TRACK] This document specifies the architecture for Multiprotocol Label Switching (MPLS). [STANDARDS-TRACK] This document specifies the encoding to be used by an LSR in order to transmit labeled packets on Point-to-Point Protocol (PPP) data links, on LAN data links, and possibly on other data links as well. This document also specifies rules and procedures for processing the various fields of the label stack encoding. [STANDARDS-TRACK] This document describes the use of RSVP (Resource Reservation Protocol), including all the necessary extensions, to establish label-switched paths (LSPs) in MPLS (Multi-Protocol Label Switching). Since the flow along an LSP is completely identified by the label applied at the ingress node of the path, these paths may be treated as tunnels. A key application of LSP tunnels is traffic engineering with MPLS as specified in RFC 2702. [STANDARDS-TRACK] This document defines a flexible solution for support of Differentiated Services (Diff-Serv) over Multi-Protocol Label Switching (MPLS) networks. [STANDARDS-TRACK] This document describes Time To Live (TTL) processing in hierarchical Multi-Protocol Label Switching (MPLS) networks and is motivated by the need to formalize a TTL-transparent mode of operation for an MPLS label-switched path. It updates RFC 3032, "MPLS Label Stack Encoding". TTL processing in both Pipe and Uniform Model hierarchical tunnels are specified with examples for both "push" and "pop" cases. The document also complements RFC 3270, "MPLS Support of Differentiated Services" and ties together the terminology introduced in that document with TTL processing in hierarchical MPLS networks. [STANDARDS-TRACK] This document describes extensions to Multi-Protocol Label Switching (MPLS) Resource ReserVation Protocol - Traffic Engineering (RSVP-TE) signaling required to support Generalized MPLS. Generalized MPLS extends the MPLS control plane to encompass time-division (e.g., Synchronous Optical Network and Synchronous Digital Hierarchy, SONET/SDH), wavelength (optical lambdas) and spatial switching (e.g., incoming port or fiber to outgoing port or fiber). This document presents a RSVP-TE specific description of the extensions. A generic functional description can be found in separate documents. [STANDARDS-TRACK] To improve scalability of Generalized Multi-Protocol Label Switching (GMPLS) it may be useful to aggregate Label Switched Paths (LSPs) by creating a hierarchy of such LSPs. A way to create such a hierarchy is by (a) a Label Switching Router (LSR) creating a Traffic Engineering Label Switched Path (TE LSP), (b) the LSR forming a forwarding adjacency (FA) out of that LSP (by advertising this LSP as a Traffic Engineering (TE) link into the same instance of ISIS/OSPF as the one that was used to create the LSP), (c) allowing other LSRs to use FAs for their path computation, and (d) nesting of LSPs originated by other LSRs into that LSP (by using the label stack construct). This document describes the mechanisms to accomplish this. [PROPOSED STANDARD] This document describes the preferred design of a Pseudowire Emulation Edge-to-Edge (PWE3) Control Word to be used over an MPLS packet switched network, and the Pseudowire Associated Channel Header. The design of these fields is chosen so that an MPLS Label Switching Router performing MPLS payload inspection will not confuse a PWE3 payload with an IP payload. [STANDARDS-TRACK] This document describes Virtual Circuit Connectivity Verification (VCCV), which provides a control channel that is associated with a pseudowire (PW), as well as the corresponding operations and management functions (such as connectivity verification) to be used over that control channel. VCCV applies to all supported access circuit and transport types currently defined for PWs. [STANDARDS-TRACK] RFC 3270 defines how to support the Diffserv architecture in MPLS networks, including how to encode Diffserv Code Points (DSCPs) in an MPLS header. DSCPs may be encoded in the EXP field, while other uses of that field are not precluded. RFC 3270 makes no statement about how Explicit Congestion Notification (ECN) marking might be encoded in the MPLS header. This document defines how an operator might define some of the EXP codepoints for explicit congestion notification, without precluding other uses. [STANDARDS-TRACK] The early Multiprotocol Label Switching (MPLS) documents defined the form of the MPLS label stack entry. This includes a three-bit field called the "EXP field". The exact use of this field was not defined by these documents, except to state that it was to be "reserved for experimental use". Although the intended use of the EXP field was as a "Class of Service" (CoS) field, it was not named a CoS field by these early documents because the use of such a CoS field was not considered to be sufficiently defined. Today a number of standards documents define its usage as a CoS field. To avoid misunderstanding about how this field may be used, it has become increasingly necessary to rename this field. This document changes the name of the field to the "Traffic Class field" ("TC field"). In doing so, it also updates documents that define the current use of the EXP field. [STANDARDS-TRACK] This document generalizes the applicability of the pseudowire (PW) Associated Channel Header (ACH), enabling the realization of a control channel associated to MPLS Label Switched Paths (LSPs) and MPLS Sections in addition to MPLS pseudowires. In order to identify the presence of this Associated Channel Header in the label stack, this document also assigns one of the reserved MPLS label values to the Generic Associated Channel Label (GAL), to be used as a label based exception mechanism. Load balancing is a powerful tool for engineering traffic across a network. This memo suggests ways of improving load balancing across MPLS networks using the concept of "entropy labels". It defines the concept, describes why entropy labels are useful, enumerates properties of entropy labels that allow maximal benefit, and shows how they can be signaled and used for various applications. This document updates RFCs 3031, 3107, 3209, and 5036. [STANDARDS-TRACK] RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This document provides the overall architecture for Deterministic Networking (DetNet), which provides a capability to carry specified unicast or multicast data flows for real-time applications with extremely low data loss rates and bounded latency within a network domain. Techniques used include 1) reserving data-plane resources for individual (or aggregated) DetNet flows in some or all of the intermediate nodes along the path of the flow, 2) providing explicit routes for DetNet flows that do not immediately change with the network topology, and 3) distributing data from DetNet flow packets over time and/or space to ensure delivery of each packet's data in spite of the loss of a path. DetNet operates at the IP layer and delivers service over lower-layer technologies such as MPLS and Time- Sensitive Networking (TSN) as defined by IEEE 802.1. This document provides an overall framework for the Deterministic Networking (DetNet) data plane. It covers concepts and considerations that are generally common to any DetNet data plane specification. It describes related Controller Plane considerations as well. Informative References Work in Progress ZTE Corp. Huawei This document defines format and use principals of the Deterministic Network (DetNet) service Associated Channel (ACH) over a DetNet network with the MPLS data plane. The DetNet service ACH can be used to carry test packets of active Operations, Administration, and Maintenance protocols that are used to detect DetNet failures and measure performance metrics. Work in Progress Work in Progress Work in Progress Huawei Technologies Huawei Technologies ETRI LabN Consulting, L.L.C. Individual China Mobile This document contains the specification for Deterministic Networking flow configuration YANG Model. The model allows for provisioning of end-to-end DetNet service along the path without dependency on any signaling protocol. The YANG module defined in this document conforms to the Network Management Datastore Architecture (NMDA). Work in Progress IEEE IEEE This memo describes version 1 of RSVP, a resource reservation setup protocol designed for an integrated services Internet. RSVP provides receiver-initiated setup of resource reservations for multicast or unicast data flows, with good scaling and robustness properties. [STANDARDS-TRACK] This document defines the IP header field, called the DS (for differentiated services) field. [STANDARDS-TRACK] This memo describes the principles of Traffic Engineering (TE) in the Internet. The document is intended to promote better understanding of the issues surrounding traffic engineering in IP networks, and to provide a common basis for the development of traffic engineering capabilities for the Internet. The principles, architectures, and methodologies for performance evaluation and performance optimization of operational IP networks are discussed throughout this document. This memo provides information for the Internet community. All RFCs are required to have a Security Considerations section. Historically, such sections have been relatively weak. This document provides guidelines to RFC authors on how to write a good Security Considerations section. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. This document describes an architecture for Pseudo Wire Emulation Edge-to-Edge (PWE3). It discusses the emulation of services such as Frame Relay, ATM, Ethernet, TDM, and SONET/SDH over packet switched networks (PSNs) using IP or MPLS. It presents the architectural framework for pseudo wires (PWs), defines terminology, and specifies the various protocol elements and their functions. This memo provides information for the Internet community. This document describes an updated version of the "Security Architecture for IP", which is designed to provide security services for traffic at the IP layer. This document obsoletes RFC 2401 (November 1998). [STANDARDS-TRACK] An Ethernet pseudowire (PW) is used to carry Ethernet/802.3 Protocol Data Units (PDUs) over an MPLS network. This enables service providers to offer "emulated" Ethernet services over existing MPLS networks. This document specifies the encapsulation of Ethernet/802.3 PDUs within a pseudowire. It also specifies the procedures for using a PW to provide a "point-to-point Ethernet" service. [STANDARDS-TRACK] This document describes extensions to Resource Reservation Protocol - Traffic Engineering (RSVP-TE) for the set up of Traffic Engineered (TE) point-to-multipoint (P2MP) Label Switched Paths (LSPs) in Multi- Protocol Label Switching (MPLS) and Generalized MPLS (GMPLS) networks. The solution relies on RSVP-TE without requiring a multicast routing protocol in the Service Provider core. Protocol elements and procedures for this solution are described. There can be various applications for P2MP TE LSPs such as IP multicast. Specification of how such applications will use a P2MP TE LSP is outside the scope of this document. [STANDARDS-TRACK] This document specifies the Path Computation Element (PCE) Communication Protocol (PCEP) for communications between a Path Computation Client (PCC) and a PCE, or between two PCEs. Such interactions include path computation requests and path computation replies as well as notifications of specific states related to the use of a PCE in the context of Multiprotocol Label Switching (MPLS) and Generalized MPLS (GMPLS) Traffic Engineering. PCEP is designed to be flexible and extensible so as to easily allow for the addition of further messages and objects, should further requirements be expressed in the future. [STANDARDS-TRACK] This document provides a security framework for Multiprotocol Label Switching (MPLS) and Generalized Multiprotocol Label Switching (GMPLS) Networks. This document addresses the security aspects that are relevant in the context of MPLS and GMPLS. It describes the security threats, the related defensive techniques, and the mechanisms for detection and reporting. This document emphasizes RSVP-TE and LDP security considerations, as well as inter-AS and inter-provider security considerations for building and maintaining MPLS and GMPLS networks across different domains or different Service Providers. This document is not an Internet Standards Track specification; it is published for informational purposes. This document specifies an architectural framework for the application of Multiprotocol Label Switching (MPLS) to the construction of packet-switched transport networks. It describes a common set of protocol functions -- the MPLS Transport Profile (MPLS-TP) -- that supports the operational models and capabilities typical of such networks, including signaled or explicitly provisioned bidirectional connection-oriented paths, protection and restoration mechanisms, comprehensive Operations, Administration, and Maintenance (OAM) functions, and network operation in the absence of a dynamic control plane or IP forwarding support. Some of these functions are defined in existing MPLS specifications, while others require extensions to existing specifications to meet the requirements of the MPLS-TP. This document defines the subset of the MPLS-TP applicable in general and to point-to-point transport paths. The remaining subset, applicable specifically to point-to-multipoint transport paths, is outside the scope of this document. This document is a product of a joint Internet Engineering Task Force (IETF) / International Telecommunication Union Telecommunication Standardization Sector (ITU-T) effort to include an MPLS Transport Profile within the IETF MPLS and Pseudowire Emulation Edge-to-Edge (PWE3) architectures to support the capabilities and functionalities of a packet transport network as defined by the ITU-T. This document is not an Internet Standards Track specification; it is published for informational purposes. This document describes the support of Metro Ethernet Forum (MEF) Ethernet traffic parameters as described in MEF10.1 when using Generalized Multi-Protocol Label Switching (GMPLS) Resource ReSerVation Protocol - Traffic Engineering (RSVP-TE) signaling. [STANDARDS-TRACK] This document describes how to connect pseudowires (PWs) between different Packet Switched Network (PSN) domains or between two or more distinct PW control plane domains, where a control plane domain uses a common control plane protocol or instance of that protocol for a given PW. The different PW control plane domains may belong to independent autonomous systems, or the PSN technology is heterogeneous, or a PW might need to be aggregated at a specific PSN point. The PW packet data units are simply switched from one PW to another without changing the PW payload. [STANDARDS-TRACK] Point-to-point Multiprotocol Label Switching (MPLS) and Generalized MPLS (GMPLS) Traffic Engineering Label Switched Paths (TE LSPs) may be established using signaling techniques, but their paths may first need to be determined. The Path Computation Element (PCE) has been identified as an appropriate technology for the determination of the paths of point-to-multipoint (P2MP) TE LSPs. This document describes extensions to the PCE Communication Protocol (PCEP) to handle requests and responses for the computation of paths for P2MP TE LSPs. This document obsoletes RFC 6006. Segment Routing (SR) leverages the source-routing paradigm. A node steers a packet through a controlled set of instructions, called segments, by prepending the packet with an SR header. In the MPLS data plane, the SR header is instantiated through a label stack. This document specifies the forwarding behavior to allow instantiating SR over the MPLS data plane (SR-MPLS). This document specifies the Deterministic Networking (DetNet) data plane operation for IP hosts and routers that provide DetNet service to IP-encapsulated data. No DetNet-specific encapsulation is defined to support IP flows; instead, the existing IP-layer and higher-layer protocol header information is used to support flow identification and DetNet service delivery. This document builds on the DetNet architecture (RFC 8655) and data plane framework (RFC 8938).

Acknowledgements The authors wish to thank , , , , , , , , , , , , and for their various contributions to this work.

Contributors The editor of this document wishes to thank and acknowledge the following person who contributed substantially to the content of this document and should be considered a coauthor: LabN Consulting, L.L.C.

dfedyk@labn.net

Authors' Addresses Ericsson

Magyar Tudosok krt. 11. Budapest Hungary 1117 balazs.a.varga@ericsson.com

Ericsson

Magyar Tudosok krt. 11. Budapest Hungary 1117 janos.farkas@ericsson.com

LabN Consulting, L.L.C.

lberger@labn.net

Malis Consulting

agmalis@gmail.com

Futurewei Technologies

sb@stewartbryant.com

jouni.nospam@gmail.com