Ericsson

gregimirsky@gmail.com

Huawei

mach.chen@huawei.com

Ericsson

Magyar Tudosok krt. 11. Budapest Hungary 1117 balazs.a.varga@ericsson.com

RTG detnet DetNet OAM This document defines format and usage principles of the Deterministic Networking (DetNet) service Associated Channel over a DetNet network with the MPLS data plane. The DetNet service Associated Channel can be used to carry test packets of active Operations, Administration, and Maintenance (OAM) protocols that are used to detect DetNet failures and measure performance metrics.

Status of This Memo This is an Internet Standards Track document. This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841. Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at .

Copyright Notice Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents () in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.

Table of Contents

• .  Introduction
• .  Conventions Used in This Document
  • .  Terminology and Acronyms
  • .  Key Words
• .  Active OAM for DetNet Networks with the MPLS Data Plane
  • .  DetNet Active OAM Encapsulation
  • .  DetNet PREOF Interaction with Active OAM
• .  OAM Interworking Models
  • .  OAM of DetNet MPLS Interworking with OAM of TSN
  • .  OAM of DetNet MPLS Interworking with OAM of DetNet IP
• .  IANA Considerations
  • .  DetNet Associated Channel Header (d-ACH) Flags Registry
• .  Security Considerations
• .  References
  • .  Normative References
  • .  Informative References
• Acknowledgments
• Authors' Addresses

Introduction introduces and explains Deterministic Networking (DetNet) architecture and how the Packet Replication, Elimination, and Ordering Functions (PREOF) can be used to ensure a low packet drop ratio in a DetNet domain. Operations, Administration, and Maintenance (OAM) protocols are used to detect and localize network defects and to monitor network performance. Some OAM functions (e.g., failure detection) are usually performed proactively in the network, while others (e.g., defect localization) are typically performed on demand. These tasks can be achieved through a combination of active and hybrid OAM methods, as classified in . This document presents a format for active OAM in DetNet networks with the MPLS data plane. Also, this document defines format and usage principles of the DetNet service Associated Channel over a DetNet network with the MPLS data plane .

Conventions Used in This Document

Terminology and Acronyms The term "DetNet OAM" in this document is used interchangeably with a "set of OAM protocols, methods, and tools for Deterministic Networking".

BFD:

Bidirectional Forwarding Detection

CFM:

Connectivity Fault Management

d-ACH:

DetNet Associated Channel Header

DetNet:

Deterministic Networking

DetNet Node:

A node that is an actor in the DetNet domain. Examples of DetNet nodes include DetNet domain edge nodes and DetNet nodes that perform PREOF within the DetNet domain.

E2E:

End to end

F-Label:

A DetNet "forwarding" label. The F-Label identifies the Label Switched Path (LSP) used to forward a DetNet flow across an MPLS Packet Switched Network (PSN), e.g., a hop-by-hop label used between label switching routers.

OAM:

Operations, Administration, and Maintenance

PREOF:

Packet Replication, Elimination, and Ordering Functions

PW:

Pseudowire

S-Label:

A DetNet "service" label. An S-Label is used between DetNet nodes that implement the DetNet service sub-layer functions. An S-Label is also used to identify a DetNet flow at the DetNet service sub-layer.

TSN:

Time-Sensitive Networking

Underlay Network or Underlay Layer:

The network that provides connectivity between the DetNet nodes. One example of an underlay layer is an MPLS network that provides LSP connectivity between DetNet nodes.

Key Words The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Active OAM for DetNet Networks with the MPLS Data Plane OAM protocols and mechanisms act within the data plane of the particular networking layer; thus, it is critical that the data plane encapsulation supports OAM mechanisms that comply with the OAM requirements listed in . Operation of a DetNet data plane with an MPLS underlay network is specified in . Within the MPLS underlay network, DetNet flows are to be encapsulated analogous to pseudowires (PWs) as specified in and . For reference, the Generic PW MPLS Control Word (as defined in and used with DetNet) is reproduced in .

Generic PW MPLS Control Word Format 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0| Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

PREOF in the DetNet domain is composed of a combination of nodes that perform replication and elimination functions. The Elimination sub-function always uses the S-Label in conjunction with the packet sequencing information (i.e., the Sequence Number encoded in the DetNet Control Word). The Replication sub-function uses the S-Label information only.

DetNet Active OAM Encapsulation DetNet OAM, like PW OAM, uses the PW Associated Channel Header defined in . At the same time, a DetNet PW can be viewed as a Multi-Segment PW, where DetNet service sub-layer functions are at the segment endpoints. However, DetNet service sub-layer functions operate per packet level (not per segment). These per-packet level characteristics of PREOF require additional fields for proper OAM packet processing. MPLS encapsulation of a DetNet active OAM packet is shown in .

DetNet Active OAM Packet Encapsulation in the MPLS Data Plane +---------------------------------+ | | | DetNet OAM Packet | | | +---------------------------------+ <--\ | DetNet Associated Channel Header| | +---------------------------------+ +--> DetNet active OAM | S-Label | | MPLS encapsulation +---------------------------------+ | | [ F-Label(s) ] | | +---------------------------------+ <--/ | Data-Link | +---------------------------------+ | Physical | +---------------------------------+

displays encapsulation of a test packet for a DetNet active OAM protocol in case of MPLS over UDP/IP .

DetNet Active OAM Packet Encapsulation in MPLS over UDP/IP +---------------------------------+ | | | DetNet OAM Packet | | | +---------------------------------+ <--\ | DetNet Associated Channel Header| | +---------------------------------+ +--> DetNet active OAM | S-Label | | MPLS encapsulation +---------------------------------+ | | [ F-label(s) ] | | +---------------------------------+ <--+ | UDP Header | | +---------------------------------+ +--> DetNet data plane | IP Header | | IP encapsulation +---------------------------------+ <--/ | Data-Link | +---------------------------------+ | Physical | +---------------------------------+

displays the format of the DetNet Associated Channel Header (d-ACH).

d-ACH Format 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 1|Version|Sequence Number| Channel Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Node ID |Level| Flags |Session| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The d-ACH encodes the following fields:

Bits 0..3:

These MUST be 0b0001. This allows the packet to be distinguished from an IP packet and from a DetNet data packet .

Version:

A 4-bit field. This document specifies version 0.

Sequence Number:

An unsigned circular 8-bit field. Because a DetNet active OAM test packet includes d-ACH, does not apply to handling the Sequence Number field in DetNet OAM over the MPLS data plane. The sequence number space is circular with no restriction on the initial value. The originator DetNet node MUST set the value of the Sequence Number field before the transmission of a packet. The initial value SHOULD be random (unpredictable). The originator node SHOULD increase the value of the Sequence Number field by 1 for each active OAM packet. The originator MAY use other strategies, e.g., for negative testing of Packet Ordering Functions.

Channel Type:

A 16-bit field and the value of the DetNet Associated Channel Type. It MUST be one of the values listed in the IANA "MPLS Generalized Associated Channel (G-ACh) Types (including Pseudowire Associated Channel Types)" registry .

Node ID:

An unsigned 20-bit field. The value of the Node ID field identifies the DetNet node that originated the packet. A DetNet node MUST be provisioned with a Node ID that is unique in the DetNet domain. Methods for distributing Node ID are outside the scope of this specification.

Level:

A 3-bit field. Semantically, the Level field is analogous to the Maintenance Domain Level in . The Level field is used to cope with the "all active path forwarding" (defined by the TSN Task Group of the IEEE 802.1 WG ) characteristics of the PREOF concept. A hierarchical relationship between OAM domains can be created using the Level field value, as illustrated by Figure 18.7 in .

Flags:

A 5-bit field. The Flags field contains five 1-bit flags. creates the IANA "DetNet Associated Channel Header (d-ACH) Flags" registry for new flags to be defined. The flags defined in this specification are presented in .

Session ID:

A 4-bit field. The Session field distinguishes OAM sessions originating from the same node (a given Maintenance End Point may have multiple simultaneously active OAM sessions) at the given Level.

DetNet Associated Channel Header Flags Field Format 0 1 2 3 4 +-+-+-+-+-+ |U|U|U|U|U| +-+-+-+-+-+

U:

Unused and for future use. MUST be 0 on transmission and ignored on receipt.

According to , a DetNet flow is identified by the S-Label that MUST be at the bottom of the stack. An active OAM packet MUST include d-ACH immediately following the S-Label.

DetNet PREOF Interaction with Active OAM At the DetNet service sub-layer, special functions (notably PREOF) MAY be applied to the particular DetNet flow to potentially reduce packet loss, improve the probability of on-time packet delivery, and ensure in-order packet delivery. PREOF relies on sequencing information in the DetNet service sub-layer. For a DetNet active OAM packet, PREOF MUST use the Sequence Number field value as the source of this sequencing information. App-flow and OAM use different sequence number spaces. PREOF algorithms are executed with respect to the sequence number space identified by the flow's characteristic information. Although the Sequence Number field in d-ACH has a range from 0 through 255, it provides sufficient space because the rate of DetNet active OAM packets is significantly lower compared to the rate of DetNet packets in an App-flow; therefore, wrapping around is not an issue.

OAM Interworking Models Interworking of two OAM domains that utilize different networking technology can be realized by either a peering model or a tunneling model. In a peering model, OAM domains are within the corresponding network domain. When using the peering model, state changes that are detected by a Fault Management OAM protocol can be mapped from one OAM domain into another or a notification, e.g., an alarm can be sent to a central controller. In the tunneling model of OAM interworking, usually only one active OAM protocol is used. Its test packets are tunneled through another domain along with the data flow, thus ensuring fate sharing among test and data packets.

OAM of DetNet MPLS Interworking with OAM of TSN DetNet active OAM can provide end-to-end (E2E) fault management and performance monitoring for a DetNet flow. In the case of DetNet with an MPLS data plane and an IEEE 802.1 Time-Sensitive Networking (TSN) sub-network, it implies the interworking of DetNet active OAM with TSN OAM, of which the data plane aspects are specified in . When the peering model () is used in the Connectivity Fault Management (CFM) OAM protocol , the node that borders both TSN and DetNet MPLS domains MUST support . specifies the mapping of defect states between Ethernet Attachment Circuits and associated Ethernet PWs that are part of an E2E emulated Ethernet service and are also applicable to E2E OAM across DetNet MPLS and TSN domains. The CFM can provide fast detection of a failure in the TSN segment of the DetNet service. In the DetNet MPLS domain, Bidirectional Forwarding Detection (BFD), as specified in and , can be used. To provide E2E failure detection, the TSN and DetNet MPLS segments could be treated as concatenated such that the diagnostic codes (see ) MAY be used to inform the upstream DetNet MPLS node of a TSN segment failure. Performance monitoring can be supported by in the DetNet MPLS and by in TSN domains, respectively. Performance objectives for each domain should refer to metrics that are composable or are defined for each domain separately. The following considerations apply when using the tunneling model of OAM interworking between DetNet MPLS and TSN domains based on general principles described in :

• Active OAM test packets MUST be mapped to the same TSN Stream ID as the monitored DetNet flow.
• Active OAM test packets MUST be processed in the TSN domain based on their S-Label and Class of Service marking (the Traffic Class field value).

Mapping between a DetNet flow and TSN Stream in the TSN sub-network is described in . The mapping has to be done only on the edge node of the TSN sub-network, and intermediate TSN nodes do not need to recognize the S-Label. An edge node has two components:

1. A passive Stream identification function.
2. An active Stream identification function.

The first component identifies the DetNet flow (using Clause 6.8 of ), and the second component creates the TSN Stream by manipulating the Ethernet header. That manipulation simplifies the identification of the TSN Stream in the intermediate TSN nodes by avoiding the need for them to look outside of the Ethernet header. DetNet MPLS OAM packets use the same S-Label as the DetNet flow data packets. The above-described mapping function treats these OAM packets as data packets of the DetNet flow. As a result, DetNet MPLS OAM packets are fate sharing within the TSN sub-network. As an example of the mapping between DetNet MPLS and TSN, see Annex C.1 of that, in support of , describes how to match MPLS DetNet flows and achieve TSN Streams. Note that the tunneling model of the OAM interworking requires that the remote peer of the E2E OAM domain supports the active OAM protocol selected on the ingress endpoint. For example, if BFD is used for proactive path continuity monitoring in the DetNet MPLS domain, BFD support (as defined in ) is necessary at any TSN endpoint of the DetNet service.

OAM of DetNet MPLS Interworking with OAM of DetNet IP Interworking between active OAM segments in DetNet MPLS and DetNet IP domains can also be realized using either the peering model or the tunneling model, as discussed in . Using the same protocol, e.g., BFD over both segments, simplifies the mapping of errors in the peering model. For example, respective BFD sessions in DetNet MPLS and DetNet IP domains can be in a concatenated relationship as described in . To provide performance monitoring over a DetNet IP domain, the Simple Two-way Active Measurement Protocol (STAMP) and its extensions can be used to measure packet loss and packet delay metrics. Such performance metrics can be used to calculate composable metrics within DetNet MPLS and DetNet IP domains to reflect the end-to-end DetNet service performance.

IANA Considerations

DetNet Associated Channel Header (d-ACH) Flags Registry IANA has created the "DetNet Associated Channel Header (d-ACH) Flags" registry within the "DetNet Associated Channel Header (d-ACH) Flags" registry group. The registration procedure is "IETF Review" . There are five flags in the 5-bit Flags field, as defined in . DetNet Associated Channel Header (d-ACH) Flags Registry

Bit	Description
0-4	Unassigned

Security Considerations Security considerations discussed in DetNet specifications , , , and are applicable to this document. Security concerns and issues related to MPLS OAM tools like LSP Ping and BFD over PW also apply to this specification.

References Normative References In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. This document specifies the mapping of defect states between Ethernet Attachment Circuits (ACs) and associated Ethernet pseudowires (PWs) connected in accordance with the Pseudowire Emulation Edge-to-Edge (PWE3) architecture to realize an end-to-end emulated Ethernet service. It standardizes the behavior of Provider Edges (PEs) with respect to Ethernet PW and AC defects. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This document provides the overall architecture for Deterministic Networking (DetNet), which provides a capability to carry specified unicast or multicast data flows for real-time applications with extremely low data loss rates and bounded latency within a network domain. Techniques used include 1) reserving data-plane resources for individual (or aggregated) DetNet flows in some or all of the intermediate nodes along the path of the flow, 2) providing explicit routes for DetNet flows that do not immediately change with the network topology, and 3) distributing data from DetNet flow packets over time and/or space to ensure delivery of each packet's data in spite of the loss of a path. DetNet operates at the IP layer and delivers service over lower-layer technologies such as MPLS and Time- Sensitive Networking (TSN) as defined by IEEE 802.1. This document specifies the Deterministic Networking (DetNet) data plane when operating over an MPLS Packet Switched Network. It leverages existing pseudowire (PW) encapsulations and MPLS Traffic Engineering (MPLS-TE) encapsulations and mechanisms. This document builds on the DetNet architecture and data plane framework. This document specifies the MPLS Deterministic Networking (DetNet) data plane operation and encapsulation over an IP network. The approach is based on the operation of MPLS-over-UDP technology. Informative References IANA IEEE IEEE IEEE 802.1 TSN Standards ITU-T Ericsson CNRS IMT Atlantique Universidad Carlos III de Madrid Ericsson Ericsson Work in Progress This document describes an architecture for Pseudo Wire Emulation Edge-to-Edge (PWE3). It discusses the emulation of services such as Frame Relay, ATM, Ethernet, TDM, and SONET/SDH over packet switched networks (PSNs) using IP or MPLS. It presents the architectural framework for pseudo wires (PWs), defines terminology, and specifies the various protocol elements and their functions. This memo provides information for the Internet community. This document describes the preferred design of a Pseudowire Emulation Edge-to-Edge (PWE3) Control Word to be used over an MPLS packet switched network, and the Pseudowire Associated Channel Header. The design of these fields is chosen so that an MPLS Label Switching Router performing MPLS payload inspection will not confuse a PWE3 payload with an IP payload. [STANDARDS-TRACK] This document describes the Equal Cost Multipath (ECMP) behavior of currently deployed MPLS networks. This document makes best practice recommendations for anyone defining an application to run over an MPLS network that wishes to avoid the reordering that can result from transmission of different packets from the same flow over multiple different equal cost paths. These recommendations rely on inspection of the IP version number field in packets. Despite the heuristic nature of the recommendations, they provide a relatively safe way to operate MPLS networks, even if future allocations of IP version numbers were made for some purpose. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. This document describes a protocol intended to detect faults in the bidirectional path between two forwarding engines, including interfaces, data link(s), and to the extent possible the forwarding engines themselves, with potentially very low latency. It operates independently of media, data protocols, and routing protocols. [STANDARDS-TRACK] This document describes Connectivity Verification (CV) Types using Bidirectional Forwarding Detection (BFD) with Virtual Circuit Connectivity Verification (VCCV). VCCV provides a control channel that is associated with a pseudowire (PW), as well as the corresponding operations and management functions such as connectivity verification to be used over that control channel. [STANDARDS-TRACK] This memo utilizes IP performance metrics that are applicable to both complete paths and sub-paths, and it defines relationships to compose a complete path metric from the sub-path metrics with some accuracy with regard to the actual metrics. This is called "spatial composition" in RFC 2330. The memo refers to the framework for metric composition, and provides background and motivation for combining metrics to derive others. The descriptions of several composed metrics and statistics follow. [STANDARDS-TRACK] Many service provider service level agreements (SLAs) depend on the ability to measure and monitor performance metrics for packet loss and one-way and two-way delay, as well as related metrics such as delay variation and channel throughput. This measurement capability also provides operators with greater visibility into the performance characteristics of their networks, thereby facilitating planning, troubleshooting, and network performance evaluation. This document specifies protocol mechanisms to enable the efficient and accurate measurement of these performance metrics in MPLS networks. [STANDARDS-TRACK] This memo provides clear definitions for Active and Passive performance assessment. The construction of Metrics and Methods can be described as either "Active" or "Passive". Some methods may use a subset of both Active and Passive attributes, and we refer to these as "Hybrid Methods". This memo also describes multiple dimensions to help evaluate new methods as they emerge. This document describes a simple and efficient mechanism to detect data-plane failures in Multiprotocol Label Switching (MPLS) Label Switched Paths (LSPs). It defines a probe message called an "MPLS echo request" and a response message called an "MPLS echo reply" for returning the result of the probe. The MPLS echo request is intended to contain sufficient information to check correct operation of the data plane and to verify the data plane against the control plane, thereby localizing faults. This document obsoletes RFCs 4379, 6424, 6829, and 7537, and updates RFC 1122. Many protocols make use of points of extensibility that use constants to identify various protocol parameters. To ensure that the values in these fields do not have conflicting uses and to promote interoperability, their allocations are often coordinated by a central record keeper. For IETF protocols, that role is filled by the Internet Assigned Numbers Authority (IANA). To make assignments in a given registry prudently, guidance describing the conditions under which new values should be assigned, as well as when and how modifications to existing values can be made, is needed. This document defines a framework for the documentation of these guidelines by specification authors, in order to assure that the provided guidance for the IANA Considerations is clear and addresses the various issues that are likely in the operation of a registry. This is the third edition of this document; it obsoletes RFC 5226. This document describes the Simple Two-way Active Measurement Protocol (STAMP), which enables the measurement of both one-way and round-trip performance metrics, like delay, delay variation, and packet loss. This document describes optional extensions to Simple Two-way Active Measurement Protocol (STAMP) that enable measurement of performance metrics. The document also defines a STAMP Test Session Identifier and thus updates RFC 8762. This document specifies the Deterministic Networking (DetNet) MPLS data plane when operating over an IEEE 802.1 Time-Sensitive Networking (TSN) sub-network. This document does not define new procedures or processes. Whenever this document makes statements or recommendations, they are taken from normative text in the referenced RFCs. A DetNet (deterministic network) provides specific performance guarantees to its data flows, such as extremely low data loss rates and bounded latency (including bounded latency variation, i.e., "jitter"). As a result, securing a DetNet requires that in addition to the best practice security measures taken for any mission-critical network, additional security measures may be needed to secure the intended operation of these novel service properties. This document addresses DetNet-specific security considerations from the perspectives of both the DetNet system-level designer and component designer. System considerations include a taxonomy of relevant threats and attacks, and associations of threats versus use cases and service properties. Component-level considerations include ingress filtering and packet arrival-time violation detection. This document also addresses security considerations specific to the IP and MPLS data plane technologies, thereby complementing the Security Considerations sections of those documents.

Acknowledgments The authors extend their appreciation to for his insightful comments and productive discussion that helped to improve the document. The authors are enormously grateful to for his detailed comments and the inspiring discussion that made this document clearer and stronger. The authors recognize helpful reviews and suggestions from , , , and . And special thanks to for his fantastic help in improving the document.

Authors' Addresses Ericsson

gregimirsky@gmail.com

Huawei

mach.chen@huawei.com

Ericsson

Magyar Tudosok krt. 11. Budapest Hungary 1117 balazs.a.varga@ericsson.com