Operations, Administration, and Maintenance (OAM) in Segment Routing over IPv6 (SRv6)Cisco Systemszali@cisco.comCisco Systemscfilsfil@cisco.comSoftbanksatoru.matsushima@g.softbank.co.jpBell Canadadaniel.voyer@bell.caHuaweimach.chen@huawei.com
int
6manSRv6Segment RoutingOAMThis document describes how the existing IPv6 mechanisms for ping
and traceroute can be used in a Segment Routing over IPv6 (SRv6) network.
The document also specifies the OAM flag (O-flag) in the Segment Routing Header (SRH)
for performing controllable and predictable flow sampling from segment endpoints.
In addition, the document describes how a centralized monitoring system performs a
path continuity check between any nodes within an SRv6 domain.
Status of This Memo
This is an Internet Standards Track document.
This document is a product of the Internet Engineering Task Force
(IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by
the Internet Engineering Steering Group (IESG). Further
information on Internet Standards is available in Section 2 of
RFC 7841.
Information about the current status of this document, any
errata, and how to provide feedback on it may be obtained at
.
Copyright Notice
Copyright (c) 2022 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
() in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this
document must include Revised BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Revised BSD License.
Table of Contents
. Introduction
. Requirements Language
. Abbreviations
. Terminology and Reference Topology
. OAM Mechanisms
. OAM Flag in the Segment Routing Header
. OAM Flag Processing
. OAM Operations
. Security Considerations
. Privacy Considerations
. IANA Considerations
. References
. Normative References
. Informative References
. Illustrations
. Ping in SRv6 Networks
. Pinging an IPv6 Address via a Segment List
. Pinging a SID
. Traceroute in SRv6 Networks
. Traceroute to an IPv6 Address via a Segment List
. Traceroute to a SID
. Hybrid OAM Using the OAM Flag
. Monitoring of SRv6 Paths
Acknowledgements
Contributors
Authors' Addresses
Introduction
As Segment Routing over IPv6 (SRv6)
simply adds a new type
of Routing Extension Header, existing IPv6 OAM mechanisms can be used
in an SRv6 network. This document describes how the existing
IPv6 mechanisms for ping and traceroute can be used in an SRv6 network.
This includes illustrations of pinging an SRv6 Segment Identifier (SID) to
verify that the SID is reachable and is locally programmed at the target node.
This also includes illustrations for
tracerouting to an SRv6 SID for hop-by-hop
fault localization as well as path tracing to a SID.
This document also introduces enhancements for the OAM mechanism for SRv6
networks that allow controllable and predictable flow sampling from segment
endpoints using, e.g., the IP Flow Information Export (IPFIX) protocol
. Specifically, the document
specifies the OAM flag (O-flag) in the SRH as a marking bit in the user
packets to trigger telemetry data collection and export at the segment
endpoints.
This document also outlines how the centralized OAM technique in
can be extended for SRv6 to perform a path continuity check between
any nodes within an SRv6 domain.
Specifically, the document illustrates how a centralized monitoring system can
monitor arbitrary SRv6 paths by
creating loopback probes that
originate and terminate at the centralized monitoring system.
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED",
"MAY", and "OPTIONAL" in this document are to be interpreted as
described in BCP 14
when, and only when, they appear in all capitals, as shown here.
Abbreviations The following abbreviations are used in this document:
SID:
Segment Identifier
SL:
Segments Left
SR:
Segment Routing
SRH:
Segment Routing Header
SRv6:
Segment Routing over IPv6
PSP:
Penultimate Segment Pop
USP:
Ultimate Segment Pop
ICMPv6:
Internet Control Message Protocol for the Internet Protocol version 6
IS-IS:
Intermediate System to Intermediate System
OSPF:
Open Shortest Path First
IGP:
Interior Gateway Protocol (e.g., OSPF and IS-IS)
BGP-LS:
Border Gateway Protocol - Link State
Terminology and Reference TopologyThe terminology and
simple topology in this section are used for illustration throughout the document. Reference Topology
+--------------------------| N100 |---------------------------------+
| |
| ====== link1====== link3------ link5====== link9------ ====== |
||N1||------||N2||------| N3 |------||N4||------| N5 |---||N7||
|| ||------|| ||------| |------|| ||------| |---|| ||
====== link2====== link4------ link6======link10------ ======
| | | |
---+-- | ------ | --+---
|CE1 | +-------| N6 |---------+ |CE2 |
------ link7 | | link8 ------
------
In the reference topology:
Node j has an IPv6 loopback address 2001:db8:L:j::/128.
Nodes N1, N2, N4, and N7 are SRv6-capable nodes.
Nodes N3, N5, and N6 are IPv6 nodes that are not SRv6-capable nodes.
Such nodes are referred to as "non-SRv6-capable nodes".
CE1 and CE2 are Customer Edge devices of any data plane
capability (e.g., IPv4, IPv6, and L2).
A SID at node j with locator block 2001:db8:K::/48 and function U is represented
by 2001:db8:K:j:U::.
Node N100 is a controller.
The IPv6 address of the nth link between nodes i and j at the i side
is represented as 2001:db8:i:j:in::. For example, in , the IPv6 address of link6
(the second link between nodes N3 and N4) at node N3 is
2001:db8:3:4:32::. Similarly, the IPv6 address of link5 (the first
link between nodes N3 and N4) at node N3 is 2001:db8:3:4:31::.
2001:db8:K:j:Xin:: is explicitly allocated as the End.X SID
at node j
towards neighbor node i via the nth link between nodes i and j.
For example, 2001:db8:K:2:X31:: represents End.X at node N2 towards node N3 via link3 (the first
link between nodes N2 and N3). Similarly, 2001:db8:K:4:X52:: represents the End.X at
node N4 towards node N5 via link10 (the second
link between nodes N4 and N5). Please refer to for
a description of End.X SID.
A SID list is represented as <S1, S2, S3>, where
S1 is the first SID
to visit, S2 is the second SID to visit, and S3 is the last SID to
visit along the SR path.
(SA,DA) (S3, S2, S1; SL)(payload) represents an IPv6 packet with:
IPv6 header with source address SA, destination address DA, and
SRH as the next header
SRH with SID list <S1, S2, S3> with SegmentsLeft = SL Note the difference between the < > and () symbols:
<S1, S2, S3>
represents a SID list where S1 is the first SID and S3 is the last
SID to traverse. (S3, S2, S1; SL) represents the same SID list but
encoded in the SRH format where the rightmost SID in the SRH is the
first SID and the leftmost SID in the SRH is the last SID. When
referring to an SR Policy in a high-level use case, it is simpler
to use the <S1, S2, S3> notation. When referring to an
illustration of the detailed packet behavior, the (S3, S2, S1; SL)
notation is more convenient.
(payload) represents the payload of the packet.
OAM MechanismsThis section defines OAM enhancements for SRv6 networks.
OAM Flag in the Segment Routing Header describes the Segment
Routing Header (SRH) and how SR-capable nodes use it. The SRH
contains an 8-bit Flags field. This document defines the following bit in the
SRH Flags field to carry the O-flag:
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
| |O| |
+-+-+-+-+-+-+-+-+
Where:
O-flag:
OAM flag in the SRH Flags field defined in .
OAM Flag Processing The O-flag in the SRH is used as a marking bit in user packets to trigger
telemetry data collection and export at the segment endpoints.
An SR domain ingress edge node encapsulates packets traversing the SR
domain as defined in . The SR domain ingress edge node
MAY use the O-flag in the SRH for marking the packet to trigger
the telemetry data collection and export at the segment endpoints.
Based on local configuration, the SR domain ingress edge node
may implement a classification and sampling mechanism to mark a packet with the O-flag in the SRH.
Specification of the classification and sampling method is outside the scope of this
document.
This document does not specify the data elements that need to be exported
and the associated configurations.
Similarly, this document does not define any formats for exporting the data
elements.
Nonetheless, without the loss of generality, this document assumes that the
IP Flow Information Export (IPFIX) protocol is used for exporting
the traffic flow information from the network devices to a controller for
monitoring and analytics.
Similarly, without the loss of generality, this document assumes that requested information
elements are configured
by the management plane through data set templates (e.g., as in IPFIX
).
Implementation of the O-flag is OPTIONAL. If a node does not support the
O-flag, then it simply ignores it upon reception. If a node supports
the O-flag, it can optionally advertise its potential via
control plane protocol(s).
The following is appended to line S01 of the pseudocode
associated with the SID S (as defined in ) when N
receives a packet destined to S, S is a local SID, and the O-flag is
processed.
S01.1. IF the O-flag is set and local configuration permits
O-flag processing {
a. Make a copy of the packet.
b. Send the copied packet, along with a timestamp,
to the OAM process for telemetry data collection
and export. ;; Ref1
}
Ref1: To provide an accurate timestamp, an implementation should
copy and record the timestamp as soon as possible during packet
processing. Timestamp and any other metadata are not carried in
the packet forwarded to the next hop.
Please note that the O-flag processing happens before execution of regular
processing of the local SID S. Specifically, line S01.1 of the pseudocode
specified in this document is inserted between lines S01
and S02 of the pseudocode defined in .
Based on the
requested information elements configured
by the management plane through data set templates ,
the OAM process exports the requested information elements.
The information elements include parts of the packet header and/or parts of
the packet payload for flow identification.
The OAM process uses information elements defined in
IPFIX and Packet Sampling (PSAMP) for exporting the requested sections
of the mirrored packets.
If the penultimate segment of a segment list is a PSP SID,
telemetry data from the ultimate segment cannot be requested. This is because,
when the penultimate segment is a PSP SID,
the SRH is removed at the penultimate segment, and the O-flag is
not processed at the ultimate segment.
The processing node MUST
rate-limit the number of packets punted to the OAM process
to a configurable rate.
This is to avoid impacting the
performance of the OAM and
telemetry collection processes. Failure to implement the rate
limit can lead to a denial-of-service attack, as detailed in .
The OAM process MUST NOT process the copy of the packet or respond
to any Upper-Layer header
(like ICMP, UDP,
etc.) payload to prevent multiple evaluations of the datagram.
The OAM process is expected to be located on the routing node processing the packet.
Although the specification of the OAM process or the external controller
operations are beyond the scope of this document, the OAM process SHOULD NOT be
topologically distant from the routing node, as this is likely to create significant security
and congestion issues.
How to correlate the data collected from different nodes at an
external controller is also outside the scope of this document.
illustrates use of the O-flag for implementing
a hybrid OAM mechanism, where the "hybrid" classification
is based on .
OAM Operations IPv6 OAM operations can be performed for any SRv6 SID whose behavior
allows Upper-Layer header processing for an applicable OAM payload
(e.g., ICMP, UDP).
Ping to an SRv6 SID is used to verify
that the SID is reachable and is locally programmed at the target node.
Traceroute to a SID is used for hop-by-hop
fault localization as well as path tracing to a SID.
illustrates the ICMPv6-based ping and UDP-based traceroute mechanisms
for ping and traceroute to an SRv6 SID. Although this document only
illustrates ICMPv6-based ping and UDP-based traceroute to an SRv6 SID, the procedures are
equally applicable to other OAM mechanisms that probe an SRv6 SID
(e.g., Bidirectional Forwarding Detection (BFD) ,
Seamless BFD (S-BFD) , and Simple Two-way Active Measurement Protocol (STAMP) probe message processing
).
Specifically, as
long as local configuration allows the Upper-Layer header processing of
the applicable OAM payload for SRv6 SIDs, the existing IPv6 OAM
techniques can be used to target a probe to a (remote) SID.
IPv6 OAM operations can be performed with the target SID in the IPv6
destination address without an SRH or with an SRH where the target SID is the last segment.
In general, OAM operations to a target SID may not exercise all of its
processing depending on its behavior definition.
For example, ping to an End.X SID
only validates the SID is locally programmed at the target node
and does not validate switching to the
correct outgoing interface.
To exercise the behavior
of a target SID, the OAM operation should construct the probe in a manner
similar to a data packet that exercises the SID behavior, i.e. to include
that SID as a transit SID in either an SRH or IPv6 DA of an outer IPv6 header
or as appropriate
based on the definition of the SID behavior.
Security Considerations defines the notion of an SR domain and
use of the SRH within the SR domain.
The use of OAM procedures described in this document is restricted to an SR domain.
For example, similar to SID manipulation, O-flag manipulation is not considered
a threat within the SR domain.
Procedures for securing an SR domain are defined in Sections and of
.
As noted in ,
compromised nodes within the SR domain may mount attacks. The O-flag
may be set by an attacking node attempting a denial-of-service attack on the
OAM process at the segment endpoint node.
An implementation correctly implementing
the rate limiting described in is not susceptible to that
denial-of-service attack.
Additionally, SRH flags are protected by the Hashed Message Authentication Code (HMAC) TLV, as
described in .
Once an HMAC is generated for a segment list with the O-flag set,
it can be used for an arbitrary amount of traffic using that
segment list with the O-flag set.
The security properties of the channel used to send exported packets marked
by the O-flag will depend on the specific OAM processes used.
An on-path attacker able to observe this OAM channel could conduct
traffic analysis, or potentially eavesdropping (depending on the OAM configuration),
of this telemetry for the entire SR domain from such a vantage point.
This document does not
impose any additional security challenges to be considered beyond the
security threats described in , ,
,
, and .
Privacy Considerations The per-packet marking capabilities of the O-flag provide a granular
mechanism to collect telemetry. When this collection is deployed by an operator
with the knowledge and consent of the users, it will enable a variety of diagnostics
and monitoring to support the OAM and security operations use cases needed for
resilient network operations. However, this collection mechanism will also
provide an explicit protocol mechanism to operators for surveillance and
pervasive monitoring use cases done contrary to the user's consent.
IANA ConsiderationsIANA has registered the following in the "Segment
Routing Header Flags" subregistry in the "Internet Protocol Version
6 (IPv6) Parameters" registry:
Bit
Description
Reference
2
O-flag
RFC 9259
ReferencesNormative ReferencesKey words for use in RFCs to Indicate Requirement LevelsIn many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements.Ambiguity of Uppercase vs Lowercase in RFC 2119 Key WordsRFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings.IPv6 Segment Routing Header (SRH)Segment Routing can be applied to the IPv6 data plane using a new type of Routing Extension Header called the Segment Routing Header (SRH). This document describes the SRH and how it is used by nodes that are Segment Routing (SR) capable.Segment Routing over IPv6 (SRv6) Network ProgrammingThe Segment Routing over IPv6 (SRv6) Network Programming framework enables a network operator or an application to specify a packet processing program by encoding a sequence of instructions in the IPv6 packet header.Each instruction is implemented on one or several nodes in the network and identified by an SRv6 Segment Identifier in the packet.This document defines the SRv6 Network Programming concept and specifies the base set of SRv6 behaviors that enables the creation of interoperable overlays with underlay optimization.Informative ReferencesInternet Control Message ProtocolOSPF Version 2This memo documents version 2 of the OSPF protocol. OSPF is a link- state routing protocol. [STANDARDS-TRACK]Internet Control Message Protocol (ICMPv6) for the Internet Protocol Version 6 (IPv6) SpecificationThis document describes the format of a set of control messages used in ICMPv6 (Internet Control Message Protocol). ICMPv6 is the Internet Control Message Protocol for Internet Protocol version 6 (IPv6). [STANDARDS-TRACK]Extended ICMP to Support Multi-Part MessagesThis document redefines selected ICMP messages to support multi-part operation. A multi-part ICMP message carries all of the information that ICMP messages carried previously, as well as additional information that applications may require.Multi-part messages are supported by an ICMP extension structure. The extension structure is situated at the end of the ICMP message. It includes an extension header followed by one or more extension objects. Each extension object contains an object header and object payload. All object headers share a common format.This document further redefines the above mentioned ICMP messages by specifying a length attribute. All of the currently defined ICMP messages to which an extension structure can be appended include an "original datagram" field. The "original datagram" field contains the initial octets of the datagram that elicited the ICMP error message. Although the original datagram field is of variable length, the ICMP message does not include a field that specifies its length. Therefore, in order to facilitate message parsing, this document allocates eight previously reserved bits to reflect the length of the "original datagram" field.The proposed modifications change the requirements for ICMP compliance. The impact of these changes on compliant implementations is discussed, and new requirements for future implementations are presented.This memo updates RFC 792 and RFC 4443. [STANDARDS-TRACK]Packet Sampling (PSAMP) Protocol SpecificationsThis document specifies the export of packet information from a Packet SAMPling (PSAMP) Exporting Process to a PSAMP Collecting Process. For export of packet information, the IP Flow Information eXport (IPFIX) protocol is used, as both the IPFIX and PSAMP architecture match very well, and the means provided by the IPFIX protocol are sufficient. The document specifies in detail how the IPFIX protocol is used for PSAMP export of packet information. [STANDARDS-TRACK]Extending ICMP for Interface and Next-Hop IdentificationThis memo defines a data structure that can be appended to selected ICMP messages. The ICMP extension defined herein can be used to identify any combination of the following: the IP interface upon which a datagram arrived, the sub-IP component of an IP interface upon which a datagram arrived, the IP interface through which the datagram would have been forwarded had it been forwardable, and the IP next hop to which the datagram would have been forwarded.Devices can use this ICMP extension to identify interfaces and their components by any combination of the following: ifIndex, IPv4 address, IPv6 address, name, and MTU. ICMP-aware devices can use these extensions to identify both numbered and unnumbered interfaces. [STANDARDS-TRACK]Bidirectional Forwarding Detection (BFD)This document describes a protocol intended to detect faults in the bidirectional path between two forwarding engines, including interfaces, data link(s), and to the extent possible the forwarding engines themselves, with potentially very low latency. It operates independently of media, data protocols, and routing protocols. [STANDARDS-TRACK]Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow InformationThis document specifies the IP Flow Information Export (IPFIX) protocol, which serves as a means for transmitting Traffic Flow information over the network. In order to transmit Traffic Flow information from an Exporting Process to a Collecting Process, a common representation of flow data and a standard means of communicating them are required. This document describes how the IPFIX Data and Template Records are carried over a number of transport protocols from an IPFIX Exporting Process to an IPFIX Collecting Process. This document obsoletes RFC 5101.Information Model for IP Flow Information Export (IPFIX)This document defines the data types and management policy for the information model for the IP Flow Information Export (IPFIX) protocol. This information model is maintained as the IANA "IPFIX Information Elements" registry, the initial contents of which were defined by RFC 5102. This information model is used by the IPFIX protocol for encoding measured traffic information and information related to the traffic Observation Point, the traffic Metering Process, and the Exporting Process. Although this model was developed for the IPFIX protocol, it is defined in an open way that allows it to be easily used in other protocols, interfaces, and applications. This document obsoletes RFC 5102.Active and Passive Metrics and Methods (with Hybrid Types In-Between)This memo provides clear definitions for Active and Passive performance assessment. The construction of Metrics and Methods can be described as either "Active" or "Passive". Some methods may use a subset of both Active and Passive attributes, and we refer to these as "Hybrid Methods". This memo also describes multiple dimensions to help evaluate new methods as they emerge.Seamless Bidirectional Forwarding Detection (S-BFD)This document defines Seamless Bidirectional Forwarding Detection (S-BFD), a simplified mechanism for using BFD with a large proportion of negotiation aspects eliminated, thus providing benefits such as quick provisioning, as well as improved control and flexibility for network nodes initiating path monitoring.This document updates RFC 5880.Segment Routing ArchitectureSegment Routing (SR) leverages the source routing paradigm. A node steers a packet through an ordered list of instructions, called "segments". A segment can represent any instruction, topological or service based. A segment can have a semantic local to an SR node or global within an SR domain. SR provides a mechanism that allows a flow to be restricted to a specific topological path, while maintaining per-flow state only at the ingress node(s) to the SR domain.SR can be directly applied to the MPLS architecture with no change to the forwarding plane. A segment is encoded as an MPLS label. An ordered list of segments is encoded as a stack of labels. The segment to process is on the top of the stack. Upon completion of a segment, the related label is popped from the stack.SR can be applied to the IPv6 architecture, with a new type of routing header. A segment is encoded as an IPv6 address. An ordered list of segments is encoded as an ordered list of IPv6 addresses in the routing header. The active segment is indicated by the Destination Address (DA) of the packet. The next active segment is indicated by a pointer in the new routing header.A Scalable and Topology-Aware MPLS Data-Plane Monitoring SystemThis document describes features of an MPLS path monitoring system and related use cases. Segment-based routing enables a scalable and simple method to monitor data-plane liveliness of the complete set of paths belonging to a single domain. The MPLS monitoring system adds features to the traditional MPLS ping and Label Switched Path (LSP) trace, in a very complementary way. MPLS topology awareness reduces management and control-plane involvement of Operations, Administration, and Maintenance (OAM) measurements while enabling new OAM features.BGP - Link State (BGP-LS) Advertisement of IGP Traffic Engineering Performance Metric ExtensionsThis document defines new BGP - Link State (BGP-LS) TLVs in order to carry the IGP Traffic Engineering Metric Extensions defined in the IS-IS and OSPF protocols.Data Fields for In Situ Operations, Administration, and Maintenance (IOAM)In situ Operations, Administration, and Maintenance (IOAM) collects operational and telemetry information in the packet while the packet traverses a path between two points in the network. This document discusses the data fields and associated data types for IOAM. IOAM-Data-Fields can be encapsulated into a variety of protocols, such as Network Service Header (NSH), Segment Routing, Generic Network Virtualization Encapsulation (Geneve), or IPv6. IOAM can be used to complement OAM mechanisms based on, e.g., ICMP or other types of probe packets.Performance Measurement Using Simple TWAMP (STAMP) for Segment Routing NetworksCisco Systems, Inc.Cisco Systems, Inc.Bell CanadaHuaweiColtNokiaWork in ProgressIllustrations This appendix shows how some of the
existing IPv6 OAM mechanisms can be used in an SRv6 network. It also
illustrates an OAM mechanism for
performing controllable and predictable flow sampling from segment
endpoints. How the centralized OAM technique in
can be extended for SRv6 is also described in this appendix.
Ping in SRv6 Networks The existing mechanism to perform the reachability checks,
along the shortest path, continues to work without any modification.
Any IPv6 node (SRv6-capable or non-SRv6-capable) can initiate, transit,
and egress a ping packet.
The following subsections outline some additional use cases of ICMPv6 ping in
SRv6 networks.
Pinging an IPv6 Address via a Segment List If an SRv6-capable ingress node wants to ping an IPv6 address via an
arbitrary segment list <S1, S2, S3>, it needs to initiate an ICMPv6
ping with an SR header containing the SID list <S1, S2, S3>. This is
illustrated using the topology in . The user issues a ping from node N1 to a
loopback of node N5 via segment list <2001:db8:K:2:X31::, 2001:db8:K:4:X52::>.
The SID behavior used in the example is End.X,
as described in , but the procedure is
equally applicable to any other (transit) SID type.
contains sample output for a ping request initiated at node
N1 to a loopback address of node N5 via segment list <2001:db8:K:2:X31::,
2001:db8:K:4:X52::>.
Sample Ping Output at an SRv6-Capable Node
> ping 2001:db8:L:5:: via segment list 2001:db8:K:2:X31::,
2001:db8:K:4:X52::
Sending 5, 100-byte ICMPv6 Echos to B5::, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 0.625
/0.749/0.931 ms
All transit nodes process the echo request message like any other
data packet carrying an SR header and hence do not require any change.
Similarly, the egress node does not
require any change to process the ICMPv6 echo request. For example,
in the example in :
Node N1 initiates an ICMPv6 ping packet with the SRH as follows:
(2001:db8:L:1::, 2001:db8:K:2:X31::)
(2001:db8:L:5::, 2001:db8:K:4:X52::, 2001:db8:K:2:X31::, SL=2,
NH = ICMPv6)(ICMPv6 Echo Request).
Node N2, which is an SRv6-capable node, performs the standard
SRH processing. Specifically, it executes the End.X behavior
indicated by the 2001:db8:K:2:X31:: SID and forwards the packet on link3 to node N3.
Node N3, which is a non-SRv6-capable node, performs the standard
IPv6 processing. Specifically, it forwards the echo request
based on DA 2001:db8:K:4:X52:: in the IPv6 header.
Node N4, which is an SRv6-capable node, performs the standard
SRH processing. Specifically, it observes the End.X behavior
(2001:db8:K:4:X52::) and forwards the packet on link10 towards node N5.
If 2001:db8:K:4:X52:: is a PSP SID,
the penultimate node (node N4) does not, should not, and cannot differentiate
between the data packets and OAM probes.
Specifically, if 2001:db8:K:4:X52:: is a PSP SID,
node N4 executes the SID like any other data packet with DA = 2001:db8:K:4:X52::
and removes the SRH.
The echo request packet at node N5 arrives as an IPv6 packet with or
without an SRH. If node N5 receives the packet with an SRH, it skips SRH processing (SL=0).
In either case, node N5 performs the
standard ICMPv6 processing on the echo request and responds with the
echo reply message to node N1. The echo reply message is IP routed.
Pinging a SID
The ping mechanism described above can also be used to perform SID
reachability checks and to validate that the SID is locally programmed at
the target node.
This is explained in the
following example. The example uses ping to an End SID, as described in ,
but the procedure is
equally applicable to ping any other SID behaviors.
Consider the example where the user wants to ping a remote
SID 2001:db8:K:4::, via 2001:db8:K:2:X31::, from node N1.
The ICMPv6 echo request is processed at the individual nodes
along the path as follows:
Node N1 initiates an ICMPv6 ping packet with the SRH as follows:
(2001:db8:L:1::, 2001:db8:K:2:X31::)
(2001:db8:K:4::, 2001:db8:K:2:X31::; SL=1;
NH=ICMPv6)(ICMPv6 Echo Request).
Node N2, which is an SRv6-capable node, performs the standard
SRH processing. Specifically, it executes the End.X behavior
indicated by the 2001:db8:K:2:X31:: SID on the echo request packet. If
2001:db8:K:2:X31:: is a PSP SID, node N4 executes the SID like any
other data packet with DA = 2001:db8:K:2:X31:: and removes the
SRH.
Node N3, which is a non-SRv6-capable node, performs
the standard IPv6 processing. Specifically, it forwards the
echo request based on DA = 2001:db8:K:4:: in the IPv6 header.
When node N4 receives the packet, it
processes the target SID (2001:db8:K:4::).
If the target SID (2001:db8:K:4::) is not locally instantiated
and does not represent a local interface,
the packet is discarded
If the target SID (2001:db8:K:4::) is locally instantiated or
represents a local interface, the node processes
the Upper-Layer header.
As part of the Upper-Layer header processing, node N4 responds
to the ICMPv6 echo request message with an
echo reply message. The echo reply message is IP routed.
Traceroute in SRv6 Networks The existing traceroute
mechanisms, along the shortest path, continue to work without any modification.
Any IPv6 node (SRv6-capable or a non-SRv6-capable) can initiate, transit,
and egress a traceroute probe.
The following subsections outline some additional use cases of traceroute
in SRv6 networks.
Traceroute to an IPv6 Address via a Segment List If an SRv6-capable ingress node wants to traceroute to an IPv6 address
via an arbitrary segment list <S1, S2, S3>, it needs to initiate
a traceroute probe with an SR header containing the SID list
<S1, S2, S3>. The user issues a traceroute
from node N1 to a loopback of node N5 via segment list
<2001:db8:K:2:X31::, 2001:db8:K:4:X52::>.
The SID behavior used in the example is End.X, as described in
,
but the procedure is equally applicable to any other (transit) SID
type.
contains sample output for the traceroute
request.
Sample Traceroute Output at an SRv6-Capable Node
> traceroute 2001:db8:L:5:: via segment list 2001:db8:K:2:X31::,
2001:db8:K:4:X52::
Tracing the route to 2001:db8:L:5::
1 2001:db8:2:1:21:: 0.512 msec 0.425 msec 0.374 msec
DA: 2001:db8:K:2:X31::,
SRH:(2001:db8:L:5::, 2001:db8:K:4:X52::, 2001:db8:K:2:X31::, SL=2)
2 2001:db8:3:2:31:: 0.721 msec 0.810 msec 0.795 msec
DA: 2001:db8:K:4:X52::,
SRH:(2001:db8:L:5::, 2001:db8:K:4:X52::, 2001:db8:K:2:X31::, SL=1)
3 2001:db8:4:3::41:: 0.921 msec 0.816 msec 0.759 msec
DA: 2001:db8:K:4:X52::,
SRH:(2001:db8:L:5::, 2001:db8:K:4:X52::, 2001:db8:K:2:X31::, SL=1)
4 2001:db8:5:4::52:: 0.879 msec 0.916 msec 1.024 msec
DA: 2001:db8:L:5::
In the sample traceroute output, the information displayed at each hop
is obtained using the contents of the "Time Exceeded" or
"Destination Unreachable" ICMPv6 responses. These ICMPv6 responses
are IP routed.
In the sample traceroute output, the information for link3 is
returned by node N3, which is a
non-SRv6-capable node. Nonetheless, the ingress node is able to display
SR header contents as the packet travels through the non-SRv6-capable node.
This is because the "Time Exceeded" ICMPv6 message can
contain as much of the invoking packet as possible without the
ICMPv6 packet exceeding the minimum IPv6 MTU . The SR
header is included in these ICMPv6 messages initiated by the
non-SRv6-capable transit nodes that are not running SRv6 software.
Specifically, a node generating an ICMPv6 message containing a copy of
the invoking packet does not need to understand the extension
header(s) in the invoking packet.
The segment list information returned for the first hop is returned by node N2,
which is an SRv6-capable node. Just like for the second hop, the ingress node
is able to display SR header contents for the first hop.
There is no difference in processing of the traceroute probe at an
SRv6-capable and a non-SRv6-capable node. Similarly, both SRv6-capable and
non-SRv6-capable nodes may use the address of the interface on
which probe was received as the source address in the ICMPv6
response. ICMPv6 extensions defined in can be used to
display information about the IP interface through which the
datagram would have been forwarded had it been forwardable, the
IP next hop to which the datagram would have been forwarded, the IP
interface upon which the datagram arrived, and the sub-IP component of an
IP interface upon which the datagram arrived.
The IP address of the interface on which the traceroute probe was received
is useful. This information can also be used to verify if SIDs
2001:db8:K:2:X31:: and 2001:db8:K:4:X52:: are executed correctly by nodes N2 and N4,
respectively. Specifically, the information displayed for the second hop
contains the incoming interface address 2001:db8:2:3:31:: at node N3.
This matches the expected interface bound to End.X behavior
2001:db8:K:2:X31:: (link3). Similarly, the information displayed for the fourth hop
contains the incoming interface address 2001:db8:4:5::52:: at node N5.
This matches the expected interface bound to the End.X behavior
2001:db8:K:4:X52:: (link10).
Traceroute to a SIDThe mechanism to traceroute an IPv6 address via a segment list
described in the previous section can also be used to traceroute a
remote SID behavior, as explained in the following example. The
example uses traceroute to an End SID, as described in , but the procedure is equally
applicable to tracerouting any other SID behaviors.
Please note that traceroute to a SID is
exemplified using UDP probes. However, the procedure is equally
applicable to other implementations of traceroute mechanism.
The UDP encoded message to traceroute a SID would use the UDP ports
assigned by IANA for "traceroute use".
Consider the example where the user wants to traceroute a remote SID
2001:db8:K:4::, via 2001:db8:K:2:X31::, from node N1. The
traceroute probe is processed at the individual nodes along the path
as follows:
Node N1 initiates a traceroute probe packet as follows
(2001:db8:L:1::, 2001:db8:K:2:X31::)
(2001:db8:K:4::, 2001:db8:K:2:X31::; SL=1; NH=UDP)(Traceroute probe).
The first traceroute probe is sent with the hop-count value set to 1.
The hop-count value is incremented by 1 for each subsequent traceroute probe.
When node N2 receives the packet with hop-count = 1, it
processes the hop-count expiry. Specifically, node N2
responds with the ICMPv6 message with type "Time Exceeded" and code
"hop limit exceeded in transit". The ICMPv6 response
is IP routed.
When node N2 receives the packet with hop-count > 1, it
performs the standard SRH processing. Specifically, it executes
the End.X behavior indicated by the
2001:db8:K:2:X31:: SID on the traceroute probe.
If 2001:db8:K:2:X31:: is a PSP SID,
node N2 executes the SID like any other data packet with DA = 2001:db8:K:2:X31::
and removes the SRH.
When node N3, which is a non-SRv6-capable node, receives the packet
with hop-count = 1, it processes the
hop-count expiry. Specifically, node N3 responds with the
ICMPv6 message with type "Time Exceeded" and code "Hop limit
exceeded in transit". The ICMPv6 response is IP routed.
When node N3, which is a non-SRv6-capable node, receives the packet
with hop-count > 1, it performs the standard IPv6 processing.
Specifically, it forwards the traceroute probe based on DA
2001:db8:K:4:: in the IPv6 header.
When node N4 receives the packet with DA set to the local SID 2001:db8:K:4::, it
processes the End SID.
If the target SID (2001:db8:K:4::) is not locally instantiated and
does not represent a local interface, the packet is discarded.
If the target SID (2001:db8:K:4::) is locally instantiated or represents a
local interface, the node processes
the Upper-Layer header.
As part of the Upper-Layer header processing, node N4 responds
with the ICMPv6 message with type "Destination Unreachable" and code
"Port Unreachable". The ICMPv6 response
is IP routed.
displays a sample traceroute output for this example.
Sample Output for Hop-by-Hop Traceroute to a SID
> traceroute 2001:db8:K:4:X52:: via segment list 2001:db8:K:2:X31::
Tracing the route to SID 2001:db8:K:4:X52::
1 2001:db8:2:1:21:: 0.512 msec 0.425 msec 0.374 msec
DA: 2001:db8:K:2:X31::,
SRH:(2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=1)
2 2001:db8:3:2:21:: 0.721 msec 0.810 msec 0.795 msec
DA: 2001:db8:K:4:X52::,
SRH:(2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=0)
3 2001:db8:4:3:41:: 0.921 msec 0.816 msec 0.759 msec
DA: 2001:db8:K:4:X52::,
SRH:(2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=0)
Hybrid OAM Using the OAM Flag This section illustrates a hybrid OAM mechanism using
the O-flag. Without loss of the generality, the illustration
assumes node N100 is a centralized controller.
This illustration is different from the "in situ OAM" defined in . This is because in situ OAM records
operational and telemetry information in the packet as the packet
traverses a path between two points in the network . The illustration in this subsection does not require
the recording of OAM data in the packet.
The illustration does not assume any formats for exporting the data
elements or the data elements that need to be exported.
The illustration assumes system clocks among all nodes in the SR domain are synchronized.
Consider the example where the user wants to monitor sampled IPv4
VPN 999 traffic going from CE1 to CE2 via a low-latency SR Policy P installed
at node N1.
To exercise a low-latency path, the SR Policy P forces the packet via segments
2001:db8:K:2:X31:: and 2001:db8:K:4:X52::.
The VPN SID at node N7 associated with VPN 999 is 2001:db8:K:7:DT999::.
2001:db8:K:7:DT999:: is a USP SID.
Nodes N1, N4, and N7 are capable of processing the O-flag, but
node N2 is not capable of processing the O-flag.
Node N100 is the centralized controller capable of processing and correlating
the copy of the packets sent from nodes N1, N4, and N7.
Node N100 is aware of O-flag processing capabilities.
Node N100, with help from nodes N1, N4, and N7, implements a hybrid
OAM mechanism using the O-flag as follows:
A packet P1 is sent from CE1 to node N1. The packet is:P1: (IPv4 header)(payload)
Node N1 steers packet P1 through the SR Policy P.
Based on local configuration, node N1 also implements logic to sample
traffic steered through SR Policy P for hybrid OAM purposes.
Specification for the sampling logic is beyond the scope of this document.
Consider the case where packet P1 is classified as a packet to be monitored
via the hybrid OAM.
Node N1 sets the O-flag during the encapsulation required by SR Policy P.
As part of setting the O-flag, node N1 also sends a timestamped copy
of packet P1 to a local
OAM process. The packet is:P1: (2001:db8:L:1::, 2001:db8:K:2:X31::) (2001:db8:K:7:DT999::,
2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=2; O-flag=1;
NH=IPv4)(IPv4 header)(payload)The local OAM process sends a full or partial copy of
packet P1 to node N100.
The OAM process includes the
recorded timestamp, additional
OAM information (like incoming and outgoing interface), and
any applicable metadata.
Node N1 forwards the original packet towards the next
segment 2001:db8:K:2:X31::.
When node N2 receives the packet with the O-flag set, it
ignores the O-flag. This is because node N2 is not capable of
processing the O-flag. Node N2 performs the standard SRv6 SID and
SRH processing. Specifically, it executes the End.X behavior
indicated by the 2001:db8:K:2:X31:: SID and forwards packet P1 over
link3 towards node N3. The packet is:P1: (2001:db8:L:1::, 2001:db8:K:4:X52::) (2001:db8:K:7:DT999::,
2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=1; O-flag=1;
NH=IPv4)(IPv4 header)(payload)
When node N3, which is a non-SRv6-capable node, receives packet P1, it performs the standard IPv6 processing.
Specifically, it forwards packet P1 based on DA
2001:db8:K:4:X52:: in the IPv6 header.
When node N4 receives packet P1, it processes the O-flag. The packet is:P1: (2001:db8:L:1::, 2001:db8:K:4:X52::)
(2001:db8:K:7:DT999::, 2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=1; O-flag=1;
NH=IPv4)(IPv4 header)(payload)
As part of processing the O-flag, it sends a timestamped copy of
the packet to a local OAM process.
Based on local configuration, the local OAM process sends a full or partial
copy of packet
P1 to node N100. The OAM process includes the
recorded timestamp, additional
OAM information (like incoming and outgoing interface, etc.), and
any applicable metadata.
Node N4 performs the standard SRv6 SID and SRH processing on the original packet P1.
Specifically, it executes
the End.X behavior indicated by the 2001:db8:K:4:X52:: SID and forwards packet P1
over link10 towards node N5. The packet is:P1: (2001:db8:L:1::, 2001:db8:K:7:DT999::)
(2001:db8:K:7:DT999::, 2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=0; O-flag=1;
NH=IPv4)(IPv4 header)(payload)
When node N5, which is a non-SRv6-capable node, receives packet P1,
it performs the standard IPv6 processing.
Specifically, it forwards the packet based on DA
2001:db8:K:7:DT999:: in the IPv6 header.
When node N7 receives packet P1, it processes the O-flag. The packet is:P1: (2001:db8:L:1::, 2001:db8:K:7:DT999::)
(2001:db8:K:7:DT999::, 2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=0; O-flag=1;
NH=IPv4)(IPv4 header)(payload)
As part of processing the O-flag, it sends a timestamped copy of
the packet to a local OAM process.
The local OAM process sends a full or partial copy of packet
P1 to node N100. The OAM process includes the
recorded timestamp, additional
OAM information (like incoming and outgoing interface, etc.), and
any applicable metadata.
Node N7 performs the standard SRv6 SID and SRH processing on the original packet P1.
Specifically, it executes the VPN SID indicated by the 2001:db8:K:7:DT999:: SID
and, based on lookup in table 100, forwards packet P1
towards CE2. The packet is:P1: (IPv4 header)(payload)
Node N100 processes and correlates the copy of the packets
sent from nodes N1, N4, and N7 to find segment-by-segment delays and
provide other hybrid OAM information related to packet P1.
For segment-by-segment delay computation, it is assumed that clocks
are synchronized across the SR domain.
The process continues for any other sampled packets.
Monitoring of SRv6 Paths In the recent past, network operators demonstrated interest in performing
network OAM functions in a centralized manner.
describes such a centralized OAM mechanism. Specifically,
describes a procedure that can be used to perform path continuity
checks between any nodes within an SR domain from a centralized
monitoring system. However, while focuses on SR networks with MPLS data
plane, this document describes how
the concept can be used to perform path monitoring in an SRv6 network
from a centralized controller.
In the reference topology in , node N100 uses an IGP protocol
like OSPF or IS-IS to get a view of the topology within the IGP domain.
Node N100 can also use BGP-LS to get the complete view of an inter-domain
topology. The controller leverages the visibility of
the topology to monitor the paths between the various endpoints.
Node N100 advertises an End
SID 2001:db8:K:100:1::. To monitor any
arbitrary SRv6 paths, the controller can create a loopback probe that originates and
terminates on node N100. To distinguish between a failure in the monitored path
and loss of connectivity between the controller and the network,
node N100 runs a suitable mechanism to monitor its connectivity to the monitored network.
The following example illustrates loopback probes in which node N100
needs to verify a
segment list <2001:db8:K:2:X31::, 2001:db8:K:4:X52::>:
Node N100 generates an OAM packet (2001:db8:L:100::,
2001:db8:K:2:X31::)(2001:db8:K:100:1::, 2001:db8:K:4:X52::, 2001:db8:K:2:X31::,
SL=2)(OAM Payload). The controller routes the probe packet towards the first
segment, which is 2001:db8:K:2:X31::.
Node N2 executes the End.X behavior indicated by the 2001:db8:K:2:X31:: SID and
forwards the packet
(2001:db8:L:100::,
2001:db8:K:4:X52::)(2001:db8:K:100:1::, 2001:db8:K:4:X52::, 2001:db8:K:2:X31::,
SL=1)(OAM Payload) on link3 to node N3.
Node N3, which is a non-SRv6-capable node, performs the standard
IPv6 processing. Specifically, it forwards the packet
based on DA 2001:db8:K:4:X52:: in the IPv6 header.
Node N4 executes the End.X behavior indicated by the 2001:db8:K:4:X52:: SID and
forwards the packet
(2001:db8:L:100::,
2001:db8:K:100:1::)(2001:db8:K:100:1::, 2001:db8:K:4:X52::, 2001:db8:K:2:X31::,
SL=0)(OAM Payload) on link10 to node N5.
Node N5, which is a non-SRv6-capable node, performs the standard
IPv6 processing. Specifically, it forwards the packet
based on DA 2001:db8:K:100:1:: in the IPv6 header.
Node N100 executes the standard SRv6 END behavior. It
decapsulates the header and consumes the probe for OAM processing. The information
in the OAM payload is used to detect missing probes, round-trip delay, etc.
The OAM payload type or
the information carried in the OAM probe is a local implementation
decision at the controller and is outside the scope of this document.
Acknowledgements The authors would like to thank , ,
, , , , and
for their review comments. ContributorsThe following people contributed to this document:
Bloomberg LProbert@raszuk.netIndividualjohn@leddy.netLinkedIngdawra.ietf@gmail.comProximusbart.peirens@proximus.comCisco Systems, Inc.naikumar@cisco.comCisco Systems, Inc.cpignata@cisco.comCisco Systems, Inc.rgandhi@cisco.comCisco Systems, Inc.fbrockne@cisco.comCisco Systems, Inc.ddukes@cisco.comHuaweichengli13@huawei.comIndividualfaisal.ietf@gmail.comAuthors' AddressesCisco Systemszali@cisco.comCisco Systemscfilsfil@cisco.comSoftbanksatoru.matsushima@g.softbank.co.jpBell Canadadaniel.voyer@bell.caHuaweimach.chen@huawei.com