IPv6 Application of the Alternate-Marking MethodHuaweiRiesstrasse, 25Munich80992Germanygiuseppe.fioccola@huawei.comHuawei156 Beiqing Rd.Beijing100095Chinazhoutianran@huawei.comTelecom Italiamauro.cociglio@outlook.comChina Mobile32 Xuanwumenxi Ave.Beijing100032Chinaqinfengwei@chinamobile.comChina Unicom9 Shouti South Rd.Beijing100089Chinapangran@chinaunicom.cn
Internet
6MANExtensionHeaderOptionDestinationHop-By-HopPerformanceMeasurementMonitoringPassiveHybridLossDelayDelay VariationMultipointClusterClosed-LoopThis document describes how the Alternate-Marking Method can be used
as a passive performance measurement tool in an IPv6 domain. It defines an
Extension Header Option to encode Alternate-Marking information in
both the Hop-by-Hop Options Header and Destination Options Header.Status of This Memo
This is an Internet Standards Track document.
This document is a product of the Internet Engineering Task Force
(IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by
the Internet Engineering Steering Group (IESG). Further
information on Internet Standards is available in Section 2 of
RFC 7841.
Information about the current status of this document, any
errata, and how to provide feedback on it may be obtained at
.
Copyright Notice
Copyright (c) 2022 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
() in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this
document must include Revised BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Revised BSD License.
Table of Contents
. Introduction
. Terminology
. Requirements Language
. Alternate-Marking Application to IPv6
. Controlled Domain
. Alternate-Marking Measurement Domain
. Definition of the AltMark Option
. Data Fields Format
. Use of the AltMark Option
. Alternate-Marking Method Operation
. Packet Loss Measurement
. Packet Delay Measurement
. Flow Monitoring Identification
. Multipoint and Clustered Alternate Marking
. Data Collection and Calculation
. Security Considerations
. IANA Considerations
. References
. Normative References
. Informative References
Acknowledgements
Authors' Addresses
Introduction and
describe a passive performance measurement method, which can be used to measure packet loss,
latency, and jitter on live traffic. Since this method is based on marking consecutive
batches of packets, the method is often referred to as the Alternate-Marking Method.This document defines how the Alternate-Marking Method can be used to measure
performance metrics in IPv6. The rationale is to apply the Alternate-Marking methodology to IPv6 and
therefore allow detailed packet loss, delay, and delay variation measurements both hop by hop and end to end
to exactly locate the issues in an IPv6 network.Alternate Marking is an on-path telemetry technique and consists of synchronizing the measurements
in different points of a network by switching the value of a marking bit and therefore dividing the packet flow
into batches. Each batch represents a measurable entity recognizable by all network nodes along the path.
By counting the number of packets in each batch and comparing the values measured by different nodes,
it is possible to precisely measure the packet loss. Similarly, the alternation of the values
of the marking bits can be used as a time reference to calculate the delay and delay variation.
The Alternate-Marking operation is further described in .This document introduces a TLV (type-length-value) that can be encoded in the Options Headers
(Hop-by-Hop or Destination), according to , for the purpose of the
Alternate-Marking Method application in an IPv6 domain.The Alternate-Marking Method MUST be applied to IPv6 only in a controlled environment, as further described
in . provides further discussion of network behaviors
that can be applied only within limited domains.The threat model for the application of the Alternate-Marking Method in an IPv6 domain is reported in
.TerminologyThis document uses the terms related to the Alternate-Marking Method
as defined in and .Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED",
"MAY", and "OPTIONAL" in this document are to be interpreted as
described in BCP 14
when, and only when, they appear in all capitals, as shown here.
Alternate-Marking Application to IPv6The Alternate-Marking Method requires a marking field. Several alternatives
could be considered such as IPv6 Extension Headers, IPv6 Address, and Flow Label.
But, it is necessary to analyze the drawbacks for all the available possibilities,
more specifically:
reusing an existing Extension Header for Alternate Marking leads to a
non-optimized implementation;
using the IPv6 destination address to encode the Alternate-Marking processing
is very expensive; and
using the IPv6 Flow Label for Alternate Marking conflicts with the utilization
of the Flow Label for load distribution purposes .
In the end, a Hop-by-Hop or a Destination Option is the best choice.The approach for the Alternate-Marking application to IPv6 specified in this memo
is compliant with . It involves the following operations:
The source node is the only one that writes the Options Header to mark alternately
the flow (for both the Hop-by-Hop and Destination Option). The intermediate nodes and destination node
MUST only read the marking values of the Option without modifying the Options Header.
In case of a Hop-by-Hop Options Header carrying Alternate-Marking bits, the Options Header is not
inserted or deleted on the path, but it can be read by any node along the path. The intermediate nodes
may be configured to support this Option or not, and the measurement can be done only for
the nodes configured to read the Option. As further discussed in ,
the presence of the Hop-by-Hop Option should not affect the traffic throughput both on nodes
that do not recognize this Option and on the nodes that support it. However, it is worth
mentioning that there is a difference between theory and practice. Indeed, in a real implementation,
it is possible for packets with a Hop-by-Hop Option to be skipped or processed in the slow path.
While some proposals are trying to address this problem and make Hop-by-Hop Options more practical
(see and ), these aspects
are out of the scope for this document.
In case of a Destination Options Header carrying Alternate-Marking bits, it is not
processed, inserted, or deleted by any node along the path until the packet reaches
the destination node. Note that, if there is also a Routing Header (RH), any visited
destination in the route list can process the Options Header.
A Hop-by-Hop Options Header is also useful to signal to routers on the path to process the
Alternate Marking. However, as said, routers will only examine this Option if properly configured.The optimization of both implementation and the scaling of the Alternate-Marking Method is
also considered, and a way to identify flows is required. The Flow Monitoring Identification
(FlowMonID) field, as introduced in , goes in this direction, and it is used to identify
a monitored flow.The FlowMonID is different from the Flow Label field of the IPv6 header .
The Flow Label field in the IPv6 header is used by a source to label sequences of packets to be treated
in the network as a single flow and, as reported in , it can be used
for load balancing (LB) and equal-cost multipath (ECMP). The reuse of the Flow Label field for identifying monitored flows
is not considered because it may change the application intent and forwarding behavior. Also, the Flow Label
may be changed en route, and this may also invalidate the integrity of the measurement. Those reasons make the definition
of the FlowMonID necessary for IPv6. Indeed, the FlowMonID is designed and only used to identify the monitored flow.
Flow Label and FlowMonID within the same packet are totally disjoint, have different scopes,
are used to identify flows based on different criteria, and are intended for different use cases.The rationale for the FlowMonID is further discussed in . This 20-bit field
allows easy and flexible identification of the monitored flow and enables improved measurement correlation
and finer granularity since it can be used in combination with the conventional TCP/IP 5-tuple to identify a flow.
An important point that will be discussed in is the uniqueness of the FlowMonID
and how to allow disambiguation of the FlowMonID in case of collision.The following section highlights an important requirement for the application of the Alternate Marking to IPv6.
The concept of the controlled domain is explained and is considered an essential precondition, as also highlighted
in .Controlled DomainIPv6 has much more flexibility than IPv4 and innovative applications have been proposed, but
for security and compatibility reasons, some of these applications are limited to a controlled environment.
This is also the case of the Alternate-Marking application to IPv6 as assumed hereinafter.
In this regard, reports further examples of specific limited domain solutions.The IPv6 application of the Alternate-Marking Method MUST be deployed in a controlled domain.
It is not common that the user traffic originates and terminates within the controlled domain, as also noted in
. For this reason, it will typically only be applicable in an overlay network,
where user traffic is encapsulated at one domain border and decapsulated at the other domain border, and the encapsulation
incorporates the relevant extension header for Alternate Marking.
This requirement also implies that an implementation MUST filter packets that carry Alternate-Marking
data and are entering or leaving the controlled domain.A controlled domain is a managed network where it is required to select, monitor, and control the access to the network
by enforcing policies at the domain boundaries in order to discard undesired external packets entering the domain
and check the internal packets leaving the domain. It does not necessarily mean that a controlled domain is a single administrative domain
or a single organization. A controlled domain can correspond to a single administrative domain or can be composed by
multiple administrative domains under a defined network management. Indeed, some scenarios may imply that the Alternate-Marking Method
involves more than one domain, but in these cases, it is RECOMMENDED that the multiple domains create a whole controlled domain
while traversing the external domain by employing IPsec authentication and encryption or
other VPN technology that provides full packet confidentiality and integrity protection. In a few words, it must be possible
to control the domain boundaries and eventually use specific precautions if the traffic traverses the Internet.The security considerations reported in also highlight this requirement.Alternate-Marking Measurement DomainThe Alternate-Marking measurement domain can overlap with the controlled domain or may be a subset
of the controlled domain. The typical scenarios for the application of the Alternate-Marking Method
depend on the controlled domain boundaries; in particular:
The user equipment can be the starting or ending node only when/if it is fully managed and belongs
to the controlled domain. In this case, the user-generated IPv6 packets contain the Alternate-Marking data.
But, in practice, this is not common due to the fact that the user equipment cannot be totally secured in the majority of cases.
The Customer Premises Equipment (CPE) or the Provider Edge (PE) routers are most likely to be the starting or
ending nodes since they can be border routers of the controlled domain. For instance, the CPE, which connects the user's premises
with the service provider's network, belongs to a controlled domain only if it is managed by the service provider and
if additional security measures are taken to keep it trustworthy.
Typically, the CPE or the PE can encapsulate a received packet in an outer IPv6 header, which contains the Alternate-Marking data.
They are also able to filter and drop packets from outside of the domain with inconsistent fields
to make effective the relevant security rules at the domain boundaries; for example, a simple security check
can be to insert the Alternate-Marking data if and only if the destination is within the controlled domain.
Definition of the AltMark OptionThe definition of a TLV for the Extension Header Option,
carrying the data fields dedicated to the Alternate-Marking Method,
is reported below.Data Fields FormatThe following figure shows the data fields format for enhanced
Alternate-Marking TLV (AltMark). This AltMark data can be encapsulated in the IPv6 Options Headers
(Hop-by-Hop or Destination Option).
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Option Type | Opt Data Len |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| FlowMonID |L|D| Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Where:
Option Type:
8-bit identifier of the type of Option that needs to be
allocated. Unrecognized Types MUST be ignored
on processing. For the Hop-by-Hop Options Header or Destination
Options Header,
defines how to encode the three high-order bits of the
Option Type field. The two high-order bits specify the
action that must be taken if the processing IPv6 node does
not recognize the Option Type; for AltMark, these two bits
MUST be set to 00 (skip over this Option and
continue processing the header). The third-highest-order
bit specifies whether the Option Data can change en route to
the packet's final destination; for AltMark, the value of
this bit MUST be set to 0 (Option Data does
not change en route). In this way, since the three
high-order bits of the AltMark Option are set to 000, it
means that nodes can simply skip this Option if they do not
recognize it and that the data of this Option does not change en
route; indeed the source is the only one that can write it.
Opt Data Len:
4. It is the length of the Option Data Fields of this
Option in bytes.
FlowMonID:
20-bit unsigned integer. The FlowMon identifier is
described in .
As further discussed below, it has been picked as 20 bits
since it is a reasonable value and a good compromise in
relation to the chance of collision. It MUST
be set pseudo-randomly by the source node or by a
centralized controller.
L:
Loss flag for Packet Loss Measurement as described in
.
D:
Delay flag for Single Packet Delay Measurement as
described in .
Reserved:
Reserved for future use. These bits
MUST be set to zero on transmission and
ignored on receipt.
Use of the AltMark OptionThe AltMark Option is the best way to implement the Alternate-Marking Method, and it is carried
by the Hop-by-Hop Options Header and the Destination Options Header.
In case of Destination Option, it is processed only by the source and destination nodes: the source node inserts it
and the destination node processes it.
In case of the Hop-by-Hop Option, it may be examined by any node along the path if explicitly configured to do so.It is important to highlight that the Option Layout can be used both as the Destination Option and as the
Hop-by-Hop Option depending on the use cases, and it is based on the chosen type of performance measurement.
In general, it is needed to perform both end-to-end and hop-by-hop measurements, and the Alternate-Marking
methodology allows, by definition, both performance measurements.
In many cases, the end-to-end measurement
may not be enough, and the hop-by-hop measurement is required. To meet this need, the most complete choice is the
Hop-by-Hop Options Header.IPv6, as specified in , allows nodes to optionally process
Hop-by-Hop headers. Specifically, the Hop-by-Hop Options Header is not inserted or deleted, but it may
be examined or processed by any node along a packet's delivery path, until the packet reaches the node
(or each of the set of nodes in the case of multicast) identified in the Destination
Address field of the IPv6 header. Also, it is expected that nodes along a packet's delivery path
only examine and process the Hop-by-Hop Options Header if explicitly configured to do so.Another scenario is the presence of a Routing Header.
Both Hop-by-Hop Options and Destination Options Headers can be used when a Routing Header is present.
Depending on where the Destination Options are situated in the header chain (before or after the Routing Header if any),
Destination Options Headers can be processed by either intermediate routers specified in the Routing Header or the destination node.
As an example, a type of Routing Header, referred to as a Segment Routing Header (SRH), has been defined in
for the Segment Routing over IPv6 (SRv6) data place, and more details about the SRv6 application can be found in
.In summary, using these tools, it is possible to control on which nodes measurement occurs:
Destination Option not preceding a Routing Header => measurement
only by node in Destination Address
Hop-by-Hop Option => every router on the path with feature
enabled
Destination Option preceding a Routing Header => every destination
node in the route list
In general, Hop-by-Hop and Destination Options are the most suitable ways
to implement Alternate Marking.It is worth mentioning that Hop-by-Hop Options are not strongly recommended in
and , unless there is a clear justification to standardize it, because nodes may be
configured to ignore the Options Header or drop or assign packets containing an Options Header to a slow processing path.
In case of the AltMark Data Fields described in this document, the motivation to standardize a Hop-by-Hop Option
is that it is needed for Operations, Administration, and Maintenance (OAM). An intermediate node can read it or not, but
this does not affect the packet behavior. The source node is the only one that writes the Hop-by-Hop Option to alternately mark
the flow; therefore, the performance measurement can be done for those nodes configured to read this Option,
while the others are simply not considered for the metrics.The Hop-by-Hop Option defined in this document is designed to take advantage of the property of how Hop-by-Hop
Options are processed. Nodes that do not support this Option would be expected to ignore it if encountered,
according to the procedures of . This can mean that, in this case, the performance measurement
does not account for all links and nodes along a path. The definition of the Hop-by-Hop Options in this document is also
designed to minimize throughput impact both on nodes that do not recognize the Option and on nodes that support it.
Indeed, the three high-order bits of the Options Header defined in this document are 000 and, in theory, as per
and , this means "skip if not recognized and data does not change en route". also mentions that the nodes only examine and process the Hop-by-Hop Options Header if explicitly configured to do so. For these reasons, this Hop-by-Hop Option should not affect the throughput.
However, in practice, it is important to be aware that things may be different in the implementation, and it can happen that packets
with Hop by Hop are forced onto the slow path, but this is a general issue, as also explained in .
It is also worth mentioning that the application to a controlled domain should avoid the risk of arbitrary nodes dropping packets
with Hop-by-Hop Options.Alternate-Marking Method OperationThis section describes how the method operates.
introduces several applicable methods, which are reported below, and an additional field is introduced
to facilitate the deployment and improve the scalability.Packet Loss MeasurementThe measurement of the packet loss is really straightforward in comparison to the existing mechanisms,
as detailed in .
The packets of the flow are grouped into batches, and all the packets within a batch are marked by setting
the L bit (Loss flag) to a same value. The source node can switch the value of the L bit between 0 and 1
after a fixed number of packets or according to a fixed timer, and this depends on the
implementation. The source node is the only one that marks the packets to create the batches, while
the intermediate nodes only read the marking values and identify the packet batches.
By counting the number of packets in each batch and comparing the values measured by
different network nodes along the path, it is possible to measure the packet loss that occurred
in any single batch between any two nodes. Each batch represents a measurable entity
recognizable by all network nodes along the path.Both fixed number of packets and a fixed timer can be used by the source node to create packet batches.
But, as also explained in , the timer-based batches are preferable because
they are more deterministic than the counter-based batches.
Unlike the timer-based batches, there is no definitive rule
for counter-based batches, which are not considered in .
Using a fixed timer for the switching offers
better control over the method; indeed, the length of the batches can be chosen large enough to simplify
the collection and the comparison of the measures taken by different network nodes. In the implementation,
the counters can be sent out by each node to the controller that is responsible for the calculation.
It is also possible to exchange this information by using other on-path techniques, but this is out of scope
for this document.Packets with different L values may get swapped at batch boundaries, and in this case,
it is required that each marked packet can be assigned to the right batch by each router.
It is important to mention that for the application of this method, there are two elements
to consider: the clock error between network nodes and the network delay.
These can create
offsets between the batches and out-of-order packets.
The mathematical formula
on timing aspects, explained in , must be satisfied,
and it takes into consideration the different causes of reordering such as clock error and network delay.
The assumption is to define the available counting interval to get stable counters and to avoid these issues.
Specifically, if the effects of network delay are ignored, the condition to implement the methodology is that
the clocks in different nodes MUST be synchronized to the same clock reference with an accuracy of +/- B/2 time units,
where B is the fixed time duration of the batch. In this way, each marked packet can be assigned to the right batch by each node.
Usually, the counters can be taken in the middle of the batch period to be sure to read quiescent counters.
In a few words, this implies that the length of the batches MUST be chosen large enough so that the method
is not affected by those factors. The length of the batches can be determined based on the specific deployment scenario.Packet Loss Measurement and Single-Marking Methodology Using L Bit
L bit=1 ----------+ +-----------+ +----------
| | | |
L bit=0 +-----------+ +-----------+
Batch n ... Batch 3 Batch 2 Batch 1
<---------> <---------> <---------> <---------> <--------->
Traffic Flow
===========================================================>
L bit ...1111111111 0000000000 11111111111 00000000000 111111111...
===========================================================>
It is worth mentioning that the duration of the batches is considered stable over time in the previous figure.
In theory, it is possible to change the length of batches over time and among different flows for more flexibility.
But, in practice, it could complicate the correlation of the information.Packet Delay MeasurementThe same principle used to measure packet loss can also be applied to
one-way delay measurement. Delay metrics MAY be calculated using the following two
possibilities:
Single-Marking Methodology:
This approach uses only the L bit to calculate both packet loss
and delay. In this case, the D flag MUST be set to zero on transmit and ignored by the
monitoring points. The alternation of the values of the L bit can be used as a time reference to calculate
the delay. Whenever the L bit changes and a new batch starts, a network node can store the timestamp
of the first packet of the new batch; that timestamp can be compared with the timestamp of the
first packet of the same batch on a second node to compute packet delay. But, this measurement
is accurate only if no packet loss occurs and if there is no packet reordering at the edges
of the batches. A different approach can also be considered, and it is based on the concept of the
mean delay. The mean delay for each batch is calculated by considering the average arrival time
of the packets for the relative batch. There are limitations also in this case indeed; each node needs
to collect all the timestamps and calculate the average timestamp for each batch. In addition, the
information is limited to a mean value.
Double-Marking Methodology:
This approach is more complete and uses the L bit only to calculate
packet loss, and the D bit (Delay flag) is fully dedicated to delay measurements. The idea is to use
the first marking with the L bit to create the alternate flow and, within the batches identified by the L bit,
a second marking is used to select the packets for measuring delay. The D bit creates a new set of marked packets
that are fully identified over the network so that a network node can store the timestamps of these packets;
these timestamps can be compared with the timestamps of the same packets on a second node to compute packet
delay values for each packet. The most efficient and robust mode is to select a single double-marked packet
for each batch; in this way, there is no time gap to consider between the double-marked packets to avoid their reorder.
Regarding the rule for the selection of the packet to be double-marked, the same considerations in
also apply here, and the double-marked packet can be chosen within the available counting interval that
is not affected by factors such as clock errors.
If a double-marked packet is lost, the delay measurement for the considered batch is simply discarded,
but this is not a big problem because it is easy to recognize the problematic batch and skip the measurement
just for that one. So in order to have more information about the delay and to overcome out-of-order issues,
this method is preferred.
In summary, the approach with Double Marking is better than the approach with Single Marking. Moreover,
the two approaches provide slightly different pieces of information, and the data consumer can combine them
to have a more robust data set.Similar to what is said in for the packet counters, in the implementation, the timestamps can be
sent out to the controller that is responsible for the calculation or exchanged using other on-path techniques.
But, this is out of scope for this document.Double-Marking Methodology Using L Bit and D Bit
L bit=1 ----------+ +-----------+ +----------
| | | |
L bit=0 +-----------+ +-----------+
D bit=1 + + + + +
| | | | |
D bit=0 ------+----------+----------+----------+------------+-----
Traffic Flow
===========================================================>
L bit ...1111111111 0000000000 11111111111 00000000000 111111111...
D bit ...0000010000 0000010000 00000100000 00001000000 000001000...
===========================================================>
Likewise, to packet delay measurement (both for Single Marking and Double Marking), the method can also be used
to measure the inter-arrival jitter.Flow Monitoring IdentificationThe Flow Monitoring Identification (FlowMonID) identifies the flow to be measured and
is required for some general reasons:
First, it helps to reduce the per-node configuration. Otherwise, each node needs to
configure an access control list (ACL) for each of the monitored flows.
Moreover, using a flow identifier allows a flexible granularity for the flow definition;
indeed, it can be used together with other identifiers (e.g., 5-tuple).
Second, it simplifies the counters handling. Hardware processing of flow tuples (and ACL matching)
is challenging and often incurs into performance issues, especially in tunnel interfaces.
Third, it eases the data export encapsulation and correlation for the collectors.
The FlowMonID MUST only be used as a monitored flow identifier in order to determine a monitored flow
within the measurement domain. This entails not only an easy identification but improved correlation as well.The FlowMonID allocation procedure can be stateful or stateless. In case of a stateful approach, it is required that
the FlowMonID historic information can be stored and tracked in order to assign unique values within the domain.
This may imply a complex procedure, and it is considered out of scope for this document.
The stateless approach is described hereinafter where FlowMonID values are pseudo-randomly generated.The value of 20 bits has been selected for the FlowMonID since it is a good compromise and implies a low rate
of ambiguous FlowMonIDs that can be considered acceptable in most of the applications. The disambiguation issue
can be solved by tagging the pseudo-randomly generated FlowMonID with additional flow information.
In particular, it is RECOMMENDED to consider the 3-tuple FlowMonID, source, and destination addresses:
If the 20-bit FlowMonID is set independently and pseudo-randomly in a distributed way, there is a chance of collision.
Indeed, by using the well-known birthday problem in probability theory, if the 20-bit FlowMonID
is set independently and pseudo-randomly without any additional input entropy, there is a 50% chance of collision
for 1206 flows. So, for more entropy, FlowMonID is combined with source and destination addresses.
Since there is a 1% chance of collision for 145 flows, it is possible to monitor 145 concurrent flows per host pairs
with a 1% chance of collision.
If the 20-bit FlowMonID is set pseudo-randomly but in a centralized way, the controller can instruct the nodes properly
in order to guarantee the uniqueness of the FlowMonID. With 20 bits, the number of combinations is 1048576, and the controller
should ensure that all the FlowMonID values are used without any collision. Therefore, by considering source and destination addresses
together with the FlowMonID, it is possible to monitor 1048576 concurrent flows per host pairs.
A consistent approach MUST be used in the Alternate-Marking deployment to avoid the mixture of different ways of identifying.
All the nodes along the path and involved in the measurement SHOULD use the same mode for identification.
As mentioned, it is RECOMMENDED to use the FlowMonID for identification purposes in combination with source and destination addresses
to identify a flow. By considering source and destination addresses together with the FlowMonID, it is possible to monitor
145 concurrent flows per host pairs with a 1% chance of collision in case of pseudo-randomly generated FlowMonID, or
1048576 concurrent flows per host pairs in case of a centralized controller. It is worth mentioning that
the solution with the centralized control allows finer granularity and therefore adds even more flexibility to the flow identification.The FlowMonID field is set at the source node, which is the ingress point of the measurement domain, and
can be set in two ways:
It can be algorithmically generated by the source node, which can set it pseudo-randomly with some
chance of collision. This approach cannot guarantee the uniqueness of FlowMonID since conflicts and collisions are possible.
But, considering the recommendation to use FlowMonID with source and destination addresses, the conflict probability is reduced due to
the FlowMonID space available for each endpoint pair (i.e., 145 flows with 1% chance of collision).
It can be assigned by the central controller. Since the controller knows the network topology,
it can allocate the value properly to avoid or minimize ambiguity and guarantee the uniqueness. In this regard,
the controller can verify that there is no ambiguity between different pseudo-randomly generated FlowMonIDs on the same path.
The conflict probability is really small given that the FlowMonID is coupled with source and destination addresses,
and up to 1048576 flows can be monitored for each endpoint pair. When all values in the FlowMonID space are consumed,
the centralized controller can keep track and reassign the values that are not used any more by old flows.
If the FlowMonID is set by the source node, the intermediate nodes can read the FlowMonIDs from the packets in flight
and act accordingly. If the FlowMonID is set by the controller, both possibilities are feasible for the intermediate nodes,
which can learn by reading the packets or can be instructed by the controller.The FlowMonID setting by the source node may seem faster and more scalable than the FlowMonID setting by the controller. But,
it is supposed that the controller does not slow the process since it can enable the Alternate-Marking Method and its parameters (like FlowMonID)
together with the flow instantiation, as further described in and .Multipoint and Clustered Alternate MarkingThe Alternate-Marking Method can be extended to any kind of multipoint-to-multipoint paths.
only applies to point-to-point unicast flows,
while the Clustered Alternate-Marking Method, introduced in ,
is valid for multipoint-to-multipoint unicast flows, anycast, and ECMP flows. describes the network clustering approach, which
allows a flexible and optimized performance measurement.
A cluster is the smallest identifiable non-trivial subnetwork of the entire network graph
that still satisfies the condition that the number of packets that goes in is the same number that goes out.
With network clustering, it is possible to partition the network into clusters
at different levels in order to perform the needed degree of detail.For Multipoint Alternate Marking, FlowMonID can identify in general
a multipoint-to-multipoint flow and not only a point-to-point flow.Data Collection and CalculationThe nodes enabled to perform performance monitoring collect the value
of the packet counters and timestamps. There are several alternatives to implement
data collection and calculation, but this is not specified in this document.There are documents on the control plane mechanisms of Alternate Marking, e.g.,
and .Security ConsiderationsThis document aims to apply a method to the performance measurements that does
not directly affect Internet security nor applications that run on
the Internet. However, implementation of this method must be mindful
of security and privacy concerns.There are two types of security concerns:
potential harm caused by the measurements and potential harm to the measurements.
Harm caused by the measurement:
Alternate Marking implies the insertion of an Options Header to the IPv6
packets by the source node, but this must be performed in a way that does not
alter the quality of service experienced by the packets and that preserves
stability and performance of routers doing the measurements. As already
discussed in , the design of the AltMark
Option has been chosen with throughput in mind, such that it can be
implemented without affecting the user experience.
Harm to the measurement:
Alternate-Marking measurements could be harmed by routers altering the
fields of the AltMark Option (e.g., marking of the packets or FlowMonID) or by a
malicious attacker adding the AltMark Option to the packets in order to consume
the resources of network devices and entities involved. As described above,
the source node is the only one that writes the Options Header while the
intermediate nodes and destination node only read it without modifying the
Options Header. But, for example, an on-path attacker can modify the flags,
whether intentionally or accidentally, or deliberately insert an Option to the
packet flow or delete the Option from the packet flow. The consequent effect
could be to give the appearance of loss or delay or to invalidate the measurement
by modifying Option identifiers, such as FlowMonID. The malicious implication
can be to cause actions from the network administrator where an intervention
is not necessary or to hide real issues in the network. Since the measurement
itself may be affected by network nodes intentionally altering the bits of the
AltMark Option or injecting Options Headers as a means for Denial of Service
(DoS), the Alternate Marking MUST be applied in the context of
a controlled domain, where the network nodes are locally administered and this
type of attack can be avoided. For this reason, the implementation of the
method is not done on the end node if it is not fully managed and does not
belong to the controlled domain. Packets generated outside the controlled
domain may consume router resources by maliciously using the Hop-by-Hop Option, but
this can be mitigated by filtering these packets at the controlled domain
boundary. This can be done because if the end node does not belong to the
controlled domain, it is not supposed to add the AltMark Hop-by-Hop Option, and it
can be easily recognized.
An attacker that does not belong to the controlled domain can maliciously send packets with the AltMark Option.
But, if Alternate Marking is not supported in the controlled domain, no problem happens because the AltMark Option is treated
as any other unrecognized Option and will not be considered by the nodes since they are not configured to deal with it; so,
the only effect is the increased packet size (by 48 bits).
If Alternate Marking is supported in the controlled domain, it is necessary to keep the measurements from being affected,
and external packets with the AltMark Option MUST be filtered.
As any other Hop-by-Hop Options or Destination Options, it is possible to filter AltMark Options entering or leaving the domain,
e.g., by using ACL extensions for filtering.The flow identifier (FlowMonID), together with the two marking bits (L and D), comprises the AltMark Option.
As explained in , there is a chance of collision if the FlowMonID
is set pseudo-randomly, but there is a solution for this issue. In general, this may not be a problem, and a low rate of
ambiguous FlowMonIDs can be acceptable since this does not cause significant harm to the operators or
their clients, and this harm may not justify the complications of avoiding it. But, for large scale measurements,
a big number of flows could be monitored and the probability of a collision is higher; thus, the disambiguation
of the FlowMonID field can be considered.The privacy concerns also need to be analyzed even if the method only relies on information contained
in the Options Header without any release of user data. Indeed, from a confidentiality perspective,
although the AltMark Option does not contain user data, the metadata can be used for network reconnaissance
to compromise the privacy of users by allowing attackers to collect information about network performance and network paths.
The AltMark Option contains two kinds of metadata: the marking bits (L and D) and the flow identifier (FlowMonID).
The marking bits are the small information that is exchanged between the network nodes. Therefore, due to this intrinsic
characteristic, network reconnaissance through passive eavesdropping on data plane traffic is difficult.
Indeed, an attacker cannot gain information about network performance from a single monitoring point. The only way for an attacker
can be to eavesdrop on multiple monitoring points at the same time, because they have to do the same kind of calculation
and aggregation as Alternate Marking requires.
The FlowMonID field is used in the AltMark Option as the identifier of the monitored flow. It represents more sensitive information
for network reconnaissance and may allow a flow tracking type of attack because an attacker could collect information
about network paths.
Furthermore, in a pervasive surveillance attack, the information that can be derived over time is more.
But, as further described hereinafter, the application of the Alternate Marking to a controlled domain
helps to mitigate all the above aspects of privacy concerns.At the management plane, attacks can be set up by misconfiguring or by maliciously configuring the AltMark Option.
Thus, AltMark Option configuration MUST be secured in a way that authenticates authorized users and verifies the
integrity of configuration procedures. Solutions to ensure the integrity of the AltMark Option are outside the
scope of this document. Also, attacks on the reporting of the statistics between the monitoring points and the
network management system (e.g., centralized controller) can interfere with the proper functioning of the system.
Hence, the channels used to report back flow statistics MUST be secured.As stated above, the precondition for the application of the Alternate Marking is that it MUST be applied
in specific controlled domains, thus confining the potential attack vectors within the network domain.
A limited administrative domain provides the network administrator with the means to select, monitor, and
control the access to the network, making it a trusted domain. In this regard, it is expected to enforce policies
at the domain boundaries to filter both external packets with the AltMark Option entering the domain and
internal packets with the AltMark Option leaving the domain. Therefore, the trusted domain is unlikely subject
to the hijacking of packets since packets with AltMark Option are processed and used only within the controlled domain.As stated, the application to a controlled domain ensures control over the packets entering and leaving the domain,
but despite that, leakages may happen for different reasons such as a failure or a fault. In this case, nodes
outside the domain are expected to ignore packets with the AltMark Option since they are not configured to handle it and
should not process it.Additionally, note that the AltMark Option is carried by the Options Header
and it will have some impact on the packet sizes for the monitored flow and on the path MTU
since some packets might exceed the MTU. However, the relative small size (48 bits in total)
of these Options Headers and its application to a controlled domain help to mitigate the problem.It is worth mentioning that the security concerns may change based on the specific deployment scenario
and related threat analysis, which can lead to specific security solutions that are beyond the scope of this document.
As an example, the AltMark Option can be used as a Hop-by-Hop or Destination Option and, in case of a Destination Option,
multiple administrative domains may be traversed by the AltMark Option that is not confined to a single administrative domain.
In this case, the user, who is aware of the kind of risks, may still want to use Alternate Marking for telemetry and test purposes, but
the controlled domain must be composed by more than one administrative domain. To this end, the inter-domain links need
to be secured (e.g., by IPsec or VPNs) in order to avoid external threats and realize the whole controlled domain.It might be theoretically possible to modulate the marking or the other fields of the AltMark Option to serve as a covert channel
to be used by an on-path observer. This may affect both the data and management plane, but, here too, the application to a
controlled domain helps to reduce the effects.The Alternate-Marking application described in this document relies on a time synchronization
protocol. Thus, by attacking the time protocol, an attacker can potentially compromise the integrity
of the measurement. A detailed discussion about the threats against time protocols and
how to mitigate them is presented in . Network Time Security (NTS),
described in , is a mechanism that can be employed. Also, the time,
which is distributed to the network nodes through the time protocol, is centrally taken from an external accurate time source
such as an atomic clock or a GPS clock. By attacking the time source, it is possible to compromise the integrity
of the measurement as well. There are security measures that can be taken to mitigate the GPS spoofing attacks, and a
network administrator should certainly employ solutions to secure the network domain.IANA ConsiderationsIANA has allocated the Option Type in
the "Destination Options and Hop-by-Hop Options" subregistry of the
"Internet Protocol Version 6 (IPv6) Parameters" registry () as follows:
Destination Options and Hop-by-Hop Options Registry
Hex Value
Binary Value
Description
Reference
act
chg
rest
0x12
00
0
10010
AltMark
RFC 9343
ReferencesNormative ReferencesKey words for use in RFCs to Indicate Requirement LevelsIn many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements.Ambiguity of Uppercase vs Lowercase in RFC 2119 Key WordsRFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings.Internet Protocol, Version 6 (IPv6) SpecificationThis document specifies version 6 of the Internet Protocol (IPv6). It obsoletes RFC 2460.Alternate-Marking MethodClustered Alternate-Marking MethodInformative ReferencesBGP SR Policy Extensions to Enable IFITChina MobileUnionPayChina TelecomHuaweiHuawei Segment Routing (SR) policy is a set of candidate SR paths consisting
of one or more segment lists and necessary path attributes. It
enables instantiation of an ordered list of segments with a specific
intent for traffic steering. In-situ Flow Information Telemetry
(IFIT) refers to network OAM data plane on-path telemetry techniques,
in particular the most popular are In-situ OAM (IOAM) and Alternate
Marking. This document defines extensions to BGP to distribute SR
policies carrying IFIT information. So that IFIT methods can be
enabled automatically when the SR policy is applied.
Work in ProgressIPv6 Hop-by-Hop Options Processing ProceduresCheck Point SoftwareUniversity of AberdeenWork in ProgressPath Computation Element Communication Protocol (PCEP) Extensions to Enable IFITUnionPayChina TelecomHuaweiHuaweiHuaweiWork in ProgressOperational Issues with Processing of the Hop-by-Hop Options HeaderHuawei TechnologiesHuawei TechnologiesChina TelecomChina UnicomVerizon Inc.Work in ProgressSecurity Architecture for the Internet ProtocolThis document describes an updated version of the "Security Architecture for IP", which is designed to provide security services for traffic at the IP layer. This document obsoletes RFC 2401 (November 1998). [STANDARDS-TRACK]IPv6 Flow Label SpecificationThis document specifies the IPv6 Flow Label field and the minimum requirements for IPv6 nodes labeling flows, IPv6 nodes forwarding labeled packets, and flow state establishment methods. Even when mentioned as examples of possible uses of the flow labeling, more detailed requirements for specific use cases are out of the scope for this document.The usage of the Flow Label field enables efficient IPv6 flow classification based only on IPv6 main header fields in fixed positions. [STANDARDS-TRACK]Using the IPv6 Flow Label for Equal Cost Multipath Routing and Link Aggregation in TunnelsThe IPv6 flow label has certain restrictions on its use. This document describes how those restrictions apply when using the flow label for load balancing by equal cost multipath routing and for link aggregation, particularly for IP-in-IPv6 tunneled traffic. [STANDARDS-TRACK]Transmission and Processing of IPv6 Extension HeadersVarious IPv6 extension headers have been standardised since the IPv6 standard was first published. This document updates RFC 2460 to clarify how intermediate nodes should deal with such extension headers and with any that are defined in the future. It also specifies how extension headers should be registered by IANA, with a corresponding minor update to RFC 2780.Security Requirements of Time Protocols in Packet Switched NetworksAs time and frequency distribution protocols are becoming increasingly common and widely deployed, concern about their exposure to various security threats is increasing. This document defines a set of security requirements for time protocols, focusing on the Precision Time Protocol (PTP) and the Network Time Protocol (NTP). This document also discusses the security impacts of time protocol practices, the performance implications of external security practices on time protocols, and the dependencies between other security services and time synchronization.IPv6 Segment Routing Header (SRH)Segment Routing can be applied to the IPv6 data plane using a new type of Routing Extension Header called the Segment Routing Header (SRH). This document describes the SRH and how it is used by nodes that are Segment Routing (SR) capable.Limited Domains and Internet ProtocolsThere is a noticeable trend towards network behaviors and semantics that are specific to a particular set of requirements applied within a limited region of the Internet. Policies, default parameters, the options supported, the style of network management, and security requirements may vary between such limited regions. This document reviews examples of such limited domains (also known as controlled environments), notes emerging solutions, and includes a related taxonomy. It then briefly discusses the standardization of protocols for limited domains. Finally, it shows the need for a precise definition of "limited domain membership" and for mechanisms to allow nodes to join a domain securely and to find other members, including boundary nodes.This document is the product of the research of the authors. It has been produced through discussions and consultation within the IETF but is not the product of IETF consensus.Network Time Security for the Network Time ProtocolThis memo specifies Network Time Security (NTS), a mechanism for using Transport Layer Security (TLS) and Authenticated Encryption with Associated Data (AEAD) to provide cryptographic security for the client-server mode of the Network Time Protocol (NTP).NTS is structured as a suite of two loosely coupled sub-protocols. The first (NTS Key Establishment (NTS-KE)) handles initial authentication and key establishment over TLS. The second (NTS Extension Fields for NTPv4) handles encryption and authentication during NTP time synchronization via extension fields in the NTP packets, and holds all required state only on the client via opaque cookies.Segment Routing Header encapsulation for Alternate Marking MethodHuaweiHuaweiTelecom Italia This document describes how the Alternate Marking Method can be used
as the passive performance measurement tool in an SRv6 network. It
defines how Alternate Marking data fields are transported as part of
the Segment Routing with IPv6 data plane (SRv6) header.
Work in ProgressAcknowledgementsThe authors would like to thank , , , , ,
, , , , , ,
, , , , , and
for their valuable comments and suggestions.Authors' AddressesHuaweiRiesstrasse, 25Munich80992Germanygiuseppe.fioccola@huawei.comHuawei156 Beiqing Rd.Beijing100095Chinazhoutianran@huawei.comTelecom Italiamauro.cociglio@outlook.comChina Mobile32 Xuanwumenxi Ave.Beijing100032Chinaqinfengwei@chinamobile.comChina Unicom9 Shouti South Rd.Beijing100089Chinapangran@chinaunicom.cn