An Extension for Application-Layer Traffic Optimization (ALTO): Path Vector

An Extension for Application-Layer Traffic Optimization (ALTO): Path Vector Sichuan University

No.24 South Section 1, Yihuan Road Chengdu 610000 China kaigao@scu.edu.cn

Samsung

Republic of Korea younglee.tx@gmail.com

Nokia Bell Labs

Route de Villejust Nozay 91460 France sabine.randriamasy@nokia-bell-labs.com

Yale University

51 Prospect Street New Haven 06511 CT United States of America yry@cs.yale.edu

Tongji University

4800 Caoan Road Shanghai 201804 China jingxuan.n.zhang@gmail.com

tsv alto network visibility abstract network element shared bottleneck This document is an extension to the base Application-Layer Traffic Optimization (ALTO) protocol. It extends the ALTO cost map and ALTO property map services so that an application can decide to which endpoint(s) to connect based not only on numerical/ordinal cost values but also on fine-grained abstract information regarding the paths. This is useful for applications whose performance is impacted by specific components of a network on the end-to-end paths, e.g., they may infer that several paths share common links and prevent traffic bottlenecks by avoiding such paths. This extension introduces a new abstraction called the "Abstract Network Element" (ANE) to represent these components and encodes a network path as a vector of ANEs. Thus, it provides a more complete but still abstract graph representation of the underlying network(s) for informed traffic optimization among endpoints.

Status of This Memo This document is not an Internet Standards Track specification; it is published for examination, experimental implementation, and evaluation. This document defines an Experimental Protocol for the Internet community. This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Not all documents approved by the IESG are candidates for any level of Internet Standard; see Section 2 of RFC 7841. Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at .

Copyright Notice Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents () in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.

Table of Contents

. Introduction
. Requirements Language
. Terminology
. Requirements and Use Cases
- . Design Requirements
- . Sample Use Cases
  - . Exposing Network Bottlenecks
  - . Resource Exposure for CDNs and Service Edges
. Path Vector Extension: Overview
- . Abstract Network Element (ANE)
  - . ANE Entity Domain
  - . Ephemeral and Persistent ANEs
  - . Property Filtering
- . Path Vector Cost Type
- . Multipart Path Vector Response
  - . Identifying the Media Type of the Object Root
  - . References to Part Messages
. Specification: Basic Data Types
- . ANE Name
- . ANE Entity Domain
  - . Entity Domain Type
  - . Domain-Specific Entity Identifier
  - . Hierarchy and Inheritance
  - . Media Type of Defining Resource
- . ANE Property Name
- . Initial ANE Property Types
  - . Maximum Reservable Bandwidth
  - . Persistent Entity ID
  - . Examples
- . Path Vector Cost Type
  - . Cost Metric: "ane-path"
  - . Cost Mode: "array"
- . Part Resource ID and Part Content ID
. Specification: Service Extensions
- . Notation
- . Multipart Filtered Cost Map for Path Vector
  - . Media Type
  - . HTTP Method
  - . Accept Input Parameters
  - . Capabilities
  - . Uses
  - . Response
- . Multipart Endpoint Cost Service for Path Vector
  - . Media Type
  - . HTTP Method
  - . Accept Input Parameters
  - . Capabilities
  - . Uses
  - . Response
. Examples
- . Sample Setup
- . Information Resource Directory
- . Multipart Filtered Cost Map
- . Multipart Endpoint Cost Service Resource
- . Incremental Updates
- . Multi-Cost
. Compatibility with Other ALTO Extensions
- . Compatibility with Legacy ALTO Clients/Servers
- . Compatibility with Multi-Cost Extension
- . Compatibility with Incremental Update Extension
- . Compatibility with Cost Calendar Extension
. General Discussion
- . Constraint Tests for General Cost Types
- . General Multi-Resource Query
. Security Considerations
. IANA Considerations
- . "ALTO Cost Metrics" Registry
- . "ALTO Cost Modes" Registry
- . "ALTO Entity Domain Types" Registry
- . "ALTO Entity Property Types" Registry
  - . New ANE Property Type: Maximum Reservable Bandwidth
  - . New ANE Property Type: Persistent Entity ID
. References
- . Normative References
- . Informative References
Acknowledgments
Authors' Addresses

Introduction Network performance metrics are crucial for assessing the Quality of Experience (QoE) of applications. The Application-Layer Traffic Optimization (ALTO) protocol allows Internet Service Providers (ISPs) to provide guidance, such as topological distances between different end hosts, to overlay applications. Thus, the overlay applications can potentially improve the perceived QoE by better orchestrating their traffic to utilize the resources in the underlying network infrastructure. The existing ALTO cost map () and Endpoint Cost Service () provide only cost information for an end-to-end path defined by its <source, destination> endpoints: the base protocol allows the services to expose the topological distances of end-to-end paths, while various extensions have been proposed to extend the capability of these services, e.g., to express other performance metrics , to query multiple costs simultaneously , and to obtain time-varying values . While numerical/ordinal cost values for end-to-end paths provided by the existing extensions are sufficient to optimize the QoE of many overlay applications, the QoE of some overlay applications also depends on the properties of particular components on the paths. For example, job completion time, which is an important QoE metric for a large-scale data analytics application, is impacted by shared bottleneck links inside the carrier network, as link capacity may impact the rate of data input/output to the job. We refer to such components of a network as Abstract Network Elements (ANEs). Predicting such information can be very complex without the help of ISPs; for example, has shown that finding the optimal bandwidth reservation for multiple flows can be NP-hard without further information than whether a reservation succeeds. With proper guidance from the ISP, an overlay application may be able to schedule its traffic for better QoE. In the meantime, it may be helpful as well for ISPs if applications could avoid using bottlenecks or challenging the network with poorly scheduled traffic. Despite the claimed benefits, ISPs are not likely to expose raw details on their network paths: first because ISPs have requirements to hide their network topologies, second because these details may increase volume and computation overhead, and last because applications do not necessarily need all the network path details and are likely not able to understand them. Therefore, it is beneficial for both ISPs and applications if an ALTO server provides ALTO clients with an "abstract network state" that provides the necessary information to applications, while hiding network complexity and confidential information. An "abstract network state" is a selected set of abstract representations of ANEs traversed by the paths between <source, destination> pairs combined with properties of these ANEs that are relevant to the overlay applications' QoE. Both an application via its ALTO client and the ISP via the ALTO server can achieve better confidentiality and resource utilization by appropriately abstracting relevant ANEs. Server scalability can also be improved by combining ANEs and their properties in a single response. This document extends the ALTO base protocol to allow an ALTO server to convey "abstract network state" for paths defined by their <source, destination> pairs. To this end, it introduces a new cost type called a "Path Vector", following the cost metric registration specified in and the updated cost mode registration specified in . A Path Vector is an array of identifiers that identifies an ANE, which can be associated with various properties. The associations between ANEs and their properties are encoded in an ALTO information resource called the "entity property map", which is specified in . For better confidentiality, this document aims to minimize information exposure of an ALTO server when providing Path Vector services. In particular, this document enables the capability, and also recommends that 1) ANEs be constructed on demand and 2) an ANE only be associated with properties that are requested by an ALTO client. A Path Vector response involves two ALTO maps: the cost map, which contains the Path Vector results; and the up-to-date entity property map, which contains the properties requested for these ANEs. To enforce consistency and improve server scalability, this document uses the "multipart/related" content type as defined in to return the two maps in a single response. As a single ISP may not have knowledge of the full Internet paths between arbitrary endpoints, this document is mainly applicable when

there is a single ISP between the requested source and destination Provider-defined Identifiers (PIDs) or endpoints -- for example, ISP-hosted Content Delivery Network (CDN) / edge, tenant interconnection in a single public cloud platform, etc., or
the Path Vectors are generated from end-to-end measurement data.

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Terminology This document extends the ALTO base protocol and the entity property map extension . In addition to the terms defined in those documents, this document also uses the following terms:

Abstract Network Element (ANE):: An abstract representation for a component in a network that handles data packets and whose properties can potentially have an impact on the end-to-end performance of traffic. An ANE can be a physical device such as a router, a link, or an interface; or an aggregation of devices such as a subnetwork or a data center. The definition of an ANE is similar to that for a network element as defined in in the sense that they both provide an abstract representation of specific components of a network. However, they have different criteria on how these particular components are selected. Specifically, a network element requires the components to be capable of exercising QoS control, while an ANE only requires the components to have an impact on end-to-end performance.
ANE name:: A string that uniquely identifies an ANE in a specific scope. An ANE can be constructed either statically in advance or on demand based on the requested information. Thus, different ANEs may only be valid within a particular scope, either ephemeral or persistent. Within each scope, an ANE is uniquely identified by an ANE name, as defined in . Note that an ALTO client must not assume ANEs in different scopes but with the same ANE name refer to the same component(s) of the network.
Path Vector (or ANE Path Vector):: Refers to a JSON array of ANE names. It is a generalization of a BGP path vector. While a standard BGP path vector () specifies a sequence of Autonomous Systems (ASes) for a destination IP prefix, the Path Vector defined in this extension specifies a sequence of ANEs for either 1) a source PID and a destination PID, as in the CostMapData object () or 2) a source endpoint and a destination endpoint, as in the EndpointCostMapData object ().
Path Vector resource:: An ALTO information resource () that supports the extension defined in this document.
Path Vector cost type:: A special cost type, which is specified in . When this cost type is present in an Information Resource Directory (IRD) entry, it indicates that the information resource is a Path Vector resource. When this cost type is present in a filtered cost map request or an Endpoint Cost Service request, it indicates that each cost value must be interpreted as a Path Vector.
Path Vector request:: The POST message sent to an ALTO Path Vector resource.
Path Vector response:: Refers to the multipart/related message returned by a Path Vector resource.

Requirements and Use Cases

Design Requirements This section gives an illustrative example of how an overlay application can benefit from the extension defined in this document. Assume that an application has control over a set of flows, which may go through shared links/nodes and share bottlenecks. The application seeks to schedule the traffic among multiple flows to get better performance. The constraints of feasible rate allocations of those flows will benefit the scheduling. However, cost maps as defined in cannot reveal such information. Specifically, consider the example network shown in . The network has seven switches ("sw1" to "sw7") forming a dumbbell topology. Switches "sw1", "sw2", "sw3", and "sw4" are access switches, and "sw5-sw7" form the backbone. End hosts "eh1" to "eh4" are connected to access switches "sw1" to "sw4", respectively. Assume that the bandwidth of link "eh1 -> sw1" and link "sw1 -> sw5" is 150 Mbps and the bandwidth of the other links is 100 Mbps.

Raw Network Topology +-----+ | | --+ sw6 +-- / | | \ PID1 +-----+ / +-----+ \ +-----+ PID2 eh1__| |_ / \ ____| |__eh2 192.0.2.2 | sw1 | \ +--|--+ +--|--+ / | sw2 | 192.0.2.3 +-----+ \ | | | |/ +-----+ \_| sw5 +---------+ sw7 | PID3 +-----+ / | | | |\ +-----+ PID4 eh3__| |__/ +-----+ +-----+ \____| |__eh4 192.0.2.4 | sw3 | | sw4 | 192.0.2.5 +-----+ +-----+ bw(eh1--sw1) = bw(sw1--sw5) = 150 Mbps bw(eh2--sw2) = bw(eh3--sw3) = bw(eh4--sw4) = 100 Mbps bw(sw1--sw5) = bw(sw3--sw5) = bw(sw2--sw7) = bw(sw4--sw7) = 100 Mbps bw(sw5--sw6) = bw(sw5--sw7) = bw(sw6--sw7) = 100 Mbps The base ALTO topology abstraction of the network is shown in . Assume that the cost map returns a hypothetical cost type representing the available bandwidth between a source and a destination.

Base Topology Abstraction +----------------------+ {eh1} | | {eh2} PID1 | | PID2 +------+ +------+ | | | | {eh3} | | {eh4} PID3 | | PID4 +------+ +------+ | | +----------------------+ Now, assume that the application wants to maximize the total rate of the traffic among a set of <source, destination> pairs -- say, "eh1 -> eh2" and "eh1 -> eh4". Let "x" denote the transmission rate of "eh1 -> eh2" and "y" denote the rate of "eh1 -> eh4". The objective function is max(x + y). With the ALTO cost map, the costs between PID1 and PID2 and between PID1 and PID4 will both be 100 Mbps. The client can get a capacity region of x <= 100 Mbps y <= 100 Mbps. With this information, the client may mistakenly think it can achieve a maximum total rate of 200 Mbps. However, this rate is infeasible, as there are only two potential cases:

Case 1:: "eh1 -> eh2" and "eh1 -> eh4" take different path segments from "sw5" to "sw7". For example, if "eh1 -> eh2" uses path "eh1 -> sw1 -> sw5 -> sw6 -> sw7 -> sw2 -> eh2" and "eh1 -> eh4" uses path "eh1 -> sw1 -> sw5 -> sw7 -> sw4 -> eh4", then the shared bottleneck links are "eh1 -> sw1" and "sw1 -> sw5". In this case, the capacity region is: x <= 100 Mbps y <= 100 Mbps x + y <= 150 Mbps and the real optimal total rate is 150 Mbps.
Case 2:: "eh1 -> eh2" and "eh1 -> eh4" take the same path segment from "sw5" to "sw7". For example, if "eh1 -> eh2" uses path "eh1 -> sw1 -> sw5 -> sw7 -> sw2 -> eh2" and "eh1 -> eh4" also uses path "eh1 -> sw1 -> sw5 -> sw7 -> sw4 -> eh4", then the shared bottleneck link is "sw5 -> sw7". In this case, the capacity region is: x <= 100 Mbps y <= 100 Mbps x + y <= 100 Mbps and the real optimal total rate is 100 Mbps.

Clearly, with more accurate and fine-grained information, the application can better predict its traffic and may orchestrate its resources accordingly. However, to provide such information, the network needs to expose abstract information beyond the simple cost map abstraction. In particular:

The ALTO server must expose abstract information about the network paths that are traversed by the traffic between a source and a destination beyond a simple numerical value, which allows the overlay application to distinguish between Cases 1 and 2 and to compute the optimal total rate accordingly.
The ALTO server must allow the client to distinguish the common ANE shared by "eh1 -> eh2" and "eh1 -> eh4", e.g., "eh1‑‑sw1" and "sw1‑‑sw5" in Case 1.
The ALTO server must expose abstract information on the properties of the ANEs used by "eh1 -> eh2" and "eh1 -> eh4". For example, an ALTO server can either expose the available bandwidth between "eh1‑‑sw1", "sw1‑‑sw5", "sw5‑‑sw7", "sw5‑‑sw6", "sw6‑‑sw7", "sw7‑‑sw2", "sw7‑‑sw4", "sw2‑‑eh2", "sw4‑‑eh4" in Case 1 or expose three abstract elements "A", "B", and "C", which represent the linear constraints that define the same capacity region in Case 1.

In general, we can conclude that to support the use case for multiple flow scheduling, the ALTO framework must be extended to satisfy the following additional requirements (ARs):

AR1:: An ALTO server must provide the ANEs that are important for assessing the QoE of the overlay application on the path of a <source, destination> pair.
AR2:: An ALTO server must provide information to identify how ANEs are shared on the paths of different <source, destination> pairs.
AR3:: An ALTO server must provide information on the properties that are important for assessing the QoE of the application for ANEs.

The extension defined in this document specifies a solution to expose such abstract information.

Sample Use Cases While the problem related to multiple flow scheduling is used to help identify the additional requirements, the extension defined in this document can be applied to a wide range of applications. This section highlights some of the reported use cases.

Exposing Network Bottlenecks One important use case for the Path Vector extension is to expose network bottlenecks. Applications that need to perform large-scale data transfers can benefit from being aware of the resource constraints exposed by this extension even if they have different objectives. One such example is the Worldwide LHC Computing Grid (WLCG) (where "LHC" means "Large Hadron Collider"), which is the largest example of a distributed computation collaboration in the research and education world. illustrates an example of using an ALTO Path Vector as an interface between the job optimizer for a data analytics system and the network manager. In particular, we assume that the objective of the job optimizer is to minimize the job completion time. In such a setting, the network-aware job optimizer (e.g., ) takes a query and generates multiple query execution plans (QEPs). It can encode the QEPs as Path Vector requests that are sent to an ALTO server. The ALTO server obtains the routing information for the flows in a QEP and finds links, routers, or middleboxes (e.g., a stateful firewall) that can potentially become bottlenecks for the QEP (e.g., see and for mechanisms to identify bottleneck links under different settings). The resource constraint information is encoded in a Path Vector response and returned to the ALTO client. With the network resource constraints, the job optimizer may choose the QEP with the optimal job completion time to be executed. It must be noted that the ALTO framework itself does not offer the capability to control the traffic. However, certain network managers may offer ways to enforce resource guarantees, such as on-demand tunnels (e.g., ), demand vectors (e.g., , ), etc. The traffic control interfaces and mechanisms are out of scope for this document.

Example Use Case for Cross-Site Bottleneck Discovery Ongoing transfers New transfer requests \----\ | | | v v +-------------+ +---------------+ | ALTO Client | <===========> | Data Transfer | +-------------+ | Scheduler | ^ | ^ | PV Request +---------------+ | | | \--------------\ | | \--------------\ | | v PV Response | v +-------------+ +-------------+ | ALTO Server | | ALTO Server | +-------------+ +-------------+ || || +---------+ +---------+ | Network | | Network | | Manager | | Manager | +---------+ +---------+ . . . _~_ __ . . . . ( )( ) .___ ~v~v~ /--( )------------( ) ( )-----/ ( ) ( ) ~w~w~ ~^~^~^~ ~v~v~ Site 1 Non-blocking Core Site 2 With the Path Vector extension, a site can reveal the bottlenecks inside its own network with necessary information (such as link capacities) to the ALTO client, instead of providing the full topology and routing information, or no bottleneck information at all. The bottleneck information can be used to analyze the impact of adding/removing data transfer flows, e.g., using the framework defined in . For example, assume that hosts "a", "b", and "c" are in Site 1 and hosts "d", "e", and "f" are in Site 2, and there are three flows in two sites: "a -> b", "c -> d", and "e -> f" ().

Example: Three Flows in Two Sites Site 1: [c] . ........................................> [d] +---+ 10 Gbps +---+ 10 Gbps +----+ 50 Gbps | A |---------| B |---------| GW |--------- Core +---+ +---+ +----+ ................... . . . v [a] [b] Site 2: [d] <........................................ [c] +---+ 5 Gbps +---+ 10 Gbps +----+ 20 Gbps | X |--------| Y |---------| GW |--------- Core +---+ +---+ +----+ .................... . . . v [e] [f] For these flows, Site 1 returns: a: { b: [ane1] }, c: { d: [ane1, ane2, ane3] } ane1: bw = 10 Gbps (link: A->B) ane2: bw = 10 Gbps (link: B->GW) ane3: bw = 50 Gbps (link: GW->Core) and Site 2 returns: c: { d: [anei, aneii, aneiii] } e: { f: [aneiv] } anei: bw = 5 Gbps (link Y->X) aneii: bw = 10 Gbps (link GW->Y) aneiii: bw = 20 Gbps (link Core->GW) aneiv: bw = 10 Gbps (link Y->GW) With this information, the data transfer scheduler can use algorithms such as the theory on bottleneck structure to predict the potential throughput of the flows.

Resource Exposure for CDNs and Service Edges At the time of this writing, a growing trend in today's applications is to bring storage and computation closer to the end users for better QoE, such as CDNs, augmented reality / virtual reality, and cloud gaming, as reported in various documents (e.g., and ). ISPs may deploy multiple layers of CDN caches or, more generally, service edges, with different latencies and available resources, including the number of CPU cores, memory, and storage. For example, illustrates a typical edge-cloud scenario where memory is measured in gigabytes (GB) and storage is measured in terabytes (TB). The "on-premise" edge nodes are closest to the end hosts and have the lowest latency, and the site-radio edge node and access central office (CO) have higher latencies but more available resources.

Example Use Case for Service Edge Exposure +-------------+ +----------------------+ | ALTO Client | <==========> | Application Provider | +-------------+ +----------------------+ PV | ^ PV | Request | | Response | Resource allocation, | | | service establishment, (End hosts | | (Edge nodes | etc. and cloud | | and metrics) | servers) | | | v | v +-------------+ +---------------------+ | ALTO Server | <=========> | Cloud-Edge Provider | +-------------+ +---------------------+ ____________________________________/\___________ / \ | (((o | | /_\ _~_ __ __ a (/\_/\) ( ) ( )~( )_ \ /------( )---------( )----\\---( ) _|_ / (______) (___) ( ) |_| -/ Site-radio Access CO (__________) /---\ Edge Node 1 | Cloud DC On premise | /---------/ (((o / | / Site-radio /_\ / Edge Node 2(/\_/\)-----/ /(_____)\ ___ / \ --- b--|_| -/ \--|_|--c /---\ /---\ On premise On premise With the extension defined in this document, an ALTO server can selectively reveal the CDNs and service edges that reside along the paths between different end hosts and/or the cloud servers, together with their properties (e.g., storage capabilities or Graphics Processing Unit (GPU) capabilities) and available Service Level Agreement (SLA) plans. See for an example where the query is made for sources [a, b] and destinations [b, c, DC]. Here, each ANE represents a service edge, and the properties include access latency, available resources, etc. Note that the properties here are only used for illustration purposes and are not part of this extension.

Example Service Edge Query Results a: { b: [ane1, ane2, ane3, ane4, ane5], c: [ane1, ane2, ane3, ane4, ane6], DC: [ane1, ane2, ane3] } b: { c: [ane5, ane4, ane6], DC: [ane5, ane4, ane3] } ane1: latency = 5 ms cpu = 2 memory = 8 GB storage = 10 TB (On premise, a) ane2: latency = 20 ms cpu = 4 memory = 8 GB storage = 10 TB (Site-radio Edge Node 1) ane3: latency = 100 ms cpu = 8 memory = 128 GB storage = 100 TB (Access CO) ane4: latency = 20 ms cpu = 4 memory = 8 GB storage = 10 TB (Site-radio Edge Node 2) ane5: latency = 5 ms cpu = 2 memory = 8 GB storage = 10 TB (On premise, b) ane6: latency = 5 ms cpu = 2 memory = 8 GB storage = 10 TB (On premise, c) With the service edge information, an ALTO client may better conduct CDN request routing or offload functionalities from the user equipment to the service edge, with considerations in place for customized quality of experience.

Path Vector Extension: Overview This section provides a non-normative overview of the Path Vector extension defined in this document. It is assumed that readers are familiar with both the base protocol and the entity property map extension . To satisfy the additional requirements listed in , this extension:

introduces the concept of an ANE as the abstraction of components in a network whose properties may have an impact on end-to-end performance of the traffic handled by those components,
extends the cost map and Endpoint Cost Service to convey the ANEs traversed by the path of a <source, destination> pair as Path Vectors, and
uses the entity property map to convey the association between the ANEs and their properties.

Thus, an ALTO client can learn about the ANEs that are important for assessing the QoE of different <source, destination> pairs by investigating the corresponding Path Vector value (AR1) and can also (1) identify common ANEs if an ANE appears in the Path Vectors of multiple <source, destination> pairs (AR2) and (2) retrieve the properties of the ANEs by searching the entity property map (AR3).

Abstract Network Element (ANE) This extension introduces the ANE as an indirect and network-agnostic way to specify a component or an aggregation of components of a network whose properties have an impact on end-to-end performance for application traffic between endpoints. ANEs allow ALTO servers to focus on common properties of different types of network components. For example, the throughput of a flow can be constrained by different components in a network: the capacity of a physical link, the maximum throughput of a firewall, the reserved bandwidth of an MPLS tunnel, etc. In the example below, assume that the throughput of the firewall is 100 Mbps and the capacity for link (A, B) is also 100 Mbps; they result in the same constraint on the total throughput of f1 and f2. Thus, they are identical when treated as an ANE. f1 | ^ f1 | | -----------------> +----------+ +---+ +---+ | Firewall | | A |-----| B | +----------+ +---+ +---+ | | -----------------> v | f2 f2 When an ANE is defined by an ALTO server, it is assigned an identifier by the ALTO server, i.e., a string of type ANEName as specified in , and a set of associated properties.

ANE Entity Domain In this extension, the associations between ANEs and their properties are conveyed in an entity property map. Thus, ANEs must constitute an "entity domain" (), and each ANE property must be an entity property (). Specifically, this document defines a new entity domain called "ane" as specified in ; defines two initial property types for the ANE entity domain.

Ephemeral and Persistent ANEs By design, ANEs are ephemeral and not to be used in further requests to other ALTO resources. More precisely, the corresponding ANE names are no longer valid beyond the scope of a Path Vector response or the incremental update stream for a Path Vector request. Compared with globally unique ANE names, ephemeral ANEs have several benefits, including better privacy for the ISP's internal structure and more flexible ANE computation. For example, an ALTO server may define an ANE for each aggregated bottleneck link between the sources and destinations specified in the request. For requests with different sources and destinations, the bottlenecks may be different but can safely reuse the same ANE names. The client can still adjust its traffic based on the information, but it is difficult to infer the underlying topology with multiple queries. However, sometimes an ISP may intend to selectively reveal some "persistent" network components that, as opposed to being ephemeral, have a longer life cycle. For example, an ALTO server may define an ANE for each service edge cluster. Once a client chooses to use a service edge, e.g., by deploying some user-defined functions, it may want to stick to the service edge to avoid the complexity of state transition or synchronization, and continuously query the properties of the edge cluster. This document provides a mechanism to expose such network components as persistent ANEs. A persistent ANE has a persistent ID that is registered in a property map, together with its properties. See Sections and for more detailed instructions on how to identify ephemeral ANEs and persistent ANEs.

Property Filtering Resource-constrained ALTO clients (see ) may benefit from the filtering of Path Vector query results at the ALTO server, as an ALTO client may only require a subset of the available properties. Specifically, the available properties for a given resource are announced in the Information Resource Directory (IRD) as a new filtering capability called "ane-property-names". The properties selected by a client as being of interest are specified in the subsequent Path Vector queries using the "ane-property-names" filter. The response only includes the selected properties for the ANEs. The "ane-property-names" capability for the cost map and the Endpoint Cost Service is specified in Sections and , respectively. The "ane-property-names" filter for the cost map and the Endpoint Cost Service is specified in Sections and accordingly.

Path Vector Cost Type For an ALTO client to correctly interpret the Path Vector, this extension specifies a new cost type called the "Path Vector cost type". The Path Vector cost type must convey both the interpretation and semantics in the "cost-mode" and "cost-metric" parameters, respectively. Unfortunately, a single "cost-mode" value cannot fully specify the interpretation of a Path Vector, which is a compound data type. For example, in programming languages such as C++, if there existed a JSON array type named JSONArray, a Path Vector would have the type of JSONArray<ANEName>. Instead of extending the "type system" of ALTO, this document takes a simple and backward-compatible approach. Specifically, the "cost-mode" of the Path Vector cost type is "array", which indicates that the value is a JSON array. Then, an ALTO client must check the value of the "cost-metric" parameter. If the value is "ane-path", it means that the JSON array should be further interpreted as a path of ANENames. The Path Vector cost type is specified in .

Multipart Path Vector Response For a basic ALTO information resource, a response contains only one type of ALTO resource, e.g., network map, cost map, or property map. Thus, only one round of communication is required: an ALTO client sends a request to an ALTO server, and the ALTO server returns a response, as shown in .

A Typical ALTO Request and Response ALTO client ALTO server |-------------- Request ---------------->| |<------------- Response ----------------| The extension defined in this document, on the other hand, involves two types of information resources: Path Vectors conveyed in an InfoResourceCostMap data component (defined in ) or an InfoResourceEndpointCostMap data component (defined in ), and ANE properties conveyed in an InfoResourceProperties data component (defined in ). Instead of two consecutive message exchanges, the extension defined in this document enforces one round of communication. Specifically, the ALTO client must include the source and destination pairs and the requested ANE properties in a single request, and the ALTO server must return a single response containing both the Path Vectors and properties associated with the ANEs in the Path Vectors, as shown in . Since the two parts are bundled together in one response message, their orders are interchangeable. See Sections and for details.

The Path Vector Extension Request and Response ALTO client ALTO server |------------- PV Request -------------->| |<----- PV Response (Cost Map Part) -----| |<--- PV Response (Property Map Part) ---| This design is based on the following considerations:

ANEs may be constructed on demand and, potentially, based on the requested properties (see for more details). If sources and destinations are not in the same request as the properties, an ALTO server either cannot construct ANEs on demand or must wait until both requests are received.
As ANEs may be constructed on demand, mappings of each ANE to its underlying network devices and resources can be specific to the request. In order to respond to the property map request correctly, an ALTO server must store the mapping of each Path Vector request until the client fully retrieves the property information. This "stateful" behavior may substantially harm server scalability and potentially lead to denial-of-service attacks.

One approach for realizing the one-round communication is to define a new media type to contain both objects, but this violates modular design. This document follows the standard-conforming usage of the "multipart/related" media type as defined in to elegantly combine the objects. Path Vectors are encoded in an InfoResourceCostMap data component or InfoResourceEndpointCostMap data component, and the property map is encoded in an InfoResourceProperties data component. They are encapsulated as parts of a multipart message. This modular composition allows ALTO servers and clients to reuse the data models of the existing information resources. Specifically, this document addresses the following practical issues using "multipart/related".

Identifying the Media Type of the Object Root ALTO uses a media type to indicate the type of an entry in the IRD (e.g., "application/alto-costmap+json" for the cost map and "application/alto-endpointcost+json" for the Endpoint Cost Service). Simply using "multipart/related" as the media type, however, makes it impossible for an ALTO client to identify the type of service provided by related entries. To address this issue, this document uses the "type" parameter to indicate the object root of a multipart/related message. For a cost map resource, the "media-type" field in the IRD entry is "multipart/related" with the parameter "type=application/alto-costmap+json"; for an Endpoint Cost Service, the parameter is "type=application/alto-endpointcost+json".

References to Part Messages As the response of a Path Vector resource is a multipart message with two different parts, it is important that each part can be uniquely identified. Following the design provided in , this extension requires that an ALTO server assign a unique identifier to each part of the multipart response message. This identifier, referred to as a Part Resource ID (see for details), is present in the part message's "Content-ID" header field. By concatenating the Part Resource ID to the identifier of the Path Vector request, an ALTO server/client can uniquely identify the Path Vector part or the property map part.

Specification: Basic Data Types

ANE Name An ANE name is encoded as a JSON string with the same format as that of the type PIDName (). The type ANEName is used in this document to indicate a string of this format.

ANE Entity Domain The ANE entity domain associates property values with the ANEs in a property map. Accordingly, the ANE entity domain always depends on a property map. It must be noted that the term "domain" here does not refer to a network domain. Rather, it is inherited from the entity domain as defined in ; the entity domain represents the set of valid entities defined by an ALTO information resource (called the "defining information resource").

Entity Domain Type The entity domain type is "ane".

Domain-Specific Entity Identifier The entity identifiers are the ANE names in the associated property map.

Hierarchy and Inheritance There is no hierarchy or inheritance for properties associated with ANEs.

Media Type of Defining Resource The defining resource for entity domain type "ane" MUST be a property map, i.e., the media type of defining resources is: application/alto-propmap+json Specifically, for ephemeral ANEs that appear in a Path Vector response, their entity domain names MUST be exactly ".ane", and the defining resource of these ANEs is the property map part of the multipart response. Meanwhile, for any persistent ANE whose defining resource is a property map resource, its entity domain name MUST have the format of "PROPMAP.ane", where PROPMAP is the resource ID of the defining resource. Persistent entities are "persistent" because standalone queries can be made by an ALTO client to their defining resource(s) when the connection to the Path Vector service is closed. For example, the defining resource of an ephemeral ANE whose entity identifier is ".ane:NET1" is the property map part that contains this identifier. The defining resource of a persistent ANE whose entity identifier is "dc-props.ane:DC1" is the property map with the resource ID "dc-props".

ANE Property Name An ANE property name is encoded as a JSON string with the same format as that of an entity property name ().

Initial ANE Property Types Two initial ANE property types are specified: "max-reservable-bandwidth" and "persistent-entity-id". Note that these property types do not depend on any information resources. As such, the "EntityPropertyName" parameter MUST only have the EntityPropertyType part.

Maximum Reservable Bandwidth The maximum reservable bandwidth property ("max-reservable-bandwidth") stands for the maximum bandwidth that can be reserved for all the traffic that traverses an ANE. The value MUST be encoded as a non-negative numerical cost value as defined in , and the unit is bits per second (bps). If this property is requested by the ALTO client but is not present for an ANE in the server response, it MUST be interpreted as meaning that the property is not defined for the ANE. This property can be offered in a setting where the ALTO server is part of a network system that provides on-demand resource allocation and the ALTO client is part of a user application. One existing example is : the ALTO server is part of a Software-Defined Networking (SDN) controller and exposes a list of traversed network elements and associated link bandwidth to the client. The encoding in differs from the Path Vector response defined in this document in that the Path Vector part and property map part are placed in the same JSON object. In such a framework, the ALTO server exposes resource availability information (e.g., reservable bandwidth) to the ALTO client. How the client makes resource requests based on the information, and how the resource allocation is achieved, respectively, depend on interfaces between the management system and the users or a higher-layer protocol (e.g., SDN network intents or MPLS tunnels), which are out of scope for this document.

Persistent Entity ID This document enables the discovery of a persistent ANE by exposing its entity identifier as the persistent entity ID property of an ephemeral ANE in the path vector response. The value of this property is encoded with the EntityID format defined in . In this format, the entity ID combines:

a defining information resource for the ANE on which a "persistent-entity-id" is queried, which is the property map resource defining the ANE as a persistent entity, together with the properties.
the persistent name of the ANE in that property map.

With this format, the client has all the needed information for further standalone query properties on the persistent ANE.

Examples To illustrate the use of "max-reservable-bandwidth", consider the following network with five nodes. Assume that the client wants to query the maximum reservable bandwidth from H1 to H2. An ALTO server may split the network into two ANEs: "ane1", which represents the subnetwork with routers A, B, and C; and "ane2", which represents the subnetwork with routers B, D, and E. The maximum reservable bandwidth for "ane1" is 15 Mbps (using path A->C->B), and the maximum reservable bandwidth for "ane2" is 20 Mbps (using path B->D->E). 20 Mbps 20 Mbps 10 Mbps +---+ +---+ +---+ /----| B |---| D |----| E |---- H2 +---+/ +---+ +---+ +---+ H1 ----| A | 15 Mbps| +---+\ +---+ \----| C | 15 Mbps +---+ To illustrate the use of "persistent-entity-id", consider the scenario in . As the life cycles of service edges are typically long, the service edges may contain information that is not specific to the query. Such information can be stored in an individual entity property map and can later be accessed by an ALTO client. For example, "ane1" in represents the on-premise service edge closest to host "a". Assume that the properties of the service edges are provided in an entity property map called "se-props" and the ID of the on-premise service edge is "9a0b55f7-7442-4d56-8a2c-b4cc6a8e3aa1"; the "persistent-entity-id" setting for "ane1" will be "se-props.ane:9a0b55f7-7442-4d56-8a2c-b4cc6a8e3aa1". With this persistent entity ID, an ALTO client may send queries to the "se-props" resource with the entity ID ".ane:9a0b55f7-7442-4d56-8a2c-b4cc6a8e3aa1".

Path Vector Cost Type This document defines a new cost type, which is referred to as the Path Vector cost type. An ALTO server MUST offer this cost type if it supports the extension defined in this document.

Cost Metric: "ane-path" The cost metric "ane-path" indicates that the value of such a cost type conveys an array of ANE names, where each ANE name uniquely represents an ANE traversed by traffic from a source to a destination. An ALTO client MUST interpret the Path Vector as if the traffic between a source and a destination logically traverses the ANEs in the same order as they appear in the Path Vector. When the Path Vector procedures defined in this document are in use, an ALTO server using the "ane-path" cost metric and the "array" cost mode (see ) MUST return as the cost value a JSON array of data type ANEName, and the client MUST also check that each element contained in the array is an ANEName (). Otherwise, the client MUST discard the response and SHOULD follow the guidance in to handle the error.

Cost Mode: "array" The cost mode "array" indicates that every cost value in the response body of a (filtered) cost map or an Endpoint Cost Service MUST be interpreted as a JSON array object. While this cost mode can be applied to all cost metrics, additional specifications will be needed to clarify the semantics of the "array" cost mode when combined with cost metrics other than "ane-path".

Part Resource ID and Part Content ID A Part Resource ID is encoded as a JSON string with the same format as that of the type ResourceID (). Even though the "client-id" assigned to a Path Vector request and the Part Resource ID MAY contain up to 64 characters by their own definition, their concatenation (see ) MUST also conform to the same length constraint. The same requirement applies to the resource ID of the Path Vector resource, too. Thus, it is RECOMMENDED to limit the length of the resource ID and client ID related to a Path Vector resource to 31 characters. A Part Content ID conforms to the format of "msg-id" as specified in and . Specifically, it has the following format: "<" PART-RESOURCE-ID "@" DOMAIN-NAME ">"

PART-RESOURCE-ID:: PART-RESOURCE-ID has the same format as the Part Resource ID. It is used to identify whether a part message is a Path Vector or a property map.
DOMAIN-NAME:: DOMAIN-NAME has the same format as "dot-atom-text" as specified in . It must be the domain name of the ALTO server.

Specification: Service Extensions

Notation This document uses the same syntax and notation as those introduced in to specify the extensions to existing ALTO resources and services.

Multipart Filtered Cost Map for Path Vector This document introduces a new ALTO resource called the "multipart filtered cost map resource", which allows an ALTO server to provide other ALTO resources associated with the cost map resource in the same response.

Media Type The media type of the multipart filtered cost map resource is "multipart/related", and the required "type" parameter MUST have a value of "application/alto-costmap+json".

HTTP Method The multipart filtered cost map is requested using the HTTP POST method.

Accept Input Parameters The input parameters of the multipart filtered cost map are supplied in the body of an HTTP POST request. This document extends the input parameters to a filtered cost map, which is defined as a JSON object of type ReqFilteredCostMap in , with a data format indicated by the media type "application/alto-costmapfilter+json", which is a JSON object of type PVReqFilteredCostMap: object { [EntityPropertyName ane-property-names<0..*>;] } PVReqFilteredCostMap : ReqFilteredCostMap; with field:

ane-property-names:: This field provides a list of selected ANE properties to be included in the response. Each property in this list MUST match one of the supported ANE properties indicated in the resource's "ane-property-names" capability (). If the field is not present, it MUST be interpreted as an empty list.

Example: Consider the network in . If an ALTO client wants to query the "max-reservable-bandwidth" setting between PID1 and PID2, it can submit the following request. POST /costmap/pv HTTP/1.1 Host: alto.example.com Accept: multipart/related;type=application/alto-costmap+json, application/alto-error+json Content-Length: 212 Content-Type: application/alto-costmapfilter+json { "cost-type": { "cost-mode": "array", "cost-metric": "ane-path" }, "pids": { "srcs": [ "PID1" ], "dsts": [ "PID2" ] }, "ane-property-names": [ "max-reservable-bandwidth" ] }

Capabilities The multipart filtered cost map resource extends the capabilities defined in . The capabilities are defined by a JSON object of type PVFilteredCostMapCapabilities: object { [EntityPropertyName ane-property-names<0..*>;] } PVFilteredCostMapCapabilities : FilteredCostMapCapabilities; with field:

ane-property-names:: This field provides a list of ANE properties that can be returned. If the field is not present, it MUST be interpreted as an empty list, indicating that the ALTO server cannot provide any ANE properties.

This extension also introduces additional restrictions for the following fields:

cost-type-names:: The "cost-type-names" field MUST include the Path Vector cost type, unless explicitly documented by a future extension. This also implies that the Path Vector cost type MUST be defined in the "cost-types" of the IRD's "meta" field.
cost-constraints:: If the "cost-type-names" field includes the Path Vector cost type, the "cost-constraints" field MUST be either "false" or not present, unless specifically instructed by a future document.
testable-cost-type-names ():: If the "cost-type-names" field includes the Path Vector cost type and the "testable-cost-type-names" field is present, the Path Vector cost type MUST NOT be included in the "testable-cost-type-names" field unless specifically instructed by a future document.

Uses This member MUST include the resource ID of the network map based on which the PIDs are defined. If this resource supports "persistent-entity-id", it MUST also include the defining resources of persistent ANEs that may appear in the response.

Response The response MUST indicate an error, using ALTO Protocol error handling as defined in , if the request is invalid. The "Content-Type" header field of the response MUST be "multipart/related" as defined by , with the following parameters:

type:: The "type" parameter is mandatory and MUST be "application/alto-costmap+json". Note that permits parameters both with and without double quotes.
start:: The "start" parameter is as defined in and is optional. If present, it MUST have the same value as the "Content-ID" header field of the Path Vector part.
boundary:: The "boundary" parameter is as defined in and is mandatory.

The body of the response MUST consist of two parts:

The Path Vector part MUST include "Content-ID" and "Content-Type" in its header. The "Content-Type" MUST be "application/alto-costmap+json". The value of "Content-ID" MUST have the same format as the Part Content ID as specified in . The body of the Path Vector part MUST be a JSON object with the same format as that defined in when the "cost-type" field is present in the input parameters and MUST be a JSON object with the same format as that defined in if the "multi-cost-types" field is present. The JSON object MUST include the "vtag" field in the "meta" field, which provides the version tag of the returned CostMapData object. The resource ID of the version tag MUST follow the format of resource-id '.' part-resource-id where "resource-id" is the resource ID of the Path Vector resource and "part-resource-id" has the same value as the PART-RESOURCE-ID in the "Content-ID" of the Path Vector part. The "meta" field MUST also include the "dependent-vtags" field, whose value is a single-element array to indicate the version tag of the network map used, where the network map is specified in the "uses" attribute of the multipart filtered cost map resource in the IRD.
The entity property map part MUST also include "Content-ID" and "Content-Type" in its header. The "Content-Type" MUST be "application/alto-propmap+json". The value of "Content-ID" MUST have the same format as the Part Content ID as specified in . The body of the entity property map part is a JSON object with the same format as that defined in . The JSON object MUST include the "dependent-vtags" field in the "meta" field. The value of the "dependent-vtags" field MUST be an array of VersionTag objects as defined by . The "vtag" of the Path Vector part MUST be included in the "dependent-vtags" field. If "persistent-entity-id" is requested, the version tags of the dependent resources that may expose the entities in the response MUST also be included. The PropertyMapData object has one member for each ANEName that appears in the Path Vector part, which is an entity identifier belonging to the self-defined entity domain as defined in . The EntityProps object for each ANE has one member for each property that is both 1) associated with the ANE and 2) specified in the "ane-property-names" field in the request. If the Path Vector cost type is not included in the "cost-type" field or the "multi-cost-type" field, the "property-map" field MUST be present and the value MUST be an empty object ({}).

A complete and valid response MUST include both the Path Vector part and the property map part in the multipart message. If any part is not present, the client MUST discard the received information and send another request if necessary. The Path Vector part, whose media type is the same as the "type" parameter of the multipart response message, is the root body part as defined in . Thus, it is the element that the application processes first. Even though the "start" parameter allows it to be placed anywhere in the part sequence, it is RECOMMENDED that the parts arrive in the same order as they are processed, i.e., the Path Vector part is always placed as the first part, followed by the property map part. When doing so, an ALTO server MAY choose not to set the "start" parameter, which implies that the first part is the object root. Example: Consider the network in . The response to the example request in is as follows, where "ANE1" represents the aggregation of all the switches in the network. HTTP/1.1 200 OK Content-Length: 911 Content-Type: multipart/related; boundary=example-1; type=application/alto-costmap+json --example-1 Content-ID: <costmap@alto.example.com> Content-Type: application/alto-costmap+json { "meta": { "vtag": { "resource-id": "filtered-cost-map-pv.costmap", "tag": "fb20b76204814e9db37a51151faaaef2" }, "dependent-vtags": [ { "resource-id": "my-default-networkmap", "tag": "75ed013b3cb58f896e839582504f6228" } ], "cost-type": { "cost-mode": "array", "cost-metric": "ane-path" } }, "cost-map": { "PID1": { "PID2": [ "ANE1" ] } } } --example-1 Content-ID: <propmap@alto.example.com> Content-Type: application/alto-propmap+json { "meta": { "dependent-vtags": [ { "resource-id": "filtered-cost-map-pv.costmap", "tag": "fb20b76204814e9db37a51151faaaef2" } ] }, "property-map": { ".ane:ANE1": { "max-reservable-bandwidth": 100000000 } } } --example-1

Multipart Endpoint Cost Service for Path Vector This document introduces a new ALTO resource called the "multipart Endpoint Cost Service", which allows an ALTO server to provide other ALTO resources associated with the Endpoint Cost Service resource in the same response.

Media Type The media type of the multipart Endpoint Cost Service resource is "multipart/related", and the required "type" parameter MUST have a value of "application/alto-endpointcost+json".

HTTP Method The multipart Endpoint Cost Service resource is requested using the HTTP POST method.

Accept Input Parameters The input parameters of the multipart Endpoint Cost Service resource are supplied in the body of an HTTP POST request. This document extends the input parameters to an Endpoint Cost Service, which is defined as a JSON object of type ReqEndpointCostMap in , with a data format indicated by the media type "application/alto-endpointcostparams+json", which is a JSON object of type PVReqEndpointCostMap: object { [EntityPropertyName ane-property-names<0..*>;] } PVReqEndpointCostMap : ReqEndpointCostMap; with field:

ane-property-names:: This document defines the "ane-property-names" field in PVReqEndpointCostMap as being the same as in PVReqFilteredCostMap. See .

Example: Consider the network in . If an ALTO client wants to query the "max-reservable-bandwidth" setting between "eh1" and "eh2", it can submit the following request. POST /ecs/pv HTTP/1.1 Host: alto.example.com Accept: multipart/related;type=application/alto-endpointcost+json, application/alto-error+json Content-Length: 238 Content-Type: application/alto-endpointcostparams+json { "cost-type": { "cost-mode": "array", "cost-metric": "ane-path" }, "endpoints": { "srcs": [ "ipv4:192.0.2.2" ], "dsts": [ "ipv4:192.0.2.18" ] }, "ane-property-names": [ "max-reservable-bandwidth" ] }

Capabilities The capabilities of the multipart Endpoint Cost Service resource are defined by a JSON object of type PVEndpointCostCapabilities, which is defined as being the same as PVFilteredCostMapCapabilities. See .

Uses If this resource supports "persistent-entity-id", it MUST also include the defining resources of persistent ANEs that may appear in the response.

type:: The "type" parameter MUST be "application/alto-endpointcost+json" and is mandatory.
start:: The "start" parameter is as defined in .
boundary:: The "boundary" parameter is as defined in and is mandatory.

The body of the response MUST consist of two parts:

The Path Vector part MUST include "Content-ID" and "Content-Type" in its header. The "Content-Type" MUST be "application/alto-endpointcost+json". The value of "Content-ID" MUST have the same format as the Part Content ID as specified in . The body of the Path Vector part MUST be a JSON object with the same format as that defined in when the "cost-type" field is present in the input parameters and MUST be a JSON object with the same format as that defined in if the "multi-cost-types" field is present. The JSON object MUST include the "vtag" field in the "meta" field, which provides the version tag of the returned EndpointCostMapData object. The resource ID of the version tag MUST follow the format of resource-id '.' part-resource-id where "resource-id" is the resource ID of the Path Vector resource and "part-resource-id" has the same value as the PART-RESOURCE-ID in the "Content-ID" of the Path Vector part.
The entity property map part MUST also include "Content-ID" and "Content-Type" in its header. The "Content-Type" MUST be "application/alto-propmap+json". The value of "Content-ID" MUST have the same format as the Part Content ID as specified in . The body of the entity property map part MUST be a JSON object with the same format as that defined in . The JSON object MUST include the "dependent-vtags" field in the "meta" field. The value of the "dependent-vtags" field MUST be an array of VersionTag objects as defined by . The "vtag" of the Path Vector part MUST be included in the "dependent-vtags" field. If "persistent-entity-id" is requested, the version tags of the dependent resources that may expose the entities in the response MUST also be included. The PropertyMapData object has one member for each ANEName that appears in the Path Vector part, which is an entity identifier belonging to the self-defined entity domain as defined in . The EntityProps object for each ANE has one member for each property that is both 1) associated with the ANE and 2) specified in the "ane-property-names" field in the request. If the Path Vector cost type is not included in the "cost-type" field or the "multi-cost-type" field, the "property-map" field MUST be present and the value MUST be an empty object ({}).

A complete and valid response MUST include both the Path Vector part and the property map part in the multipart message. If any part is not present, the client MUST discard the received information and send another request if necessary. The Path Vector part, whose media type is the same as the "type" parameter of the multipart response message, is the root body part as defined in . Thus, it is the element that the application processes first. Even though the "start" parameter allows it to be placed anywhere in the part sequence, it is RECOMMENDED that the parts arrive in the same order as they are processed, i.e., the Path Vector part is always placed as the first part, followed by the property map part. When doing so, an ALTO server MAY choose not to set the "start" parameter, which implies that the first part is the object root. Example: Consider the network in . The response to the example request in is as follows. HTTP/1.1 200 OK Content-Length: 899 Content-Type: multipart/related; boundary=example-1; type=application/alto-endpointcost+json --example-1 Content-ID: <ecs@alto.example.com> Content-Type: application/alto-endpointcost+json { "meta": { "vtag": { "resource-id": "ecs-pv.ecs", "tag": "ec137bb78118468c853d5b622ac003f1" }, "dependent-vtags": [ { "resource-id": "my-default-networkmap", "tag": "677fe5f4066848d282ece213a84f9429" } ], "cost-type": { "cost-mode": "array", "cost-metric": "ane-path" } }, "cost-map": { "ipv4:192.0.2.2": { "ipv4:192.0.2.18": [ "ANE1" ] } } } --example-1 Content-ID: <propmap@alto.example.com> Content-Type: application/alto-propmap+json { "meta": { "dependent-vtags": [ { "resource-id": "ecs-pv.ecs", "tag": "ec137bb78118468c853d5b622ac003f1" } ] }, "property-map": { ".ane:ANE1": { "max-reservable-bandwidth": 100000000 } } } --example-1

Examples This section lists some examples of Path Vector queries and the corresponding responses. Some long lines are truncated for better readability.

Sample Setup illustrates the network properties and thus the message contents. There are three subnetworks (NET1, NET2, and NET3) and two interconnection links (L1 and L2). It is assumed that each subnetwork has sufficiently large bandwidth to be reserved.

Examples of ANE Properties ----- L1 / PID1 +----------+ 10 Gbps +----------+ PID3 192.0.2.0/28+-+ +------+ +---------+ +--+192.0.2.32/28 | | MEC1 | | | | 2001:db8::3:0/16 | +------+ | +-----+ | PID2 | | | +----------+ 192.0.2.16/28+-+ | | NET3 | | | 15 Gbps | | | \ +----------+ | -------- L2 NET1 | +----------+ | +------+ | PID4 | | MEC2 | +--+192.0.2.48/28 | +------+ | 2001:db8::4:0/16 +----------+ NET2

Information Resource Directory To give a comprehensive example of the extension defined in this document, we consider the network in . Assume that the ALTO server provides the following information resources:

"my-default-networkmap":: A network map resource that contains the PIDs in the network.
"filtered-cost-map-pv":: A multipart filtered cost map resource for the Path Vector. Exposes the "max-reservable-bandwidth" property for the PIDs in "my-default-networkmap".
"ane-props":: A filtered entity property resource that exposes the information for persistent ANEs in the network.
"endpoint-cost-pv":: A multipart Endpoint Cost Service for the Path Vector. Exposes the "max-reservable-bandwidth" and "persistent-entity-id" properties.
"update-pv":: An update stream service that provides the incremental update service for the "endpoint-cost-pv" service.
"multicost-pv":: A multipart Endpoint Cost Service with both the Multi-Cost extension and Path Vector extension enabled.

Below is the IRD of the example ALTO server. To enable the extension defined in this document, the Path Vector cost type (), represented by "path-vector" below, is defined in the "cost-types" of the "meta" field and is included in the "cost-type-names" of resources "filtered-cost-map-pv" and "endpoint-cost-pv". { "meta": { "cost-types": { "path-vector": { "cost-mode": "array", "cost-metric": "ane-path" }, "num-rc": { "cost-mode": "numerical", "cost-metric": "routingcost" } } }, "resources": { "my-default-networkmap": { "uri": "https://alto.example.com/networkmap", "media-type": "application/alto-networkmap+json" }, "filtered-cost-map-pv": { "uri": "https://alto.example.com/costmap/pv", "media-type": "multipart/related; type=application/alto-costmap+json", "accepts": "application/alto-costmapfilter+json", "capabilities": { "cost-type-names": [ "path-vector" ], "ane-property-names": [ "max-reservable-bandwidth" ] }, "uses": [ "my-default-networkmap" ] }, "ane-props": { "uri": "https://alto.example.com/ane-props", "media-type": "application/alto-propmap+json", "accepts": "application/alto-propmapparams+json", "capabilities": { "mappings": { ".ane": [ "cpu" ] } } }, "endpoint-cost-pv": { "uri": "https://alto.exmaple.com/endpointcost/pv", "media-type": "multipart/related; type=application/alto-endpointcost+json", "accepts": "application/alto-endpointcostparams+json", "capabilities": { "cost-type-names": [ "path-vector" ], "ane-property-names": [ "max-reservable-bandwidth", "persistent-entity-id" ] }, "uses": [ "ane-props" ] }, "update-pv": { "uri": "https://alto.example.com/updates/pv", "media-type": "text/event-stream", "uses": [ "endpoint-cost-pv" ], "accepts": "application/alto-updatestreamparams+json", "capabilities": { "support-stream-control": true } }, "multicost-pv": { "uri": "https://alto.exmaple.com/endpointcost/mcpv", "media-type": "multipart/related; type=application/alto-endpointcost+json", "accepts": "application/alto-endpointcostparams+json", "capabilities": { "cost-type-names": [ "path-vector", "num-rc" ], "max-cost-types": 2, "testable-cost-type-names": [ "num-rc" ], "ane-property-names": [ "max-reservable-bandwidth", "persistent-entity-id" ] }, "uses": [ "ane-props" ] } } }

Multipart Filtered Cost Map The following examples demonstrate the request to the "filtered-cost-map-pv" resource and the corresponding response. The request uses the "path-vector" cost type in the "cost-type" field. The "ane-property-names" field is missing, indicating that the client only requests the Path Vector and not the ANE properties. The response consists of two parts:

The first part returns the array of data type ANEName for each source and destination pair. There are two ANEs, where "L1" represents interconnection link L1 and "L2" represents interconnection link L2.
The second part returns the property map. Note that the properties of the ANE entries are equal to the literal string "{}" (see ).

POST /costmap/pv HTTP/1.1 Host: alto.example.com Accept: multipart/related;type=application/alto-costmap+json, application/alto-error+json Content-Length: 163 Content-Type: application/alto-costmapfilter+json { "cost-type": { "cost-mode": "array", "cost-metric": "ane-path" }, "pids": { "srcs": [ "PID1" ], "dsts": [ "PID3", "PID4" ] } } HTTP/1.1 200 OK Content-Length: 952 Content-Type: multipart/related; boundary=example-1; type=application/alto-costmap+json --example-1 Content-ID: <costmap@alto.example.com> Content-Type: application/alto-costmap+json { "meta": { "vtag": { "resource-id": "filtered-cost-map-pv.costmap", "tag": "d827f484cb66ce6df6b5077cb8562b0a" }, "dependent-vtags": [ { "resource-id": "my-default-networkmap", "tag": "c04bc5da49534274a6daeee8ea1dec62" } ], "cost-type": { "cost-mode": "array", "cost-metric": "ane-path" } }, "cost-map": { "PID1": { "PID3": [ "L1" ], "PID4": [ "L1", "L2" ] } } } --example-1 Content-ID: <propmap@alto.example.com> Content-Type: application/alto-propmap+json { "meta": { "dependent-vtags": [ { "resource-id": "filtered-cost-map-pv.costmap", "tag": "d827f484cb66ce6df6b5077cb8562b0a" } ] }, "property-map": { ".ane:L1": {}, ".ane:L2": {} } } --example-1

Multipart Endpoint Cost Service Resource The following examples demonstrate the request to the "endpoint-cost-pv" resource and the corresponding response. The request uses the "path-vector" cost type in the "cost-type" field and queries the maximum reservable bandwidth ANE property and the persistent entity ID property for two IPv4 source and destination pairs (192.0.2.34 -> 192.0.2.2 and 192.0.2.34 -> 192.0.2.50) and one IPv6 source and destination pair (2001:db8::3:1 -> 2001:db8::4:1). The response consists of two parts:

The first part returns the array of data type ANEName for each valid source and destination pair. As one can see in , flow 192.0.2.34 -> 192.0.2.2 traverses NET3, L1, and NET1; and flows 192.0.2.34 -> 192.0.2.50 and 2001:db8::3:1 -> 2001:db8::4:1 traverse NET2, L2, and NET3.
The second part returns the requested properties of ANEs. Assume that NET1, NET2, and NET3 have sufficient bandwidth and their "max-reservable-bandwidth" values are set to a sufficiently large number (50 Gbps in this case). On the other hand, assume that there are no prior reservations on L1 and L2 and their "max-reservable-bandwidth" values are the corresponding link capacity (10 Gbps for L1 and 15 Gbps for L2).

Both NET1 and NET2 have a mobile edge deployed, i.e., MEC1 in NET1 and MEC2 in NET2. Assume that the ANEName values for MEC1 and MEC2 are "MEC1" and "MEC2" and their properties can be retrieved from the property map "ane-props". Thus, the "persistent-entity-id" property values for NET1 and NET2 are "ane-props.ane:MEC1" and "ane-props.ane:MEC2", respectively. POST /endpointcost/pv HTTP/1.1 Host: alto.example.com Accept: multipart/related; type=application/alto-endpointcost+json, application/alto-error+json Content-Length: 383 Content-Type: application/alto-endpointcostparams+json { "cost-type": { "cost-mode": "array", "cost-metric": "ane-path" }, "endpoints": { "srcs": [ "ipv4:192.0.2.34", "ipv6:2001:db8::3:1" ], "dsts": [ "ipv4:192.0.2.2", "ipv4:192.0.2.50", "ipv6:2001:db8::4:1" ] }, "ane-property-names": [ "max-reservable-bandwidth", "persistent-entity-id" ] } HTTP/1.1 200 OK Content-Length: 1508 Content-Type: multipart/related; boundary=example-2; type=application/alto-endpointcost+json --example-2 Content-ID: <ecs@alto.example.com> Content-Type: application/alto-endpointcost+json { "meta": { "vtags": { "resource-id": "endpoint-cost-pv.ecs", "tag": "bb6bb72eafe8f9bdc4f335c7ed3b10822a391cef" }, "cost-type": { "cost-mode": "array", "cost-metric": "ane-path" } }, "endpoint-cost-map": { "ipv4:192.0.2.34": { "ipv4:192.0.2.2": [ "NET3", "L1", "NET1" ], "ipv4:192.0.2.50": [ "NET3", "L2", "NET2" ] }, "ipv6:2001:db8::3:1": { "ipv6:2001:db8::4:1": [ "NET3", "L2", "NET2" ] } } } --example-2 Content-ID: <propmap@alto.example.com> Content-Type: application/alto-propmap+json { "meta": { "dependent-vtags": [ { "resource-id": "endpoint-cost-pv.ecs", "tag": "bb6bb72eafe8f9bdc4f335c7ed3b10822a391cef" }, { "resource-id": "ane-props", "tag": "bf3c8c1819d2421c9a95a9d02af557a3" } ] }, "property-map": { ".ane:NET1": { "max-reservable-bandwidth": 50000000000, "persistent-entity-id": "ane-props.ane:MEC1" }, ".ane:NET2": { "max-reservable-bandwidth": 50000000000, "persistent-entity-id": "ane-props.ane:MEC2" }, ".ane:NET3": { "max-reservable-bandwidth": 50000000000 }, ".ane:L1": { "max-reservable-bandwidth": 10000000000 }, ".ane:L2": { "max-reservable-bandwidth": 15000000000 } } } --example-2 In certain scenarios where the traversal order is not crucial, an ALTO server implementation may choose not to strictly follow the physical traversal order and may even obfuscate the order intentionally to preserve its own privacy or conform to its own policies. For example, an ALTO server may choose to aggregate NET1 and L1 as a new ANE with ANE name "AGGR1" and aggregate NET2 and L2 as a new ANE with ANE name "AGGR2". The "max-reservable-bandwidth" property of "AGGR1" takes the value of L1, which is smaller than that of NET1, and the "persistent-entity-id" property of "AGGR1" takes the value of NET1. The properties of "AGGR2" are computed in a similar way; the obfuscated response is as shown below. Note that the obfuscation of Path Vector responses is implementation specific and is out of scope for this document. Developers may refer to for further references. HTTP/1.1 200 OK Content-Length: 1333 Content-Type: multipart/related; boundary=example-2; type=application/alto-endpointcost+json --example-2 Content-ID: <ecs@alto.example.com> Content-Type: application/alto-endpointcost+json { "meta": { "vtags": { "resource-id": "endpoint-cost-pv.ecs", "tag": "bb975862fbe3422abf4dae386b132c1d" }, "cost-type": { "cost-mode": "array", "cost-metric": "ane-path" } }, "endpoint-cost-map": { "ipv4:192.0.2.34": { "ipv4:192.0.2.2": [ "NET3", "AGGR1" ], "ipv4:192.0.2.50": [ "NET3", "AGGR2" ] }, "ipv6:2001:db8::3:1": { "ipv6:2001:db8::4:1": [ "NET3", "AGGR2" ] } } } --example-2 Content-ID: <propmap@alto.example.com> Content-Type: application/alto-propmap+json { "meta": { "dependent-vtags": [ { "resource-id": "endpoint-cost-pv.ecs", "tag": "bb975862fbe3422abf4dae386b132c1d" }, { "resource-id": "ane-props", "tag": "bf3c8c1819d2421c9a95a9d02af557a3" } ] }, "property-map": { ".ane:AGGR1": { "max-reservable-bandwidth": 10000000000, "persistent-entity-id": "ane-props.ane:MEC1" }, ".ane:AGGR2": { "max-reservable-bandwidth": 15000000000, "persistent-entity-id": "ane-props.ane:MEC2" }, ".ane:NET3": { "max-reservable-bandwidth": 50000000000 } } } --example-2

Incremental Updates In this example, an ALTO client subscribes to the incremental update for the multipart Endpoint Cost Service resource "endpoint-cost-pv". POST /updates/pv HTTP/1.1 Host: alto.example.com Accept: text/event-stream Content-Type: application/alto-updatestreamparams+json Content-Length: 120 { "add": { "ecspvsub1": { "resource-id": "endpoint-cost-pv", "input": <ecs-input> } } } Based on the server-side process defined in , the ALTO server will send the "control-uri" first, using a Server-Sent Event (SSE) followed by the full response of the multipart message. HTTP/1.1 200 OK Connection: keep-alive Content-Type: text/event-stream event: application/alto-updatestreamcontrol+json data: {"control-uri": "https://alto.example.com/updates/streams/123"} event: multipart/related;boundary=example-3; type=application/alto-endpointcost+json,ecspvsub1 data: --example-3 data: Content-ID: <ecsmap@alto.example.com> data: Content-Type: application/alto-endpointcost+json data: data: <endpoint-cost-map-entry> data: --example-3 data: Content-ID: <propmap@alto.example.com> data: Content-Type: application/alto-propmap+json data: data: <property-map-entry> data: --example-3-- When the contents change, the ALTO server will publish the updates for each node in this tree separately, based on . event: application/merge-patch+json, ecspvsub1.ecsmap@alto.example.com data: <Merge patch for endpoint-cost-map-update> event: application/merge-patch+json, ecspvsub1.propmap@alto.example.com data: <Merge patch for property-map-update>

Multi-Cost The following examples demonstrate the request to the "multicost-pv" resource and the corresponding response. The request asks for two cost types: the first is the Path Vector cost type, and the second is a numerical routing cost. It also queries the maximum reservable bandwidth ANE property and the persistent entity ID property for two IPv4 source and destination pairs (192.0.2.34 -> 192.0.2.2 and 192.0.2.34 -> 192.0.2.50) and one IPv6 source and destination pair (2001:db8::3:1 -> 2001:db8::4:1). The response consists of two parts:

The first part returns a JSONArray that contains two JSONValue entries for each requested source and destination pair: the first JSONValue is a JSONArray of ANENames, which is the value of the Path Vector cost type; and the second JSONValue is a JSONNumber, which is the value of the routing cost.
The second part contains a property map that maps the ANEs to their requested properties.

POST /endpointcost/mcpv HTTP/1.1 Host: alto.example.com Accept: multipart/related; type=application/alto-endpointcost+json, application/alto-error+json Content-Length: 454 Content-Type: application/alto-endpointcostparams+json { "multi-cost-types": [ { "cost-mode": "array", "cost-metric": "ane-path" }, { "cost-mode": "numerical", "cost-metric": "routingcost" } ], "endpoints": { "srcs": [ "ipv4:192.0.2.34", "ipv6:2001:db8::3:1" ], "dsts": [ "ipv4:192.0.2.2", "ipv4:192.0.2.50", "ipv6:2001:db8::4:1" ] }, "ane-property-names": [ "max-reservable-bandwidth", "persistent-entity-id" ] } HTTP/1.1 200 OK Content-Length: 1419 Content-Type: multipart/related; boundary=example-4; type=application/alto-endpointcost+json --example-4 Content-ID: <ecs@alto.example.com> Content-Type: application/alto-endpointcost+json { "meta": { "vtags": { "resource-id": "endpoint-cost-pv.ecs", "tag": "84a4f9c14f9341f0983e3e5f43a371c8" }, "multi-cost-types": [ { "cost-mode": "array", "cost-metric": "ane-path" }, { "cost-mode": "numerical", "cost-metric": "routingcost" } ] }, "endpoint-cost-map": { "ipv4:192.0.2.34": { "ipv4:192.0.2.2": [[ "NET3", "AGGR1" ], 3], "ipv4:192.0.2.50": [[ "NET3", "AGGR2" ], 2] }, "ipv6:2001:db8::3:1": { "ipv6:2001:db8::4:1": [[ "NET3", "AGGR2" ], 2] } } } --example-4 Content-ID: <propmap@alto.example.com> Content-Type: application/alto-propmap+json { "meta": { "dependent-vtags": [ { "resource-id": "endpoint-cost-pv.ecs", "tag": "84a4f9c14f9341f0983e3e5f43a371c8" }, { "resource-id": "ane-props", "tag": "be157afa031443a187b60bb80a86b233" } ] }, "property-map": { ".ane:AGGR1": { "max-reservable-bandwidth": 10000000000, "persistent-entity-id": "ane-props.ane:MEC1" }, ".ane:AGGR2": { "max-reservable-bandwidth": 15000000000, "persistent-entity-id": "ane-props.ane:MEC2" }, ".ane:NET3": { "max-reservable-bandwidth": 50000000000 } } } --example-4

Compatibility with Other ALTO Extensions

Compatibility with Legacy ALTO Clients/Servers The multipart filtered cost map resource and the multipart Endpoint Cost Service resource have no backward-compatibility issues with legacy ALTO clients and servers. Although these two types of resources reuse the media types defined in the base ALTO Protocol for the "Accept" input parameters, they have different media types for responses. If the ALTO server provides these two types of resources but the ALTO client does not support them, the ALTO client will ignore the resources without incurring any incompatibility problems.

Compatibility with Multi-Cost Extension The extension defined in this document is compatible with the multi-cost extension . Such a resource has a media type of either "multipart/related; type=application/alto-costmap+json" or "multipart/related; type=application/alto-endpointcost+json". Its "cost-constraints" field must be either "false" or not present, and the Path Vector cost type must be present in the "cost-type-names" capability field but must not be present in the "testable-cost-type-names" field, as specified in Sections and .

Compatibility with Incremental Update Extension This extension is compatible with the incremental update extension . ALTO clients and servers MUST follow the specifications given in Sections and of to support incremental updates for a Path Vector resource.

Compatibility with Cost Calendar Extension The extension specified in this document is compatible with the Cost Calendar extension . When used together with the Cost Calendar extension, the cost value between a source and a destination is an array of Path Vectors, where the k-th Path Vector refers to the abstract network paths traversed in the k-th time interval by traffic from the source to the destination. When used with time-varying properties, e.g., maximum reservable bandwidth, a property of a single ANE may also have different values in different time intervals. In this case, if such an ANE has different property values in two time intervals, it MUST be treated as two different ANEs, i.e., with different entity identifiers. However, if it has the same property values in two time intervals, it MAY use the same identifier. This rule allows the Path Vector extension to represent both changes of ANEs and changes of the ANEs' properties in a uniform way. The Path Vector part is calendared in a compatible way, and the property map part is not affected by the Cost Calendar extension. The two extensions combined together can provide the historical network correlation information for a set of source and destination pairs. A network broker or client may use this information to derive other resource requirements such as Time-Block-Maximum Bandwidth, Bandwidth-Sliding-Window, and Time-Bandwidth-Product (TBP) (see for details).

General Discussion

Constraint Tests for General Cost Types The constraint test is a simple approach for querying the data. It allows users to filter query results by specifying some boolean tests. This approach is already used in the ALTO Protocol. ALTO clients are permitted to specify either the "constraints" test or the "or-constraints" test to better filter the results. However, the current syntax can only be used to test scalar cost types and cannot easily express constraints on complex cost types, e.g., the Path Vector cost type defined in this document. In practice, developing a bespoke language for general-purpose boolean tests can be a complex undertaking, and it is conceivable that such implementations already exist (the authors have not done an exhaustive search to determine whether such implementations exist). One avenue for developing such a language may be to explore extending current query languages like XQuery or JSONiq and integrating these with ALTO. Filtering the Path Vector results or developing a more sophisticated filtering mechanism is beyond the scope of this document.

General Multi-Resource Query Querying multiple ALTO information resources continuously is a general requirement. Enabling such a capability, however, must address general issues like efficiency and consistency. The incremental update extension supports submitting multiple queries in a single request and allows flexible control over the queries. However, it does not cover the case introduced in this document where multiple resources are needed for a single request. The extension specified in this document gives an example of using a multipart message to encode the responses from two specific ALTO information resources: a filtered cost map or an Endpoint Cost Service, and a property map. By packing multiple resources in a single response, the implication is that servers may proactively push related information resources to clients. Thus, it is worth looking into extending the SSE mechanism as used in the incremental update extension ; or upgrading to HTTP/2 and HTTP/3 , which provides the ability to multiplex queries and to allow servers to proactively send related information resources. Defining a general multi-resource query mechanism is out of scope for this document.

Security Considerations This document is an extension of the base ALTO Protocol, so the security considerations provided for the base ALTO Protocol fully apply when this extension is provided by an ALTO server. The Path Vector extension requires additional scrutiny of three security considerations discussed in the base protocol: confidentiality of ALTO information (), potential undesirable guidance from authenticated ALTO information (), and availability of ALTO services (). For confidentiality of ALTO information, a network operator should be aware that this extension may introduce a new risk: the Path Vector information, when used together with sensitive ANE properties such as capacities of bottleneck links, may make network attacks easier. For example, as the Path Vector information may reveal more fine-grained internal network structures than the base protocol, an attacker may identify the bottleneck link or links and start a distributed denial-of-service (DDoS) attack involving minimal flows, triggering in-network congestion. Given the potential risk of leaking sensitive information, the Path Vector extension is mainly applicable in scenarios where 1) the ANE structures and ANE properties do not impose security risks on the ALTO service provider (e.g., they do not carry sensitive information) or 2) the ALTO server and client have established a reliable trust relationship (e.g., they operate in the same administrative domain or are managed by business partners with legal contracts). Three risk types are identified in :

excess disclosure of the ALTO service provider's data to an unauthorized ALTO client,
disclosure of the ALTO service provider's data (e.g., network topology information or endpoint addresses) to an unauthorized third party, and
excess retrieval of the ALTO service provider's data by collaborating ALTO clients.

To mitigate these risks, an ALTO server MUST follow the guidelines in . Furthermore, an ALTO server MUST follow the following additional protections strategies for risk types (1) and (3). For risk type (1), an ALTO server MUST use the authentication methods specified in to authenticate the identity of an ALTO client and apply access control techniques to restrict the retrieval of sensitive Path Vector information by unprivileged ALTO clients. For settings where the ALTO server and client are not in the same trust domain, the ALTO server should reach agreements with the ALTO client regarding protection of confidentiality before granting access to Path Vector services with sensitive information. Such agreements may include legal contracts or Digital Rights Management (DRM) techniques. Otherwise, the ALTO server MUST NOT offer Path Vector services that carry sensitive information to the clients, unless the potential risks are fully assessed and mitigated. For risk type (3), an ALTO service provider must be aware that persistent ANEs may be used as "landmarks" in collaborative inferences. Thus, they should only be used when exposing public service access points (e.g., API gateways, CDN Interconnections) and/or when the granularity is coarse grained (e.g., when an ANE represents an AS, a data center, or a WAN). Otherwise, an ALTO server MUST use dynamic mappings from ephemeral ANE names to underlying physical entities. Specifically, for the same physical entity, an ALTO server SHOULD assign a different ephemeral ANE name when the entity appears in the responses to different clients or even for different requests from the same client. A RECOMMENDED assignment strategy is to generate ANE names from random numbers. Further, to protect the network topology from graph reconstruction (e.g., through isomorphic graph identification ), the ALTO server SHOULD consider protection mechanisms to reduce information exposure or obfuscate the real information. When doing so, the ALTO server must be aware that information reduction/obfuscation may lead to a potential risk of undesirable guidance from authenticated ALTO information (). Thus, implementations of ALTO servers involving reduction or obfuscation of the Path Vector information SHOULD consider reduction/obfuscation mechanisms that can preserve the integrity of ALTO information -- for example, by using minimal feasible region compression algorithms or obfuscation protocols . However, these obfuscation methods are experimental, and their practical applicability to the generic capability provided by this extension has not been fully assessed. The ALTO server MUST carefully verify that the deployment scenario satisfies the security assumptions of these methods before applying them to protect Path Vector services with sensitive network information. For availability of ALTO services, an ALTO server should be cognizant that using a Path Vector extension might introduce a new risk: frequent requests for Path Vectors might consume intolerable amounts of server-side computation and storage. This behavior can break the ALTO server. For example, if an ALTO server implementation dynamically computes the Path Vectors for each request, the service that provides the Path Vectors may become an entry point for denial-of-service attacks on the availability of an ALTO server. To mitigate this risk, an ALTO server may consider using such optimizations as precomputation-and-projection mechanisms to reduce the overhead for processing each query. An ALTO server may also protect itself from malicious clients by monitoring client behavior and stopping service to clients that exhibit suspicious behavior (e.g., sending requests at a high frequency). The ALTO service providers must be aware that providing incremental updates of "max-reservable-bandwidth" may provide information about other consumers of the network. For example, a change in value may indicate that one or more reservations have been made or changed. To mitigate this risk, an ALTO server can batch the updates and/or add a random delay before publishing the updates.

IANA Considerations

"ALTO Cost Metrics" Registry This document registers a new entry in the "ALTO Cost Metrics" registry, per . The new entry is as shown below in . "ALTO Cost Metrics" Registry

Identifier	Intended Semantics	Reference
ane-path	See	RFC 9275

"ALTO Cost Modes" Registry This document registers a new entry in the "ALTO Cost Modes" registry, per . The new entry is as shown below in . "ALTO Cost Modes" Registry

Identifier	Description	Intended Semantics	Reference
array	Indicates that the cost value is a JSON array	See	RFC 9275

"ALTO Entity Domain Types" Registry This document registers a new entry in the "ALTO Entity Domain Types" registry, per . The new entry is as shown below in . "ALTO Entity Domain Types" Registry

Identifier	Entity Identifier Encoding	Hierarchy and Inheritance	Media Type of Defining Resource	Mapping to ALTO Address Type
ane	See	None	application/alto-propmap+json	false

Identifier:: See .
Entity Identifier Encoding:: See .
Hierarchy:: None
Inheritance:: None
Media Type of Defining Resource:: See .
Mapping to ALTO Address Type:: This entity type does not map to an ALTO address type.
Security Considerations:: In some usage scenarios, ANE addresses carried in ALTO Protocol messages may reveal information about an ALTO client or an ALTO service provider. If a naming schema is used to generate ANE names, either used privately or standardized by a future extension, how (or if) the naming schema relates to private information and network proximity must be explained to ALTO implementers and service providers.

"ALTO Entity Property Types" Registry Two initial entries -- "max-reservable-bandwidth" and "persistent-entity-id" -- are registered for the ALTO domain "ane" in the "ALTO Entity Property Types" registry, per . The two new entries are shown below in , and their details can be found in Sections and of this document. Initial Entries for the "ane" Domain in the "ALTO Entity Property Types" Registry

Identifier	Intended Semantics	Media Type of Defining Resource
max-reservable-bandwidth	See	application/alto-propmap+json
persistent-entity-id	See	application/alto-propmap+json

New ANE Property Type: Maximum Reservable Bandwidth

Identifier:: "max-reservable-bandwidth"
Intended Semantics:: See .
Media Type of Defining Resource:: application/alto-propmap+json
Security Considerations:: To make better choices regarding bandwidth reservation, this property is essential for applications such as large-scale data transfers or an overlay network interconnection. It may reveal the bandwidth usage of the underlying network and can potentially be leveraged to reduce the cost of conducting denial-of-service attacks. Thus, the ALTO server MUST consider such protection mechanisms as providing the information to authorized clients only and applying information reduction and obfuscation as discussed in .

New ANE Property Type: Persistent Entity ID

Identifier:: "persistent-entity-id"
Intended Semantics:: See .
Media Type of Defining Resource:: application/alto-propmap+json
Security Considerations:: This property is useful when an ALTO server wants to selectively expose certain service points whose detailed properties can be further queried by applications. As mentioned in , the entity IDs may reveal sensitive information about the underlying network. An ALTO server should follow the security considerations provided in .

References Normative References Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types This second document defines the general structure of the MIME media typing system and defines an initial set of media types. [STANDARDS-TRACK] Key words for use in RFCs to Indicate Requirement Levels In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. The MIME Multipart/Related Content-type This document defines the Multipart/Related content-type and provides examples of its use. [STANDARDS-TRACK] Internet Message Format This document specifies the Internet Message Format (IMF), a syntax for text messages that are sent between computer users, within the framework of "electronic mail" messages. This specification is a revision of Request For Comments (RFC) 2822, which itself superseded Request For Comments (RFC) 822, "Standard for the Format of ARPA Internet Text Messages", updating it to reflect current practice and incorporating incremental changes that were specified in other RFCs. [STANDARDS-TRACK] Application-Layer Traffic Optimization (ALTO) Protocol Applications using the Internet already have access to some topology information of Internet Service Provider (ISP) networks. For example, views to Internet routing tables at Looking Glass servers are available and can be practically downloaded to many network application clients. What is missing is knowledge of the underlying network topologies from the point of view of ISPs. In other words, what an ISP prefers in terms of traffic optimization -- and a way to distribute it. The Application-Layer Traffic Optimization (ALTO) services defined in this document provide network information (e.g., basic network location structure and preferences of network paths) with the goal of modifying network resource consumption patterns while maintaining or improving application performance. The basic information of ALTO is based on abstract maps of a network. These maps provide a simplified view, yet enough information about a network for applications to effectively utilize them. Additional services are built on top of the maps. This document describes a protocol implementing the ALTO services. Although the ALTO services would primarily be provided by ISPs, other entities, such as content service providers, could also provide ALTO services. Applications that could use the ALTO services are those that have a choice to which end points to connect. Examples of such applications are peer-to-peer (P2P) and content delivery networks. Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. Multi-Cost Application-Layer Traffic Optimization (ALTO) The Application-Layer Traffic Optimization (ALTO) protocol, specified in RFC 7285, defines several services that return various metrics describing the costs between network endpoints. This document defines a new service that allows an ALTO Client to retrieve several cost metrics in a single request for an ALTO filtered cost map and endpoint cost map. In addition, it extends the constraints to further filter those maps by allowing an ALTO Client to specify a logical combination of tests on several cost metrics. Application-Layer Traffic Optimization (ALTO) Incremental Updates Using Server-Sent Events (SSE) The Application-Layer Traffic Optimization (ALTO) protocol (RFC 7285) provides network-related information, called network information resources, to client applications so that clients can make informed decisions in utilizing network resources. This document presents a mechanism to allow an ALTO server to push updates to ALTO clients to achieve two benefits: (1) updates can be incremental, in that if only a small section of an information resource changes, the ALTO server can send just the changes and (2) updates can be immediate, in that the ALTO server can send updates as soon as they are available. Application-Layer Traffic Optimization (ALTO) Cost Calendar This document is an extension to the base Application-Layer Traffic Optimization (ALTO) protocol. It extends the ALTO cost information service so that applications decide not only 'where' to connect but also 'when'. This is useful for applications that need to perform bulk data transfer and would like to schedule these transfers during an off-peak hour, for example. This extension introduces the ALTO Cost Calendar with which an ALTO Server exposes ALTO cost values in JSON arrays where each value corresponds to a given time interval. The time intervals, as well as other Calendar attributes, are specified in the Information Resources Directory and ALTO Server responses. An Extension for Application-Layer Traffic Optimization (ALTO): Entity Property Maps This document specifies an extension to the base Application-Layer Traffic Optimization (ALTO) Protocol that generalizes the concept of "endpoint properties", which have been tied to IP addresses so far, to entities defined by a wide set of objects. Further, these properties are presented as maps, similar to the network and cost maps in the base ALTO Protocol. While supporting the endpoints and related Endpoint Property Service defined in RFC 7285, the ALTO Protocol is extended in two major directions. First, from endpoints restricted to IP addresses to entities covering a wider and extensible set of objects; second, from properties for specific endpoints to entire entity property maps. These extensions introduce additional features that allow entities and property values to be specific to a given information resource. This is made possible by a generic and flexible design of entity and property types. A Cost Mode Registry for the Application-Layer Traffic Optimization (ALTO) Protocol This document creates a new IANA registry for tracking cost modes supported by the Application-Layer Traffic Optimization (ALTO) Protocol. Also, this document relaxes a constraint that was imposed by the ALTO specification on allowed cost mode values. This document updates RFC 7285. Informative References ALTO Performance Cost Metrics Huawei Yale University Samsung Huawei Nokia Bell Labs Telefonica Work in Progress Graph reconstruction--a survey University of Waterloo Vanderbilt University Journal of Graph Theory, Volume 1, Issue 3, pp. 227-268 Optimizing in the Dark: Learning an Optimal Solution through a Simple Request Interface Yale University Tongji University Yale University IBM T.J. Watson Research Center Shanghai Jiao Tong University Yale University Proceedings of the AAAI Conference on Artificial Intelligence 33, 1674-1681 CLARINET: WAN-aware optimization for analytics queries University of Wisconsin-Madison Microsoft University of Wisconsin-Madison Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation (OSDI'16), Savannah, GA, pp. 435-450 On the Bottleneck Structure of Congestion-Controlled Networks Reservoir Labs Reservoir Labs Reservoir Labs Reservoir Labs Reservoir Labs Yale University Yale University University of Virginia University of Virginia University of Virginia Proceedings of the ACM on Measurement and Analysis of Computing Systems, Volume 3, Issue 3, pp. 1-31 HUG: multi-resource fairness for correlated and elastic demands University of Michigan Stony Brook University UC Berkeley, Databricks Inc. UC Berkeley, Databricks Inc. Proceedings of the 13th USENIX Conference on Networked Systems Design and Implementation (NSDI'16), Santa Clara, CA, pp. 407-424 Intent-Based Networking - Concepts and Definitions Futurewei Rakuten Mobile Federal University of Rio Grande do Sul (UFRGS) Microsoft Work in Progress The JSON Query Language JSONiq Toward Fine-Grained, Privacy-Preserving, Efficient Multi-Domain Network Resource Discovery Yale University Tongji University Tongji University Tongji University ESNet IBM T.J. Watson Research Center ESNet Caltech Yale University IEEE/ACM, IEEE Journal on Selected Areas in Communications, Volume 37, Issue 8, pp. 1924-1940 MoWIE: Toward Systematic, Adaptive Network Information Exposure as an Enabling Technique for Cloud-Based Applications over 5G and Beyond Tencent China Mobile Tencent Tencent Tencent Tencent Nokia Bell Labs Yale University University of Minnesota Proceedings of the Workshop on Network Application Integration/CoDesign (NAI '20), ACM, Virtual Event USA, pp. 20-27 An Objective-Driven On-Demand Network Abstraction for Adaptive Applications Sichuan University Yale University Tongji University Yale University Tsinghua University IEEE/ACM Transactions on Networking (TON) Vol. 27, Issue 2, pp. 805-818 Fine-Grained, Multi-Domain Network Resource Abstraction as a Fundamental Primitive to Enable High-Performance, Collaborative Data Sciences Yale University Tongji University Tongji University Tongji University ESNet IBM T.J. Watson Research Center ESNet Caltech Yale University SC18: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 58-70 Network Element Service Specification Template This document defines a framework for specifying services provided by network elements, and available to applications, in an internetwork which offers multiple qualities of service. The document first provides some necessary context -- including relevant definitions and suggested data formats -- and then specifies a "template" which service specification documents should follow. This memo provides information for the Internet community. It does not specify an Internet standard of any kind. A Border Gateway Protocol 4 (BGP-4) This document discusses the Border Gateway Protocol (BGP), which is an inter-Autonomous System routing protocol. The primary function of a BGP speaking system is to exchange network reachability information with other BGP systems. This network reachability information includes information on the list of Autonomous Systems (ASes) that reachability information traverses. This information is sufficient for constructing a graph of AS connectivity for this reachability from which routing loops may be pruned, and, at the AS level, some policy decisions may be enforced. BGP-4 provides a set of mechanisms for supporting Classless Inter-Domain Routing (CIDR). These mechanisms include support for advertising a set of destinations as an IP prefix, and eliminating the concept of network "class" within BGP. BGP-4 also introduces mechanisms that allow aggregation of routes, including aggregation of AS paths. This document obsoletes RFC 1771. [STANDARDS-TRACK] HTTP/2 This specification describes an optimized expression of the semantics of the Hypertext Transfer Protocol (HTTP), referred to as HTTP version 2 (HTTP/2). HTTP/2 enables a more efficient use of network resources and a reduced latency by introducing field compression and allowing multiple concurrent exchanges on the same connection. This document obsoletes RFCs 7540 and 8740. HTTP/3 The QUIC transport protocol has several features that are desirable in a transport for HTTP, such as stream multiplexing, per-stream flow control, and low-latency connection establishment. This document describes a mapping of HTTP semantics over QUIC. This document also identifies HTTP/2 features that are subsumed by QUIC and describes how HTTP/2 extensions can be ported to HTTP/3. Software Defined Networking (SDN) for End-to-End Networked Science at the Exascale ESnet Computing at the Edge: But, what Edge? Telefonica Telefonica/Universidad de la República National Institute of Information and Communications Technology, Japan Universitat Politcnica de Catalunya Proceedings of NOMS 2020 - 2020 IEEE/IFIP Network Operations and Management Symposium, pp. 1-9 Achieving high utilization with software-driven WAN Microsoft Microsoft Microsoft Microsoft Microsoft Microsoft ETH Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM (SIGCOMM '13), New York, NY, pp. 15-26 Unicorn: Unified resource orchestration for multi-domain, geo-distributed data analytics Yale University Tongji University Tsinghua University California Institute of Technology Yale University Tongji University Future Generation Computer Systems, Volume 93, pp. 188-197 XQuery 3.1: An XML Query Language W3C Recommendation

Acknowledgments The authors would like to thank , , , , , , , , and for fruitful discussions. The authors thank , , , and for their contributions to earlier draft versions of this document. The authors would also like to thank , , , , , , , , , , , , and , whose feedback and suggestions were invaluable for improving the practicability and conciseness of this document; and , , , , and , who provided great support and guidance.

Authors' Addresses Sichuan University