Archive for the ‘IRTF’ Category
Appointed as IRTF Chair
I am delighted that I have been appointed as the next Chair of the Internet Research Task Force (IRTF) by the Internet Architecture Board (IAB).
I have been involved in the IRTF for many years. It is a unique organization that conducts research of importance to the evolution of the Internet protocols, applications, architecture and technology. It has initiated and supported many important technology developments for the Internet in the past, in fields such as network architecture, security and privacy, congestion control, and many more.
The IRTF focuses on longer term research issues, and its various research groups are enabling international collaboration for continuous research on critical topics for the Internet by working with academic and industry research communities.
My term starts in March 2025. I am sincerely grateful for all the support I have received, I am looking forward to working with this community to help making the Internet work better through good research work.
References
IRTF DINRG Meeting at IETF-121
The IRTF DINRG Meeting at IETF-121 takes place on 2024-11-06 at 13:00 to 14:30 UTC.
1 | DINRG Chairs’ Presentation: Status, Updates | Chairs | 05 min |
2 | Distributing DDoS Analytics among ASes | Daniel Wagner | 20 min |
3 | The Role of DNS names in Internet Decentralization | Tianyuan Yu | 20 min |
4 | Taxonomy of Internet Consolidation & Effects of Internet Consolidation | Marc McFadden | 15 min |
5 | DINRG – Next Steps | Chairs & Panelists | 30 min |
6 | Wrap-up & Buffer | Chairs | 00 min |
Documents and Links to Resources
- United We Stand: Collaborative Detection and Mitigation of
Amplification DDoS Attacks at
Scale - https://datatracker.ietf.org/doc/draft-mcfadden-consolidation-taxonomy/
- https://datatracker.ietf.org/doc/draft-mcfadden-cnsldtn-effects/
Notes
Please remember that all sessions are being recorded.
IRTF ICNRG Meeting at IETF-121
The ICNRG Meeting at IETF-121 takes place on 2024-11-05, 13:00 to 14:30 UTC.
ICNRG Agenda
1 | ICNRG Chairs’ Presentation: Status, Updates | Chairs | 05 min |
2 | FLIC Update | Marc Mosko | 15 min |
3 | CCNx Content Object Chunking | Marc Mosko | 15 min |
4 | Reflexive Forwarding Update | Hitoshi Asaeda | 20 min |
5 | ICN Challenges for Metaverse Platform Interoperability | Jungha Hong | 15 min |
6 | Distributed Micro Service Communication | Aijun Wang | 15 min |
7 | Buffer, Wrap Up and Next Steps | Chairs | 05 min |
Please remember that all sessions are being recorded.
Material
- https://datatracker.ietf.org/doc/draft-irtf-icnrg-flic/
- https://datatracker.ietf.org/doc/draft-mosko-icnrg-ccnxchunking/
- https://github.com/mmosko/ccnpy
- https://datatracker.ietf.org/doc/draft-irtf-icnrg-reflexive-forwarding/
- https://datatracker.ietf.org/doc/draft-hong-icn-metaverse-interoperability/
- https://datatracker.ietf.org/doc/draft-li-icnrg-damc/
New Internet Draft draft-irtf-icnrg-reflexive-forwarding-00
We updated our Internet Draft draft-irtf-icnrg-reflexive-forwarding-00 on Reflexive Forwarding for CCNx and NDN Protocols.
Current Information-Centric Networking protocols such as CCNx and NDN have a wide range of useful applications in content retrieval and other scenarios that depend only on a robust two-way exchange in the form of a request and response (represented by an Interest-Data exchange in the case of the two protocols noted above). A number of important applications however, require placing large amounts of data in the Interest message, and/or more than one two-way handshake. While these can be accomplished using independent Interest-Data exchanges by reversing the roles of consumer and producer, such approaches can be both clumsy for applications and problematic from a state management, congestion control, or security standpoint. This specification proposes a Reflexive Forwarding extension to the CCNx and NDN protocol architectures that eliminates the problems inherent in using independent Interest-Data exchanges for such applications. It updates RFC8569 and RFC8609.
The recent update includes a generalization of the main protocol specification, so that Reflexive Forwarding can be used in both CCNx and NDN.
IRTF DINRG at IETF-120
We have an exciting agenda for our upcoming IRTF DINRG meeting (Wednesday, July 24th, 2024 at 09:30 in Vancouver) at IETF-120. If you do not attend the IETF-120 meeting locally, please consider attending online.
1 | DINRG Chairs’ Presentation: Status, Updates | Chairs | 05 min |
2 | Exploring Decentralized Digital Identity Protocols | Kaliya Young | 20 min |
3 | DNS-Bound Client and Sender Identities | Michael Richardson | 20 min |
4 | Internet Fragmentation | Sheetal Kumar | 20 min |
5 | SOLID: Your Data, Your Choice | Hadrian Zbarcea | 20 min |
6 | Panel discussion: Internet Decentralization – Next Steps | Chairs & Panelists | 30 min |
7 | Wrap-up & Buffer | Chairs | 05 min |
Documents and Links to Resources
- Policy Network on Internet
Fragmentation - https://datatracker.ietf.org/doc/draft-ietf-dance-architecture/06/
- https://datatracker.ietf.org/doc/rfc9518/
- SOLID Project
Panel Description
Internet Decentralization – Next Steps
The previous DINRG meetings all had lively open mic discussions. However we noticed that those spontaneous conversations, while being interesting and insightful, tend to head to different issues in diverse directions. At this meeting we will continue/extend the previous discussions by gathering a small group of panelists and start the discussion with a list of questions collected from the previous meetings. We will have an open mic for all audience and share the list of discussion questions on DINRG list before the meeting; by gathering a panel and preparing a list of questions, we hope to make the discussions more effective and fruitful, moving towards our overarching goal of identifying an ordered list of issues that DINRG aims to address in coming years.
Links
IRTF ICNRG@IETF-119
The Information-Centric Networking Research Group (ICNRG) of the Internet Research Task Force (IRTF) met at IETF-119 in Brisbane. Here is my quick summary of the meeting:
Agenda:
1 | ICNRG Chairs’ Presentation: Status, Updates | Chairs |
2 | Secure Web Objects and Transactions | Dirk Kutscher |
3 | Transaction Manifests | Marc Mosko |
4 | Vanadium: Secure, Distributed Applications | Marc Mosko |
5 | Global vs. Scoped Namespaces | Marc Mosko |
Meeting material:
ICNRG Status
ICNRG recently published four news RFCs – great achievement by all involved authors and the whole group!
- RFC 9510: Alternative Delta Time Encoding for Content-Centric Networking (CCNx) Using Compact Floating-Point Arithmetic
- RFC 9531: Path Steering in Content-Centric Networking (CCNx) and Named Data Networking (NDN)
- RFC 9508: Information-Centric Networking (ICN) Ping Protocol Specification
- RFC 9507: Information-Centric Networking (ICN) Traceroute Protocol Specification
See my blog posting for a more detailed description.
Secure Web Objects and Transactions
One focus of this meeting was transactions in ICN, i.e., interactions with the intention to achieve some durable state change at a remote peer – which imposes some challenges in a system that is designed around accessing named data.
In my presentation I talked about different ways to realize transactions in ICN:
- ICN as a network layer
- Client-server communication between two nodes
- Implement transaction semantics on top of an ICN messaging service
- Recording state changes in shared data structures
- Shared namespace, potentially functioning as a transaction ledger
- Still need to think about atomicity etc
For 1) transactions as messaging over ICN networks, the following considerations apply:
- Client-server communication between two nodes
- Implement transaction semantics on top of an ICN messaging service
- Different approaches
- A: Traditional layering: Using NDN-like systems as a messaging layer
- Assign prefixes to client & servers
- Send messages back and forth, and implement reliability and transactions semantics on top
- B: ICN-native communication: Use Interest-Data as request-response abstraction for transactions
- Mapping transaction communication and state evolution more directly to ICN, e.g., Interest-Data in NDN
- Collapsing traditional network, transport, application layer functions
I mainly talked about variant 1B, ICN-native communication: Use InterestData as request-response abstraction for transactions and introduced the idea of "Secure Web Objects" (SWOs) for a data-oriened web as a motivation.
In such a system, not everything would be about accessing named data object – there is also a need for "client/server" state evolution, e.g., for online banking and similar use cases.
I introduced some ideas on RESTful ICN that we published in an earlier paper. The Restful ICN proposal leverages Reflexive Forwarding, for robust client-server communication and integrates elements of CCNx key exchange for security context setup and session resumption.
Summarizing, I wanted to initiate a discussion about how to realize transactions in information-centric systems? This discussion is not about mapping ICN to existing protocols, such as HTTP, but about actual distributed computing semantics, i.e., robust session setup and state evolution. Transactions with ICN-native communication are hard to provide with with basic Interest/Data. Reflexive Forwarding + CCNx Key Exchange + transaction semantics are an attempt to provide such a service in a mostly ICN-idiomatic way, with the downside that reflexive forwarding needs extensions to forwarders. This raises question on the minimal feature set of core ICN protocols, and to deal with extensions.
In the discussion, it was pointed out that lots of experience on distributed systems has shown that transactions or secure multi-interactions will generally require more than a single two-way exchange.
Others suggested that ICN and NDN has authentication carried out when the signed interest arrives which directly proves authentication, so that the authentication would in fact be done beforehand.
However, authentication may not be enough. For example, client authorization in client-server communication is a critical function which needs to be carefully designed in real-world networks. For example, forcing a server to do signature verification on initial request arrival has been shown in prior systems (e.g. TCP+TLS) to represent a serious computational DOS attack risk. Reflexive Forwarding in RICE tries to avoid exactly that problem, by enabling the server to iteratively authenticate and authorize clients before committing computing resources.
It was also said that whenever a protocol does authentication. you need to analyze in the context of specific examples to discuss, and that cannot only look at the problem at an abstract level.
Transaction Manifests
Marc Mosko presented another approach to transactions in ICN, called [Transaction Manifests](https://datatracker.ietf.org/meeting/119/materials/slides-119-icnrg-transaction-manifests-00 "Transaction Manifests "Transaction Manifests"). He explained that ICN can be transactional.
Typically, ICN is considered as a publish/subscribe or pre-publishing of named-data approach. Outside ICN, distributed transactions do exist, especially in DLTs. For example, considering a permissioned DLT with size N and K << N bookkeepers. In a DLT, they base their decision on the block hash history. In this talk, Marc discussed what would be an equivalent function in ICN, and introduced the notion of transaction manifests.
In ICN, there is a technology called FLIC (File-like collections), i.e., manifests for static objects. FLIC describes a single object that is re-constructed by traversing the manifest in order. In Marc's proposal, a transaction manifest describes a set of names that must be considered together. The transaction manifest names likely point to FLIC root manifests.
In the example above, transaction manifest entries entries point directly to objects. For a complete systems, you would also need a set of bookkeepers, e.g., systems like Hyperledger offering global ordering vis bespoke orderer nodes. Such bookkeeper would have to ensure that a transaction has current pre-conditions, current post-conditions, and no conflicts in post-conditions. Transaction manifests are a form of write-ahead logs (WAL), as used in databases, such as PostgreSQL.
Marc went on discussing a few challenges, such as interactions with repositories and caches, as well as distributed transaction manifests.
There was some discussion on the required ordering properties for this approach, i.e., whether, in a multi-bookkeeper system, livelocks and deadlocks could occur – and whether these could resolved without requiring a total order.
Marc is continueing to work on this. One of the next steps would be to design client-to-bookkepper and bookkeeper-to-bookkeeper protocols.
Vanadium: Secure, Distributed Applications
Marc Mosko introduced the Vanadium system, a secure, distributed RPC system based on distributed naming and discovery. Vanadium uses symmetrical authentication and encryption and may use private name discovery with Identity-Based-Encryption (IBE).
Vanadium has two parts:
- Principals and Blessings and Caveats (Security)
- Use a hierarchical name, e.g. alice:home:tv.
- Certificate based
- Blessings are scoped delegations from one principal to another for a namespace (e.g. alice grants Bob “watch” permissions to the TV)
- Caveats are restrictions on delegations (e.g. Bob can only watch 6pm – 9pm).
- 3rd party caveats must be discharged before authorization
- E.g. revocations or auditing
- The RPC mount tables (Object Naming)
- These describe how to locate RPC namespaces
- They provide relative naming
Vanadium is interesting because parts of its design resemble some ICN concepts, especially the security part:
- It uses prefix matching and encryption
- Namespaces work like groups
- The colon : separates the blesser from the blessed
- Authorizations match extensions.
- If Alice authorized “read” to alice:hometv to alice:houseguests, and if Bob has a blessing for alice:houseguests:bob, then Bob has “read” to alice:hometv.
- A special terminator :$ only matches the exact prefix.
- A blessing to alice:houseguest:$ only matches that exact prefix.
Marc then explain the object naming structure and the entity resolution in Vanadium.
More details can be found in Marc's presentation and on Vanadium's web page.
In summary, Vanadium is a permissioned RPC service. A Vanadium name encodes the endpoint plus name suffix. The endpoint does not need to resolve to a single mount table server, it could be any server that possesses an appropriate blessing. Authentication is done via pair-wise key exchange and blessing validations. It can be private if using IBE, otherwise server name leaks. Authorizations and Blessings and Caveats use hierarchical, prefixmatching names.
From an ICN perspective, the security approach seems interesting. Blessings and Caveats and discharges and namespaces as groups. One question is how this differs from SDSI co-signings. The Vanadium identity service provides an interesting mapping of OAuth2 app:email tokens to PKI and blessings. The RPC approach exhibits some differences to ICN, e.g., embedding the endpoint identifier in the name. ICN technologies in this context are public-key scoped names in CCNx and schematized trust anchors in NDN.
In the discusion, it was noted that it would be interesting to do an apples-to-apples comparison to the NDN trust schema approach; Vanadium's approach with the ability to create blessings and caveats on demand seems to be much more granular and dynamic.
Global vs. Scoped Namespaces
Marc Mosko discussed global vs. scoped namespaces. For example, how do you know that the key you are looking at is the key that you should be looking at? IPFS punts that to out-of-band mechanisms. CCNX on the other hand uses public key scoped names; you can put a public key, publisher ID in an interest and say you only wanyt this name if signed with the associated key.
It was suggested to re-visit some of the concepts in the RPC system of OSF distributed computing, where all namespaces were scoped, and name discovery starts out as local. You could then "attach" a local namespace to more global namespace via an explicit "graft" operation. The key here was that the authoritative pointers representing the namespace graph were from child to parent, as opposed to parent to child as it is with systems like DNS. Your local trust root identifier could become a name in a higher layer space, yielding a trust root higher in the hierarchy tha could be used instead of or in addition to your local trust root. Doing this can create progressively more global name spaces out of local ones.
Please check out the meeting video for the complete discussion at the meeting.
Information-Centric Networking RFCs
In the Information-Centric Networking Research Group (ICNRG) of the Internet Research Task Force (IRTF) we have recently published a set of new RFCs:
RFC 9510: Alternative Delta Time Encoding for Content-Centric Networking (CCNx) Using Compact Floating-Point Arithmetic
Content-Centric Networking (CCNx) utilizes delta time for a number of functions. When using CCNx in environments with constrained nodes or bandwidth-constrained networks, it is valuable to have a compressed representation of delta time. In order to do so, either accuracy or dynamic range has to be sacrificed. Since the current uses of delta time do not require both simultaneously, one can consider a logarithmic encoding. This document updates RFC 8609 ( to specify this alternative encoding.
RFC 9531: Path Steering in Content-Centric Networking (CCNx) and Named Data Networking (NDN)
Path steering is a mechanism to discover paths to the producers of Information-Centric Networking (ICN) Content Objects and steer subsequent Interest messages along a previously discovered path. It has various uses, including the operation of state-of-the-art multi-path congestion control algorithms and for network measurement and management. This specification derives directly from the design published in https://dl.acm.org/doi/10.1145/3125719.3125721 (4th ACM Conference on Information-Centric Networking) and, therefore, does not recapitulate the design motivations, implementation details, or evaluation of the scheme. However, some technical details are different, and where there are differences, the design documented here is to be considered definitive.
RFC 9508: Information-Centric Networking (ICN) Ping Protocol Specification
This document presents the design of an Information-Centric Networking (ICN) Ping protocol. It includes the operations of both the client and the forwarder.
Ascertaining data plane reachability to a destination and taking coarse performance measurements of Round-Trip Time (RTT) are fundamental facilities for network administration and troubleshooting. In IP, where routing and forwarding are based on IP addresses, ICMP Echo Request and ICMP Echo Reply packets are the protocol mechanisms used for this purpose, generally exercised through the familiar ping utility. In Information-Centric Networking (ICN), where routing and forwarding are based on name prefixes, the ability to ascertain the reachability of names is required.
In order to carry out meaningful experimentation and deployment of ICN protocols, new tools analogous to ping and traceroute used for TCP/IP are needed to manage and debug the operation of ICN architectures and protocols. This document describes the design of a management and debugging protocol analogous to the ping protocol of TCP/IP; this new management and debugging protocol will aid the experimental deployment of ICN protocols. As the community continues its experimentation with ICN architectures and protocols, the design of ICN Ping might change accordingly. ICN Ping is designed as a "first line of defense" tool to troubleshoot ICN architectures and protocols. As such, this document is classified as an Experimental RFC. Note that a measurement application is needed to make proper use of ICN Ping in order to compute various statistics, such as average, maximum, and minimum Round-Trip Time (RTT) values, variance in RTTs, and loss rates.
RFC 9507: Information-Centric Networking (ICN) Traceroute Protocol Specification
This document presents the design of an Information-Centric Networking (ICN) Traceroute protocol. This includes the operation of both the client and the forwarder.
In TCP/IP, routing and forwarding are based on IP addresses. To ascertain the route to an IP address and to measure the transit delays, the traceroute utility is commonly used. In Information-Centric Networking (ICN), routing and forwarding are based on name prefixes. To this end, the ability to ascertain the characteristics of at least one of the available routes to a name prefix is a fundamental requirement for instrumentation and network management. These characteristics include, among others, route properties such as which forwarders were transited and the delay incurred through forwarding.
In order to carry out meaningful experimentation and deployment of ICN protocols, new tools analogous to ping and traceroute used for TCP/IP are needed to manage and debug the operation of ICN architectures and protocols. This document describes the design of a management and debugging protocol analogous to the traceroute protocol of TCP/IP; this new management and debugging protocol will aid the experimental deployment of ICN protocols. As the community continues its experimentation with ICN architectures and protocols, the design of ICN Traceroute might change accordingly. ICN Traceroute is designed as a tool to troubleshoot ICN architectures and protocols.
HKUST Internet Research Workshop 2024
On March 15 2024, in the week before the IETF-119 meeting in Brisbane, Zili Meng and I organized the 1st HKUST Internet Research Workshop that brought together researchers in computer networking and systems around the globe to a live forum discussing innovative ideas at their early stages. The workshop took place at HKUST's Clear Water Bay campus in Hong Hong.
We ran the workshop like a “one day Dagstuhl seminar” and focused on discussion and ideas exchange and less on conference-style presentations. The objective was to identify topics and connect like-minded people for potential future collaboration, which worked out really well.
The agenda was:
- Dirk Kutscher: Networking for Distributed ML
- Zili Meng: Overview of the Low-Latency Video Delivery Pipeline
- Jianfei He: The philosophy behind computer networking
- Carsten Bormann: Towards a device-infrastructure continuum in IoT and OT networks
- Zili Meng: Network Research – Academia, Industry, or Both?
Dirk Kutscher: Networking for Distributed ML
With the ever-increasing demand for compute power from large-scale machine learning training we have started to realize that not only does Moore's Law no longer address increasing performance demand automatically, but also that the growth rate in terms of training FLOPs for transformers and other large-scale machine learning exhibits by far larger exponential factors.
This has been well illustrated by presentations in an AI data center side meeting at IETF-118, for example by Omer Shabtai who talked about Distributed Training in data centers.
WIth increasing scale, communication over networks becomes a bottleneck, and the question arises, what could be good system designs, protocols, and in-network support strategies to improve performance.
Current distributed machine learning systems typically use a technology called Collective Communication that was developed as a Message Passing Interface (MPI) abstraction for high-performance computing (HPC). Collective Communication is the combination of standardized aggregration and reduction function with communication abstractions, e.g., for "broadcasting" or "unicasting" results.
Collective Communication is implemented a few popular libraries such as OpenMPI and Nvidia's NCCL. When used in IP networks, the communication is usually mapped to iterations of peer-to-peer interactions, e.g., organizing nodes in a ring and sending data for aggregation within such rings. One potential way to achieve better performance would be to perform the aggregation "in the network", as in HPC systems, e.g., using the Scalable hierarchical aggregation protocol (SHArP). Previous work has attempted doing this with P4-based dataplane programming, however such approaches are typically limited due to the mostly stateless operation of the corresponding network elements.
In large-scale training sessions, running over shared infrastructure in multi-tenant data centers, communication needs to respond to congestion, packet loss, server overload etc., i.e., the features of typical transport protocols are needed.
I had previously discussed corresponding challenges and requirements in these Internet Drafts:
- Collective Communication Optimization
- Towards a Unified Transport Protocol for In-Network Computing in Support of RPC-based Applications
In my talk at HKIRW, I discussed ideas for corresponding transport protocols. There are interesting challenges in bringing together reliable communication, congestion control, flow control, single-destination as well multi-destination communication and in-network processing.
Zili Meng: Overview of the Low-Latency Video Delivery Pipeline
Zili talked about requirements for ultra-low latency for interactive streaming for the next-generation of immersive applications. Some application provide really stringent low-latency requirements, with a consistent service quality over many hours, and the talk suggested a better coordination between all elements of the streaming and rendering pipeline.
There was a discussion as to how achievable these requirements are in the Internet and whether applications might be re-designed in terms of providing acceptable user experience even without guaranteed high-bandwidth low-latency service, for example by employing technologies such as semantic communication, prediction, local control loops etc.
Jianfei He: The philosophy behind computer networking
In his talk, Jianfe He asked the question how the field of computer networked can be more precisely defined and how a more systematic could help with the understanding and design of future networked systems.
Specifically, he suggested considering basing design on a solid understanding of potentials and absolute constraints in a certain field, such as Shannon's theory/limit and on the notion of tradeoffs, i.e., consequences of certain design decisions, as represented by the CAP theorem in distributed systems. He mentioned two examples: 1) routing protocols and 2) transport protocols.
For routing protocols, there are well-known tradeoffs between convergence time, scaling limits, and required bandwidths. With changed network properties (bandwidth) – can we reasons about options for shifting the tradeoffs?
For transport protocols, there a goals such as reliability, congestion control etc., and tradeoff relationships between packet loss, line utilization, delay and buffer size. How would designs change if we changed the objective, e.g., to shortest flow completion times or shortest message completion time (or if we looked at collections of flows)? What if we added fairness to these objectives?
Jianfe asked the question whether it was possible to develop these tradeoffs/constraints into a more consistent theory.
Carsten Bormann: Towards a device-infrastructure continuum in IoT and OT networks
Carsten talked about requirements and available technologies for providing a secure management of IoT devices in a device-infrastructure continuum in IoT and OT networks, where scale demands high degrees of automation at run-time and only limited individual device configuration (at installation only). It is no longer possible to manually track each new "Thing" species.
Carsten mentioned technologies such as
- RFC 8250: Manufacturer's Usage Description (MUD);
- W3C Web of Things description model; and
- IETF Semantic Definition Format (SDF).
In his talk, Carsten formulated the goal of "Well-Informed Networking", i.e., an approach where networks can obtain sufficient information about the existing devices, their legitimate communication requirements, and their current status (device health).
Zili Meng: Network Research – Academia, Industry, or Both?
Zili discussed the significance of consistently high numbers industry and industry-only papers at major networking conferences. Often such papers are based on operational experience that can only obtained by companies actually operating corresponding systems.
Sometimes papers seem to get accepted not necessarily on the basis of their technical merits but because they report on "large-scale deployments".
When academics get involved in such work, it is often not in a driving position, but rather through students who work in internship at corresponding companies. Naturally, such papers are not questioning the status quo and are generally not critical of the systems they discuss.
At the workshop, we discussed the changes in the networking research field over the past years, as well as the challenges of successful collaborations between academia and industry.
Towards a Unified Transport Protocol for In-Network Computing in Support of RPC-based Applications
The emerging term In-Network Computin (INC) [inc] in particular refers applying on-path programmable networking devices (e.g., switches and routers between clients and servers) as an accelerator or function offloader to boost throughput, reduce server load, or improve latency, typically in a well-controlled data center network environment.
Some INC implementations evolved from programmable data plane systems and align with the trend of network programmability at large. In recent year, it has been shown to support many promising applications (e.g., caching, aggregation, and agreement). For example, in distributed machine learning (DML), training nodes produce data (gradients) that needs to be aggregated or reduced -- and the result could be distributed to one or multiple consumers. As another example, the NetClone system [netclone] uses in-network forwarder to replicate RPC invocation messages and to perform more informed forwarding based on observed latencies for accelerating RPC communication.
While it is possible to achieve this kind of operation purely with end-to-end communication between worker nodes, performance can be dramatically improved by offloading both the operation processing and the data dissemination to nodes in the network. These in-network processors are often conceived as semi-transparent performance enhancing on-path elements, i.e., they are not the actual endpoints in transport protocol sessions and would intercept packets with application data and potentially generate new data that they would have to transmit.
In our Internet Draft draft-song-inc-transport-protocol-req-01.txt, we are discussing this problem and are formulating some requirements for the design of future transport protocols in this space.
References
- Collective Communication: Better Network Abstractions for AI
- Computing in the Network – Lessons Learned and New Opportunities
- [I-D.yao-tsvwg-cco-problem-statement-and-usecases] Yao, K., Shiping, X., Li, Y., Huang, H., and D. KUTSCHER, "Collective Communication Optimization: Problem Statement and Use cases", Work in Progress, Internet-Draft, draft-yao-tsvwg-cco-problem-statement-and-usecases-00, 23 October 2023, https://datatracker.ietf.org/doc/html/draft-yao-tsvwg-cco-problem-statement-and-usecases-00.
- [inc] Klenk et al., B., "An In-Network Architecture for Accelerating Shared-Memory Multiprocessor Collectives", ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA), 2020, <https:dx.doi.org/10.1109/ISCA45697.2020.00085>
- [netclone] Kim, G., "NetClone: Fast, Scalable, and Dynamic Request Cloning for Microsecond-Scale RPCs", In Proceedings of the ACM SIGCOMM 2023 Conference (ACM SIGCOMM '23). Association for Computing Machinery, New York, NY, USA, 195-207, 2023 https://dl.acm.org/doi/10.1145/3603269.3604820
DINRG @ IETF-118
We have posted the agenda for our DINRG meeting at IETF-118:
Documents
- The Cloud Strikes Back: Investigating the Decentralization of IPFS
- Local-First Software
- draft-mcfadden-consolidation-taxonomy-03
Logistics
DINRG Meeting at IETF-118 – 2023-11-06, 08:30 to 10:30 UTC