Dirk Kutscher

Personal web page

Rethinking LoRa for the IoT: An Information-centric Approach

without comments

We just published our IEEE Communications Magazine article on Rethinking LoRa for the IoT with an Open Access license.

LoraWAN and the Internet

Internet of Things (IoT) interconnects numerous sensors and actuators either locally or across the global Internet. From an application perspective, IoT systems are inherently data-oriented, their purpose is often to provide access to named sensor data and control interfaces. From a device and communication perspective, things in the IoT are resource-constrained devices that are commonly powered by a small battery and communicate wirelessly.

LoRaWAN systems today integrate the LoRa physical layer with the LoRaWAN MAC layer and corresponding infrastructure support. Among the IoT radio technologies, LoRa is a versatile and popular candidate since it provides a physical layer that allows for data transmission over multiple kilometers with minimal energy consumption. At the same time, the high LoRa receiver sensitivity enables packet reception in noisy environments, which makes it attractive for industrial deployments. On the downside, LoRa achieves only low data rates requiring long on-air times, and significantly higher latencies compared to radios that are typically used for Internet access.

LoraWAN MAC Layer

The LoRaWAN MAC layer and network architecture that is often used in LoRa deployments, thus provide a vertically integrated sensor data delivery service on top of the LoRa PHY that implements media access and end-to-end network connectivity. Unfortunately, LoRaWAN cannot utilize the LoRa PHY to its best potential with respect to throughput and robustness and is mostly used for upstream-only communication. It is not intended to directly interconnect with the Internet, but relies on a bespoke middlebox architecture consisting of gateways and network servers. Overall LoRaWAN has the following main problems, as depicted in the figure below.

  1. Centralization around a network server prevents data sharing between users, across distributed applications, and requires permanent infrastructure backhaul of the wireless access network.
  2. Uplink-oriented and uncoordinated communication leads to wireless interference. Downlink traffic is rarely available in practice and suffers from scalability issues.

Data-centric Delay-tolerant End-to-End Communication over the Internet

This paper presents an overview about recent advancements to enable data-centric, long-range IoT communication based on LoRa. The proposed network system aims for delay-tolerant, bi-directional communication in the presence of vastly longer latencies and lower bandwidth compared to regular Internet systems – without relying on vertically integrated middlebox-based architectures. The resulting system resolves current LoRaWAN performance issues using two main building blocks: a new network layer based on Information-centric Networking (ICN) and a new MAC layer.

Originally designed for non-constrained wired networks to abandon the end-to-end paradigm and access data only by names instead of IP endpoints, ICN migrated to the constrained wireless IoT over the past years. ICN still lacks a lower layer definition but provides mechanisms that are beneficial for the challenging LoRa domain: Decoupling of content from endpoints separates data access from physical infrastructure. Inherent content caching and replication potentially reduce link load, thus, wireless interference, and it preserves battery resources. The ICN-LoRa system presented in this paper bases its design on IEEE 802.15.4 DSME which was originally designed for low-power personal area networks. This MAC handles media access reliably using time- and frequency multiplexing, and enables reliable bi-directional communication.

Synergizing the advantages of LoRa, DSME, and ICN enables delay-tolerant, bi-directional LoRa communication, wich enhances many existing IoT applications. Wide area data retrieval and control as for solar power stations or smart street lighting systems are facilitated by the new MAC and its ICN integration. High voltage overhead line monitoring connecting voltage sensors and transformers relies on high data reliability, even under intermittent connectivity or loss. ICN achieves this, employing content caching and replication. Traveling container monitoring (RFC 7744) is challenging due to mobility and interference from metallic surfaces, where LoRa surpasses other radio systems. Decoupling content from its location for mobile containers and an adaptation to long producer delays are naturally contributed by LoRa-ICN.

Results

In our paper, we provide the essential technical background and challenges to design a LoRa-ICN system. We identify the key performance potentials of five protocol variants based on an implementation in RIOT OS and experiments on off-the-shelf IoT devices.

LoRa is an attractive radio technology for the IoT, providing a long wireless transmission range for battery-driven devices. Its versatility is hindered, though, by common deployments with LoRaWAN. We re-visited LoRa in the IoT to provide a serverless, data-oriented communication service. We presented the design of a new media access and network layer that leverages 802.15.4 DSME and Information-centric Networking to allow for reliable LoRa transmissions. To scale to a global Internet (of Things), LoRa-ICN facilitates ubiquitous connectivity of constrained nodes and robust bi-directional communication in the presence of power-saving regimes and high loss rates.

We showed that vastly higher latencies in low-power wireless domains can be addressed by extending the default ICN node behavior at the network edge. Two protocol extensions enable ICN-style data transport between resource-constrained LoRa nodes and a domain-agnostic application on the ICN Internet. The core idea is not limited to LoRa but caters to various delay-prone scenarios. Our experiments based on common IoT hardware and software showed significant performance improvements and further optimization potential compared to Vanilla ICN.

The new LoRa-ICN system paves the way for more versatile LoRa deployments in the IoT that serve additional use cases, mixed sensor-actor topologies, or firmware updates utilizing beacon overloading.

References

This article

  • P. Kietzmann, J. Alamos, D. Kutscher, T. C. Schmidt and M. Wählisch, Rethinking LoRa for the IoT: An Information-centric Approach in IEEE Communications Magazine, doi: 10.1109/MCOM.001.2300379.

Reflexive forwarding

The ICN communication mechanisms this work is based on.

In-depth publications this work is based on

  • Peter Kietzmann, José Alamos, Dirk Kutscher, Thomas C. Schmidt, and Matthias Wählisch. 2022. Delay-tolerant ICN and its application to LoRa. In Proceedings of the 9th ACM Conference on Information-Centric Networking (ICN '22). Association for Computing Machinery, New York, NY, USA, 125–136. https://doi.org/10.1145/3517212.3558081
  • P. Kietzmann, J. Alamos, D. Kutscher, T. C. Schmidt and M. Wählisch, Long-Range ICN for the IoT: Exploring a LoRa System Design, 2022 IFIP Networking Conference (IFIP Networking), Catania, Italy, 2022, pp. 1-9, doi: 10.23919/IFIPNetworking55013.2022.9829792. https://ieeexplore.ieee.org/document/9829792
  • José Álamos, Peter Kietzmann, Thomas C. Schmidt, and Matthias Wählisch. 2022. DSME-LoRa: Seamless Long-range Communication between Arbitrary Nodes in the Constrained IoT. ACM Trans. Sen. Netw. 18, 4, Article 69 (November 2022), 43 pages. https://doi.org/10.1145/3552432

Written by dkutscher

November 29th, 2023 at 6:08 am

Posted in ICN,Publications

Tagged with , ,

DINRG @ IETF-118

without comments

We have posted the agenda for our DINRG meeting at IETF-118:

Documents

Logistics

DINRG Meeting at IETF-118 – 2023-11-06, 08:30 to 10:30 UTC

Written by dkutscher

November 1st, 2023 at 9:21 am

Posted in Events,IETF,IRTF

Tagged with , , ,

ICNRG @ IETF-118

without comments

Written by dkutscher

October 30th, 2023 at 1:10 pm

Posted in Events,ICN,IRTF

Tagged with , , ,

Collective Communication: Better Network Abstractions for AI

without comments

We have submitted two new Internet Drafts on Collective Communication:

  1. Kehan Yao , Xu Shiping , Yizhou Li , Hongyi Huang , Dirk Kutscher; Collective Communication Optimization: Problem Statement and Use cases; Internet Draft draft-yao-tsvwg-cco-problem-statement-and-usecases-00; work in progress; October 2023

  2. Kehan Yao , Xu Shiping , Yizhou Li , Hongyi Huang , Dirk Kutscher; Collective Communication Optimization: Requirement and Analysis; Internet Draft draft-yao-tsvwg-cco-requirement-and-analysis-00; work in progress; October 2023

Collective Communication refers to communication between a group of processes in distributed computing contexts, for example involving interaction types such as broadcast, reduce, all-reduce. This data-oriented communication model is employed by distributed machine learning and other data processing systems, such as stream processing. Current Internet network and transport protocols (and corresponding transport layer security) make it difficult to support these interactions in the network, e.g., for aggregating data on topologically optimal nodes for performance enhancements. These two drafts discuss use cases, problems, and initial ideas for requirements for future system and protocol design for Collective Communication. They will be discussed at IETF-118.

Written by dkutscher

October 30th, 2023 at 8:03 am

Seminar Talk: Accelerating Distributed Systems with In-Network Computation

without comments

In our invited talks series at HKUST(GZ), I am happy to be hosting Wenfei WU from Peking University on 2023-11-02, 14:00 CST, for his talk on Accelerating Distributed Systems with In-Network Computation.

Accelerating Distributed Systems with In-Network Computation

With Moore's Law slowing down, building distributed and heterogeneous systems becomes a new trend to support large-scale applications, such as large model training and big data analytics. In-Network Computing (INC) is an effective approach to building such distributed systems. INC leverages programmable network devices to process traversing data packets, and provides line-rate and low-latency data processing capabilities, which could compress traffic volume and accelerate the overall transmission and job efficiency. In this talk, we will share the progress and development of INC technologies, including INC protocol design for machine learning and data analytics, and RDMA-compatible INC solutions. These works are published in NSDI21 and ASPLOS23.

Wenfei WU

Wenfei Wu is an assistant professor from the School of Computer Science at Peking University. He obtained his Ph.D. degree from the University of Wisconsin-Madison in 2015. Dr. Wu researches into computer networks and distributed systems, and has published more than 50 papers in these areas. Dr. Wu's recent research focus is to build in-network computation (INC) methods for distributed systems; his work on INC-empowered distributed machine learning system ATP won the best paper award in NSDI 2021, and that on INC-empowered distributed data analytics system ASK won the distinguished paper award in ASPLOS 2023; Dr. Wu won other awards like IPCCC best paper runner-up in 2019, SoCC best student paper in 2013, etc.

Online Participation

https://calendar.hkust.edu.hk/events/iot-thrust-seminar-accelerating-distributed-systems-network-computation

Written by dkutscher

October 23rd, 2023 at 8:14 am

Posted in Events

Tagged with , ,

Network Abstractions for Continuous Innovation

without comments

In a joint panel at ACM ICN-2023 and IEEE ICNP-2023 in Reykjavik, Ken Calvert, Jim Kurose, Lixia Zhang, and myself discussed future network abstractions. The panel was moderated by Dave Oran. This was one of the more interesting and interactive panel sessions I participated in, so I am providing a summary here.

Since the Internet's initial rollout ~40 years ago, not only its global connectivity has brought fundamental changes to society and daily life, but its protocol suite and implementations have also gone through many iterations of changes, with SDN, NFV, and programmability among other changes over the last decade. This panel looks into next decade of network research by asking a set of questions regarding where lies the future direction to enable continued innovations.

Opportunities and Challenges for Future Network Innovations

Lixia Zhang: Rethinking Internet Architecture Fundamentals

Lixia Zhang (UCLA), quoting Einstein, said that the formulation of the problem is often more essential than the solution and pointed at the complexities of today's protocols stacks that are apparently needed to achieve desired functionality. For example, Lixia mentioned RFC 9298 on proxying UDP in HTTP, specifically on tunneling UDP to a server acting as a UDP-specific proxy over HTTP. UDP over IP was once conceived as a minial message-oriented communication service that was intended for DNS and interactive real-time communication. Due to its push-based communication model, it can be used with minimal effort for useful but also harmful application, including large-scale DDOS attacks. Proxing UDP over HTTP addresses this and other concerns, by providing a secure channel to a server in a web context, so that the server can authorize tunnel endpoints, and so that the UDP communication is congestion controlled by the underlying transport protocol (TCP or QUIC). This specification can be seen as a work-around: sending unsolicted (and un-authenticated) messages over the Internet is a major problem in today's Internet. There is no general approach for authenticating such messages and no concept for trust in peer identities. Instead of analyzing the root cause of such problems, the Internet communities (and the dominant players in that space) prefer to come up with (highly inefficient) workarounds.

This problem was discussed more generally by Oliver Spatscheck of AT&T Labs in his 2013 article titled Layers of Success, where he discussed the (actually deployed) excessive layering in production networks, for example mobile communication networks, where regular Internet traffic is routinely tunneled over GTP/UDP/IP/MPLS:

The main issue with layering is that layers hide information from each other. We could see this as a benefit, because it reduces the complexities involved in adding more layers, thus reducing the cost of introducing more services. However, hiding information can lead to complex and dynamic layer interactions that hamper the end-to-end system’s reliability and are extremely difficult if not impossible to debug and operate. So, much of the savings achieved when introducing new services is being spent operating them reliably.

According to Lixia, the excessive layering stems from more fundamental problems with today's network architecture, notably the lack of identity and trust in the core Internet protocols and the lack of functionality in the forwarding system – leading to significant problems today as exemplied by recent DDoS attacks. Quoting Einstein again, she said that we cannot solve problems by using the same kind of thinking we used when we created them, calling for a more fundamental redesign based on information-centric networking principles.

Ken Calvert: Domain-specific Networking

Ken Calvert (University of Kentucky) provided a retrospective of networking research and looked at selected papers published at the first IEEE ICNP conference in 1993. According to Ken, the dominant theme at that time was How to design, build, and analyze protocols, for example as discussed in his 1993 ICNP paper titled Beyond layering: modularity considerations for protocol architectures.

Ken offered a set of challenges and opportunities for future networking research, such as:

  • Domain-specific networking à la Ex uno pluria, a 2018 CCR editorial discussing:
    • infrastructure ossification;
    • lack of service innovation; and
    • a fragmentation into "ManyNets" that could re-create a service-infrastructure innovation cycle.
  • Incentives and "money flow"
    • Can we escape from the advertising-driven Internet app ecosystem? Should we?
  • Wide-area multicast (many-many) service
    • Building block for building distributed applications?
  • Inter-AS trust relationships
    • Ossification of the Inter-AS interface – cannot be solved by a protocol!
  • Impact ⇐ Applications ⇐ Business opportunities ($)
    • What user problem cannot be solved today?
  • "The core challenge of CS ... is a conceptual one, viz., what (abstract) mechanisms we can conceive without getting lost in the complexities of our own making." - Dijkstra

For his vision for networking in 30 years, Ken suggested that:

  • IP addresses will still be in use
    • but visible only at interfaces between different owners' infrastructures
  • Network infrastructure might consist of access ASes + separate core networks operated by the "Big Five".
  • Users might communicate via direct brain interfaces with AI systems.

Dirk Kutscher: Principled Approach to Network Programmability

I offered the perspective of introducing a principled approach to programmability that could provide better programmability (for humans and AI), based on more powerful network abstractions.

Previous work in SDN with protocols such as OpenFlow and dataplane programming languages such as P4 have only scratched the surface of what could be possible. OpenFlow was a great first idea, but it was fundamentally constrained by the IP and Ethernet-based abstractions that were built into it. It can be used for programming some applications in that domain, such as firewalls, virtual networking etc., but the idea of continuous innovation has not really materialized.

Similarly, P4 was advertized as an enabler for new levels of dataplane programmability, but even simple systems such as NetCache have to go to quite some extend to achieve minimal functionality for a proof-of-concept. Another P4 problem that is often reported is the hardware heterogeneity so that universal programmability is not really possible. In my opinion, this raises some questions with respect to applicability of current dataplane programming for in-network computing. A good example of a more productive application of P4 is the recent SIGCOMM paper on NetClone that describes as fast, scalable, and dynamic request cloning for microsecond-Scale RPCs. Here P4 is used as an accelerator for programming relatively simple functionality (protocol parsing, forwarding).

This may not be enough for future universal programmability though. During the panel discussion, I drew an analogy to computer programming language. We are not seeing the first programming language and IDEs that are designed from the ground up for better AI. What would that mean for network programmability? What abstractions and APIs would we need?

In my opinion, we would have to take a step back and think about the intended functionality and the required observability for future (automated) network programmability that is really protocol-independent. This would then entail more work on:

  • the fundamental forwarding service (informed by hardware constraints);
  • the telemetry approach;
  • suitable protocol semantics;
  • APIs for applications and management; and
  • new network emulation & debugging approach (a long the lines of "network digital twin" concepts).

Overall, I am expecting new exiciting research in the direction of principled approaches to network programmability.

Jim Kurose: Open Research Infrastructures and Softwarization

Jim reminded us that the key reason Internet research flourished was the availability of open infrastructure with no incumbent providers initially. The infrastructure was owned by researchers, labs, and universities and allowed for a lot of experimentation.

This open infrastructure has recently been challenged by ossification with the rise of production ISP services at scale, and the emergence of closed ISPs, cellular carriers, hyperscalers operating large portion of the network.

As an example for emerging environments that offer interesting opportunities for experiments and new developments, Jim mentioned 4G/5G private networks, i.e., licensed spectrum created closed ecosystems – but open to researchers, creating opportunities for:

Jim was also suggesting further opportunities in softwarization and programmability, such as (formal) methods for logical correctness and configuration management, as well as programmability to add services beyond the "minimal viable service", such as closed loop automatic control and management.

Finally Jim also mentioned opportunities in emerging new networks such as LEOs, IoT and home networks.

Written by dkutscher

October 23rd, 2023 at 7:46 am

The Metaverse as an Information-Centric Network

without comments

This is an introduction to our paper:

The Web Today

The Web today has a specific technical definition: it includes presentation layer technologies, protocols, agreed-upon ways of achieving certain semantics such as Representational State Transfer (REST), and security infrastructure. However, from a user perspective, it can be viewed as a universe of consistently navigable content and (occasionally) interoperable services. The user experience and architectural underpinnings have evolved in parallel and have influenced each other: for many end users, the Web and the network are synonymous. Rather than building up "Metaverse" as an application domain based on IP, we aim to explore "the Metaverse" as strongly intertwined with ICN, just as the modern concept of the Web and its technology stack are inseparable for a broad set of applications.

As a placeholder name for a range of new technologies and experiences, "the Metaverse" is even less well-defined than the Web. We adopt the commonly used concept of a shared, interoperable, and persistent XR. Some descriptions and early prototypes for social AR/VR systems suggest leveraging existing Internet and Web protocols to provide Metaverse services, without addressing the technical complexity and centralization of control required to provide the underlying cloud service infrastructure.

Metaverse as an Information-Centric Concept

Here, we do not take as given current designs and deployment models that consider the Metaverse as an overlay application with corresponding infrastructure dependencies, as this exacerbates the current gaps (and the resulting costs and technical complexity) between distributed applications and the underlying network architecture. Instead, we assume a fundamentally information/centric system in which most applications participate in granular 3D content exchange, context-aware integration with the physical world, and other Metaverse-relevant services.

"The Metaverse" is an information-centric concept that likely will become synonymous with the network itself. We argue that reciprocal design of the network and applications will open new opportunities for the deployment of Metaverse-suggestive experiences even today.

Experientially, this Metaverse is an extension of the Web into immersive XR modalities that are often aligned with physical space, as in augmented reality (AR). We conceive the Metaverse not only as a shared XR environment, but the next generation of the web, extending into 3D interaction/immersion and optionally overlaid on physical spaces. Instead of rendering data objects into a 2D page (within a tab within a window) on a device, we envision such objects being rendered into a shared 3D space, interacting among each other and with end users.

Architecturally, leveraging ICN concepts provides support for decentralized publishing, content interoperability and co-existence, based on general building blocks and not within separated application silos as today's initial prototypes. We claim that such properties are required to achieve the generally circulated visions of Metaverse systems, but are not achievable today because of the host- and connection-centric way in which the web operates and is presented to users in browsers.

ICN Capabilities

We point out four ICN capabilities critical to Metaverse concepts:

  1. scalable and robust multi-destination communication, overcoming IP multicast challenges, such as inter-domain routing, scalability, and routing communication overhead;
  2. leveraging wireless broadcast to support shared local views and low-latency interactivity without application-awareness in edge routers;
  3. privacy, selective attention, content filtering, and autonomous interactions, as well as ownership and control on the publishing side; and
  4. supporting in-network processing for objects replication and transformation.

Interactive Holographic Communication

For example, imagine interactive holographic communication consisting of participants' 3D video, spatial audio, and shared 3D documents. In ICN, such an application can represent virtual content as secure data objects and share them efficiently in a larger group of peers, fetching only the data necessary to reconstruct a suitable representation while being aware of the constraints of user devices and access networks.

Furthermore, while experiencing 3D objects shared by the group, each participant may also interact in the same XR environment with personal services such as wayfinding, messaging, and Internet of Things (IoT) device status. Interactions between private and shared 3D objects would be simplified if these objects use similar conventions but with different security. This concept is semantically well-aligned with ICN properties, particularly for security, as it revolves around object-level data exchange rather than hosts or channels. Integration and interoperability within a shared XR environment, without centralization, is challenging if one has to negotiate not only data interactions but also the underlying service connections and security relationship using host-centric paradigms. It also exacerbates the impact of intermittent connectivity on interactivity when the global network is required for functions such as rendezvous -- that are handled locally in ICN.

Creating Shared Environments

As a second example, consider creating a shared environment -- e.g., to pre-visualize engineering models of an aircraft – from a collection of collaboratively edited 3D documents. Imagine component documents interacting in a simulation. Documents can be modularized, linked, and overlaid in a web-like manner. Today, such cross-platform interoperability and visualization without centralized hubs is impractical, and it is difficult to create secure, granular data flows required for interaction between co-existing 3D elements to "bring them to life" in a virtual world. In an ICN approach, such modules could be independently authored and published, shared between applications, becoming building blocks of a richer, interacting system of user- and machine-generated content.

We introduce some technical challenges and research direction in our paper (link below).

Further Reading

The Metaverse as an Information-Centric Network

References

Written by dkutscher

September 19th, 2023 at 4:49 am

Posted in Publications

Tagged with , ,

Distributed Computing in Information-Centric Networking

without comments

This is an introduction to our paper:

Distributed computing is the basis for all relevant applications on the Internet. Based on well-established principles, different mechanisms, implementations, and applications have been developed that form the foundation of the modern Web.

The Internet with its stateless forwarding service and end-to-endcommunication model promotes certain types of communication for distributed computing. For example, IP addresses and/or DNS names provide different means for identifying computing components. Reliable transport protocols (e.g., TCP, QUIC) promote interconnecting modules. Communication patterns such as REST and protocol implementations such as HTTP enable certain types of distributed computing interactions, and security frameworks such as TLS and the web PKI constrain the use of public-key cryptography for different security functions.

From Distributed Computing...

Distributed computing has different facets, for example, client-server computing, web services, stream processing, distributed consensus systems, and Turing-complete distributed computing platforms. There are also different perspectives on how distributed computing should be implemented on servers and network platforms, a research area that we refer to as Computing in the Network. Active Networking, one of the earliest works on computing in the network, intended to inject programmability and customization of data packets in the network itself; however, security and complexity considerations proved to be major limiting factors, preventing its wider deployment.

Dataplane programmability refers to the ability to program behavior, including application logic, on network elements and SmartNICs, thus enabling some form in-network computing. Alternatively, different types of server platforms and light-weight execution environments are enabling other forms of distributing computation in networked systems, such as architectural patterns, such as edge computing.

... To Computing in the Network

With currently available Internet technologies, we can observe a relatively succinct layering of networking and distributed computing, i.e., distributed computing is typically implemented in overlays with Content Distribution Networks (CDNs) being prominent and ubiquitous example. Recently, there has been growing interest in revisiting this relationship, for example by the IRTF Computing in the NetworkResearch Group (COINRG) – motivated by advances in network and server platforms, e.g., through the development of programmable data plane platforms and the development of different types of distributed computing frameworks, e.g., stream processing and microservice frameworks.

This is also motivated by the recent development of new distributed computing applications such as distributed machine learning (ML), and emerging new applications such as Metaverse suggest new levels of scale in terms of data volume for distributed computing and the pervasiveness of distributed computing tasks in such systems. There are two research questions that stem from these developments:

  1. How can we build distributed computing systems in the network that can leverage the on-path location of compute functions, e.g., optimally aligning stream processing topologies with networked computing platform topologies?

  2. How can the network support distributed computing in general, so that the design and operation of such systems can be simplified, but also so that different optimizations can be achieved to improve performance and robustness?

Issues in Legacy Distributed Computing

Although there are many distributed computing applications, it is also worth noting that there are many limitations and performance issues. Factors such as network latency, data skew, checkpoint overhead, back pressure, garbage collection overhead, and issues related to performance, memory management, and serialization and deserialization overhead can all influence the efficiency. Various optimization techniques can be implemented to alleviate these issues, including memory adjustment, refining the checkpointing process, and adopting efficient data structures and algorithms.

Some performance problems and complexity issues stem from the overlay nature of current systems and their way of achieving the above-mentioned mechanisms with temporary solutions based on TCP/IP and associated protocols such as DNS. For example, Network Service Mesh has been characterized as architecturally complex because of the so-called sidecar approaches and their implementation problems.

In systems that are layered on top of HTTP or TCP (or QUIC), compute nodes typically cannot assess the network performance directly – only indirectly through observed throughput and buffer under-runs. Information-centric data-flow systems, such as IceFlow, intend to provide better visibility and thus better joint optimization potential by more direct access to data-oriented communication resources. Then, some coordination tasks that are based on exchanging updates of shared application state can be elegantly mapped to named data publication in a hierarchical namespace, as the different dataset synchronization (Sync) protocols in NDN demonstrated.

Information-Centric Distributed Computing

In our paper on Distributed Computing in ICN at ACM ICN-2023, we focus on distributed computing and on how information-centricity in the network and application layer can support the development and operation of such systems. The rich set of distributed computing systems in ICN suggests that ICN provides some benefits for distributed computing that could offer advantages such as better performance, security, and productivity when building corresponding applications.

ICN with its data-oriented operation and generally more powerful forwarding layer provides an attractive platform for distributed computing. Several different distributed computing protocols and systems have been proposed for ICN, with different feature sets and different technical approaches, including Remote Method Invocation (RMI) as an interaction model as well as more comprehensive distributed computing platforms. RMI systems such as RICE leverage the fundamental named-based forwarding service in ICN systems and map requests to Interest messages and method names to content names (although the actual implementation is more intricate). Method parameters and results are also represented as content objects, which provides an elegant platform for such interactions.

ICN generally attempts to provide a more useful service to data-oriented applications but can also be leveraged to support distributed computing specifically.

Names

Accessing named data in the network as a native service can remove the need for mapping application logic identifiers such as function names to network and process identifiers (IP addresses, port numbers), thus simplifying implementation and run-time operation, as demonstrated by systems such as Named Function Networking (NFN), RICE, and IceFlow. It is worth noting that, although ICN does not generally require an explicit mapping of names to other domain identifiers, such networks require suitable forwarding state, e.g., obtained from configuration, dynamic learning, or routing.

Data-orientedness

ICN's notion of immutable data with strong name-content binding through cryptographic signatures and hashes seems to be conducive to many distributed computing scenarios, as both static data objects and dynamic computation results in those systems such as input parameters and result values can be directly sent as ICN data objects. NFN has first demonstrated this.

Securing distributed computing could be supported better in so far as ICN does not require additional dependencies on public-key or pipe securing infrastructure, as keys and certificates are simply named data objects and centralized trust anchors are not necessarily needed. Larger data collections can be aggregated and re-purposed by manifests (FLIC), enabling "small" and "big data" computing in one single framework that is congruent to the packet-level communication in a network. IceFlow uses such an aggregation approach to share identical stream processing results objects in multiple consumer contexts.

Data-orientedness eliminates the need for connections; even reliable communication in ICN is completely data-oriented. If higher-layer (distributed computing) transactions can be mapped to the network layer data retrieval, then server complexity can be reduced (no need to maintain several connections), and consumers get direct visibility into network performance. This can enable performance optimizations, such as linking network and computing flow control loops (one realization of joint optimization), as showed by IceFlow.

Location independence and data sharing

Embracing the principle of accessing named and authenticated data also enables location independence, i.e., corresponding data can be obtained from any place in the network, such as replication points (repos) and caches. This fundamentally enables better multi-source/path capabilities as well as data sharing, i.e., multiple data retrieval operations for one named data object by different consumers can potentially be completed by a cache, repo, or peer in the network.

Stateful Forwarding

ICN provides stateful, symmetric forwarding, which enables general performance optimizations such as in-network retransmissions, more control over multipath forwarding, and load balancing. This concept could be extended to support distributed computing specifically, for example, if load balancing is performed based on RTT observations for idempotent remote-method invocations.

More Networking, less Management

The combination of data-oriented, connection-less operation, and stateful (more powerful) forwarding in ICN shifts functionality from management and orchestration layers (back) to the network layer, which can enable complexity reduction, which can be especially pronounced in distributed computing. For example, legacy stream processing and service mesh platforms typically must manage connectivity between deployment units (pods in Kubernetes). In Apache Flink, a central orchestrator manages the connections between task managers (node agents). Systems such as IceFlow have demonstrated a more self-organized and decentralized stream-processing approach, and the presented principles are applicable to other forms of distributed computing.

In summary, we can observe that ICN's general approach of having the network providing a more natural (data retrieval) platform for applications benefits distributed computing in similar ways as it benefits other applications. One particularly promising approach is the elimination of layer barriers, which enables certain optimizations.

In addition to NFN, there are other approaches that jointly optimize the utilization of network and computing resources to provide network service mesh-like platforms, such as edge intelligence using federated learning, advanced CDNs where nodes can dynamically adapt to user demands according to content popularity, such as iCDN and OpenCDN, and general computing systems, such as Compute-First Networking, IceFlow, and ICedge.

Our paper on Distributed Computing in ICN at ACM ICN-2023 provides a comprehensive analysis and understanding of distributed computing systems in ICN, based on a survey of more than 50 papers. Naturally, these different efforts cannot be directly compared due to their difference in nature. We categorized different ICN distributed computing systems, and individual approaches and highlighted their specific properties.

The scope of this study is technologies for ICN-enabled distributed computing. Specifically, we divide the different approaches into four categories, as shown in the figure above: enablers, protocols, orchestration, and applications. The contributions of this study are as follows:

  1. A discussion of the benefits and challenges of distributed computing in ICN.
  2. A categorization of different proposed distributed computing systems in ICN.
  3. A discussion of lessons learned from these systems.
  4. A discussion of existing challenges and promising directions for future work.

Recent Research on Distributed Computing in ICN

I am providing some pointers to my previous research on distributed computing in ICN below.

The paper that has led to this article:

Current work in the Computing in the Network Research Group of the IRTF:

  • Dirk Kutscher, Teemu Kärkkäinen, Jörg Ott; Directions for Computing in the Network; Internet Draft draft-irtf-coinrg-dir-00, Work in Progress; August 2023

Reflexive Forwarding and Remote Method Invocation

Providing a unified remote computation capability in ICN presents some unique challenges, among which are timer management, client authorization, and binding to state held by servers, while maintaining the advantages of ICN protocol designs like CCN and NDN. In the RICE work,we developed a unified approach to remote function invocation in ICN that exploits the attractive ICN properties of name-based routing, receiver-driven flow and congestion control, flow balance, and object-oriented security while presenting a natural programming model to the application developer. The RICE protocol is leveraging an ICN extension called Reflexive Forwarding that provides ICN-idiomatic method parameter transmission.

Distributed Computing Frameworks

Leveraging RICE as a mechanism, we have developed Compute-First Networking (CFN) in ICN, a Turing-complete distributed computing platform. IceFlow is a proposal for Dataflow in ICN in a decentralized manner.

Applications

Based on Reflexive Forwarding, we have developed a concept for RESTful ICN that leverages CCNx key exchange for setting up security contexts and keys that could then be used for secure, data-oriented REST-like communication.

Delay-Tolerant LoRa leveraged Reflexive Forwarding to enable constrained LoRa nodes to "phone home" when they want to transmit data, thus enabling new ways (without central network and application servers) for connecting LoRa networks to the Internet.

Written by dkutscher

September 19th, 2023 at 3:47 am

Reflexive Forwarding in Named Data Networking

without comments

Current Information-Centric Networking protocols such as CCNx and NDN have a wide range of useful applications in content retrieval and other scenarios that depend only on a robust two-way exchange in the form of a request and response (represented by an Interest-Data exchange in the case of the two protocols noted above). A number of important applications however, require placing large amounts of data in the Interest message, and/or more than one two-way handshake.

While these can be accomplished using independent Interest-Data exchanges by reversing the roles of consumer and producer, such approaches can be both clumsy for applications and problematic from a state management, congestion control, or security standpoint. Reflexive Forwarding is a proposed extension to the CCNx and NDN protocol architectures that eliminates the problems inherent in using independent Interest-Data exchanges for such applications.

The protocol is specified in draft-oran-icnrg-reflexive-forwarding and has been used in a few of our research projects such as:

My student intern Xinchen Jin from ShanghaiTech has implemented the Reflexing Forwarding specification in NDN (with modifications to ndn-cxx and NFD) and set up a testbed in mini-NDN for experiments over multiple forwarders.

Resources

Written by dkutscher

August 29th, 2023 at 7:24 am

Posted in Code,ICN,IRTF

Tagged with ,

Platforms, Economics, Minimal Global Broadcast

without comments

Decentralization of the Internet Research Group at IETF-117

The Decentralization of the Internet Research Group (DINRG) of the Internet Research Task Force (IRTF) had a meeting on 2023-07-27 at the 117th meeting of the Internet Engineering Task Force (IETF). DINRG aims to provide for the research and engineering community, both an open forum to discuss the Internet centralization phenomena and associated potential threats, and a platform to facilitate the coordination of efforts in identifying the causes of observed consolidations and the mitigation solutions.

For context, we recently published a workshop report that discusses some fundamental problems: ACM SIGCOMM CCR: Report of 2021 DINRG Workshop on Centralization in the Internet


The DINRG meeting at IETF-117 meeting featured three highly interesting talks by Cory Doctorow, Volker Stocker & William Lehr, and Christian Tschudin that created quite some attention and led to lively discussions during and after the meeting. There is a full meeting recording on youTube, and we have published meeting minutes. Special thanks to Ryo Yanagida, A.J. Stein, and Eve Schooler for taking notes at the meeting.

Cory Doctorow: Let The Platforms Burn: Bringing Back the Good Fire of the Old Internet

Cory Doctorow, a science fiction author, activist and journalist, talked about a trend in platform evolution that he calls enshittification, where platforms go through different phases after growing user bases quickly as per platform economics and market domination strategies and of locking in users through technical and economic barriers. In advertisement (and digital online market) platforms, the platform operator sits between the users and other companies (so-called "two-sided market" scenarios), where the user base, the obtained personal information and behavioral surveillance results become assets to attracts such companies.

For example, in order to make avertising more effective, social media platforms would increase control of users's timelines, i.e., content that is presented to them, and make it harder for users to leave the platform. Overall this results in negative user experience. For increasing advertisment revenue, platforms would then sell attention more directly, i.e., exploit their position in the advertisement market. See Cory's posting on enshittification for details.

This process and the difficulties in effectively controlling and regulating platform companies, has led to a permanent crisis that Cory compares to a fire hazard situation. Platforms were rocked by scandals private data theft, accidental leaks and intended data sharing with other insitutions etc.

While the computer and networking world has seen a constant emerging and vanishing of "platforms" (operating systems, PC companies, online services) before, the current concentrated tech market makes is impossible to let harmful (or not very user-friendly) dissolve. This is due to network effects (Metcalfe's law) and switching costs, for example when trying to leave a dominant social media platforms and thereby losing connections to friends. This monopoly situation is enabled by a legal environment with ineffective antitrust laws, which has allowed for dominating platform to constantly acquire competing companies and potentially disruptive businesses.

With new laws for content moderation and censorship, platform get even more control over their users (in the name of preventing harrassment), without making in any easier to leave platforms. In his article (and podcast epsidode) called "Let the Platforms Burn", Cory concluded

Platforms collapse "slowly, then all at once." The only way to prevent sudden platform collapse syndrome is to block interoperability so users can't escape the harms of your walled garden without giving up the benefits they give to each other.

We should stop trying to make the platforms good. We should make them gone. We should restore the "good fire" that ended with the growth of financialized Big Tech empires. We should aim for soft landings for users, and stop pretending that there's any safe way to life in the fire zone.

We should let the platforms burn.

WIth respect to the (de-)centralization discussion in DINRG and the Internet community, this raises some important question as to

  • what is the role of open interfaces, standards etc today in reality? Are we still using them to build interoperable, possibly federated systems?
  • how should technology development, standards setting and regulation evolve to effectively enable user choice (migration, platform selection)?

Discussion

There was a question whether the real issue was that platforms are making a remarkable griphold, buying each other, but they are buying the users, i.e., whether the primary concern is the size of these platforms with this method or the method itself only? Cory replied in saying that size certainly promotes distortions. Scale was problem for two reasons. The contract enforcement function dominates. When the referee is less powerful than the team, it allows teams to cheat.
Secondly, even if we stipulated that companies are well run by smart people, they all make errors, and at that scale the mistakes are much more consequential.

Another question was who is willing to implement the interoperability standards and how companies can be convinved to do that. Cory talked about companies' motivation, i.e., companies wanted walled gardens, or to have APIs with advantages (vs disadvantages) to them. What they really seeked (over competitive interoperability) was to have legal remedies for those who reverse engineer to competitively enter the market. When there was a mandate and permission for inter-operators, if restoring that power was possible, that would help to avoid unquantifiable risk.

Some of these strategies are discussed in Cory upcoming book "How to seize the means of computation”.

Volker Stocker and William Lehr: Ecosystem Evolution and Digital Infrastructure Policy Challenges: Insights & Reflections from an Economics Perspective"

Volker Stocker of the Weizenbaum Institute for the Networked Society and William Lehr of the Advanced Networking Architecture Group in CSAIL at MIT presented their research on ecosysem evolution and policy challenges from an economics perspective. Volker is an economist with broad experience in interdisciplinay research, and William is a telecommunications and Internet economist and consultant.


Volker talked about the convergence of digital and non-digital worlds and mentioned a few trends that needed attention:

  • The shift to the edge and shift to the localization of traffic.
  • Ownership and management has shifted in the Internet ecosystem: sometimes hyper giant content providers with proprietary networks, sometimes edge clouds or roving resources.
  • Potential consequences: value chain constellations are more complex, diverse & dynamic, resulting in changing ownership and governance structures, industry structures as well as competititve and innovational dynamics.

Volker made three points in his reflections on ecosystem evolution:

  1. Essential digital infrastructure is about more than connectiivty, not just connectivities like IPX and ISPs.
  2. The majority of the requisite investmenet will be private! E.g., access ISPs, CAPs, CDNs, upstream ISPs, and end-users are all investing.
  3. More and new forms of resource sharing will be needed. More network sharing agreements: active & passive sharing arrangements and optimal models are evolving.

William highlighted that the legacy Internet is not the Internet of today, and the economics of yesterday are not those of today. One of the questions is how to restore meaningful competition?

He mentioned the following challenges and paths forward:

  1. Multidisciplinary Engagement & Feedback
  2. Assymetric Info & Measurements: Metrics and data (and their provenance)
  3. Capacity to Detect and Act

Discussion

There was a discussion about how the private sector is expected to profit from the infrastructure development needed by society (assuming investments from the private sector). William replied in saying that
government built/subsidized most infrastructure in most places, with small investments needed initially. Some say significant investment should come from the utilities, which we should not dismiss. But we likely will need a strong argument on how to get there. Either we say there is a lot of money coming from public sector (for example, through taxes) or we have to find a way to manage private actors. Thus policy issues are important. Some of these questions are discussed in Williams paper on "Getting to the Broadband Future Efficiently with BEAD funding”.

Another question alluded to policy lagging behing the technical development, i.e., the mismatch of speed of innovation and speed of regulation (which is really hard at the national and internationl levels). William said that the best hope is standards and architectures that provides options and mentioned the importance of open source software.

Christian Tschudin: Minimal Global Broadcast

Christian Tschudin of the University of Basel presented a research idea called "Minimal Global Broadcast" (abstract). Christian is a computer science professor with a track record of research in Information-Centric Networking, distributed computing, and decentralized systems.

Christian started out from the observation that contacting peers in a decentralized environment is challenging. The key question is how do you learn about a peer’s current coordinates and their preferences? The platforms themselves often offer directories, but these are logically centralized rendezvous servers with a partial view and require trust in these platforms. Instead of conceptualizing an uber directory service Christian proposed a global information dissemination system that focuses on the data, asserting an allowance of “200 bytes of novelty per month
and citizen”.

This global broadcast channel can (and should) be implemented in many ways, starting from sneakernets to shortwave communication and including Internet-based online-services. Christian explained how such a service could be used to facilitate user migration and user discovery on their current preferred platform(s).

Discussion

There were some question on trust in user identities. Christian said that trust roots would be external to MGB, and that there would be different levels of trust, e.g., for inter-personal relationship vs. business relationships.

References

Written by dkutscher

August 21st, 2023 at 5:40 pm

Posted in IETF,IRTF

Tagged with , ,