Dirk Kutscher

Our paper on NetSenseML: Network-Adaptive Compression for
Efficient Distributed Machine Learning has been accepted at the 31st International European on Parallel and Distributed Computing (Euro-Par-2025).

Abstract:
Training large-scale distributed machine learning models imposes considerable demands on network infrastructure, often resulting in sudden traffic spikes that lead to congestion, increased latency, and reduced throughput, which would ultimately affect convergence times and overall training performance. While gradient compression techniques are commonly employed to alleviate network load, they frequently compromise model accuracy due to the loss of gradient information.

This paper introduces NetSenseML, a novel network adaptive distributed deep learning framework that dynamically adjusts quantization, pruning, and compression strategies in response to real-time network conditions. By actively monitoring network conditions, NetSenseML applies gradient compression only when network congestion negatively impacts convergence speed, thus effectively balancing data payload reduction and model accuracy preservation.

Our approach ensures efficient resource usage by adapting reduction techniques based on current network conditions, leading to shorter convergence times and improved training efficiency. We present the design of the NetSenseML adaptive data reduction function and experimental evaluations show that NetSenseML can improve training throughput by a factor of 1.55x to 9.84x compared to state-of-the-art compression-enabled systems for representative DDL training jobs in bandwidth-constrained conditions.

References

Yisu Wang, Xinjiao Li, Ruilong Wu, Huangxun Chen, Dirk Kutscher; NetSenseML: Network-Adaptive Compression for Efficient Distributed Machine Learning; 31st International European on Parallel and Distributed Computing (Euro-Par-2025); August 2025; accepted for publication

With the increasing amount of IP traffic in fixed and mobile access networks, the question how to share the available resources in a fair, efficient and demand-oriented way becomes more and more prevalent. With the variety of services one can find in today’s Internet, the requirements in rate, data volume and latency differ strongly. To maximize resource utilization and, at the same time, provide satisfying performance to all users, application layer knowledge is needed. As different resource allocation and adaptation mechanisms already exist in MAC, transport and application layer, an integral consideration of the problem space is required.

In wireless networks, the problem is particularly relevant due to the inherently limited resources, which render a simple “throwing bandwidth at the problem” solution impossible. Because large over-provisioning factors are economically unfeasibly, similar questions on capacity sharing also arise in fixed access networks, such as high bandwidth passive optical networks or cable networks.

NEC Laboratories Europe and the Institute of Communication Networks and Computer Engineering (IKR) of the University of Stuttgart are organizing workshop on Capacity Sharing to address these topics. The objective of this workshop is to bring together stakeholders of mobile and fixed access networks, the classic Internet world and of the application and transport community. We solicit presentations on the state-of-the-art, results of ongoing research, open issues, trends and new ideas. We are especially looking forward to (possibly provocative) visionary presentations to foster a lively discussion about how to face the upcoming challenges in the future mobile Internet. Topics of particular interest include, but are not limited to

• Application-layer adaption for mobile services
• Transport layer solutions and possible interactions with cellular/fixed access networks
• Context-aware resource allocation & cross-layer adaptation
• QoE and fairness definitions, metrics and evaluation
• Data traffic characteristics in fixed and mobile Internet
• Economic aspects on capacity sharing and business models
• Similarities and differences of capacity sharing in mobile and fixed access networks
• Related standardization activities and projects

The workshop takes place in Stuttgart on Thursday, October 13, 2011, and is organized by Mirja Kühlewind (IKR), Christian Mueller (IKR) and myself.

More information: http://www.ikr.uni-stuttgart.de/CapacitySharingWS/