Archive for the ‘scheduling’ tag
SAFS accepted at IEEE ICNP
Our paper on "SAFS: Slack-Aware Flow Scheduling for Pipelined AI Workloads" has been accepted at the 34th IEEE International Conference on Network Protocols (ICNP).
Abstract
Modern artificial intelligence (AI) systems increasingly rely on pipelined execution across accelerators, stages, and disaggregated components. In these systems, a network transfer is harmful only when it arrives after the downstream computation has exhausted the slack that would otherwise hide the communication. Existing datacenter transports observe bytes, congestion, flow sizes, or externally assigned deadlines, but they do not know whether a particular activation, gradient, or hidden-state transfer is on the current pipeline critical path.
We present Slack-Aware Flow Scheduling (SAFS), a receiverdriven transport for pipelined AI workloads. SAFS exposes each communication flow as a semantic dependency with remaining bytes and a runtime-derived no-bubble timestamp. Receivers translate this metadata into no-bubble rate demand, allocate safe scheduled rate across contending senders, and amortize control overhead with reusable rate grants. We apply the same abstraction to pipeline-parallel (PP) training and attention–feed-forward network (FFN) disaggregation (AFD) inference.
In ZeroBubble training, SAFS is best or close to best across 0–5 background flows, reducing mean iteration time under contention by protecting the stage-boundary transfer whose lateness would become exposed stall. In 16A2F inference, where 16 attention instances feed two FFN instances, SAFS reduces p99 time per output token (TPOT) by 38.7% relative to D2TCP and 66.2% relative to Homa. Targeted microbenchmarks show zero urgent deadline misses and reduce mean urgent flow completion time to 14.3 ms.
Reference
Ruilong Wu, Yisu Wang, Jiaxiang Zou, Xiangsen Chen, Yixiang Zhu, Xinjiao Li, Xinyu Chen, Dirk Kutscher; SAFS: Slack-Aware Flow Scheduling for Pipelined AI Workloads; accepted for publication at the 34th IEEE International Conference on Network Protocols (ICNP); Tempe, Arizona, USA, October 05-08, 2026