Multi-node Serving · Part 2

Prefill/Decode Disaggregation

The queueing-theoretic case for splitting prefill and decode into separate pools, and the runtime mechanics -- KV transfer, scheduling, failure handling -- that make the split work.

03Why Disaggregation Pays: Goodput, Interference, and the Ratio Decision25 min04KV Transfer and Scheduling Across the Prefill/Decode Boundary30 min