Multi-node Serving · Part 1

The Multi-Node Frame

Establishes the bandwidth/latency arithmetic and the routing problem that every later part refers back to: what changes, physically and logically, the instant a second node appears.

01Interconnect and Transport: The Physical Layer of Multi-Node Serving25 min02Replication and Routing: KV-Cache-Aware, Prefix-Aware, and SLO-Aware Request Placement30 min