Multi-Node LLM Inference: From One Node to N · Appendix

Bibliography

Every source cited across the 18 modules, deduplicated (112 entries), ordered by source type: official documentation and standards first, then papers, textbooks, repositories, talks, blogs, third-party analyses.

112 sources18 modulesGenerated 2026-09-17

All sources

  1. officialAMD. Multi-node setup for AI workloads. ROCm Documentation, rocm.docs.amd.com. https://rocm.docs.amd.com/en/latest/how-to/rocm-for-ai/system-setup/multi-node-setup.html. Accessed 2026-09-17. Cited in: Module 17.
  2. officialAMD. vLLM V1 performance optimization. ROCm Documentation, rocm.docs.amd.com. https://rocm.docs.amd.com/en/latest/how-to/rocm-for-ai/inference-optimization/vllm-optimization.html. Accessed 2026-09-17. Cited in: Module 17.
  3. officialAnyscale. Understand LLM latency and throughput metrics. docs.anyscale.com, 2026. https://docs.anyscale.com/llm/serving/benchmarking/metrics. Accessed 2026-09-17. Cited in: Module 3, Module 15.
  4. officialDeepSeek-AI. DeepSeek-R1 Release (API pricing announcement). api-docs.deepseek.com/news, 2025-01-20. https://api-docs.deepseek.com/news/news250120. Accessed 2026-09-17. Cited in: Module 6, Module 15.
  5. officialEnvoy Proxy contributors. External Processing Filter (ext_proc). Envoy Proxy, latest docs, 2026. https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/ext_proc_filter. Accessed 2026-09-17. Cited in: Module 10.
  6. officialGoogle. Perform multihost inference using Pathways (AI Hypercomputer Documentation). 2026. https://docs.cloud.google.com/ai-hypercomputer/docs/workloads/pathways-on-cloud/multihost-inference. Accessed 2026-09-17. Cited in: Module 18.
  7. officialGoogle. TPU architecture (system-architecture-tpu-vm, Google Cloud Documentation). 2026. https://docs.cloud.google.com/tpu/docs/system-architecture-tpu-vm. Accessed 2026-09-17. Cited in: Module 18.
  8. officialGoogle. TPU v6e (Google Cloud Documentation). 2026. https://docs.cloud.google.com/tpu/docs/v6e. Accessed 2026-09-17. Cited in: Module 18.
  9. officialGoogle. TPU7x (Google Cloud Documentation). 2026. https://docs.cloud.google.com/tpu/docs/tpu7x. Accessed 2026-09-17. Cited in: Module 18.
  10. officialKServe contributors. Multi-node/Multi-GPU Inference (KServe docs). https://kserve.github.io/website/docs/model-serving/generative-inference/multi-node. Accessed 2026-09-17. Cited in: Module 14.
  11. officialKubernetes contributors. Safely Drain a Node (Kubernetes Documentation). https://kubernetes.io/docs/tasks/administer-cluster/safely-drain-node/. Accessed 2026-09-17. Cited in: Module 14.
  12. officialKubernetes SIG contributors. API Reference: InferencePool (Kubernetes Gateway API Inference Extension docs). Kubernetes SIGs, live docs, 2026. https://gateway-api-inference-extension.sigs.k8s.io/reference/spec/. Accessed 2026-09-17. Cited in: Module 10.
  13. officialKubernetes SIG contributors. FAQ (Kubernetes Gateway API Inference Extension docs). Kubernetes SIGs, live docs, 2026. https://gateway-api-inference-extension.sigs.k8s.io/faq/. Accessed 2026-09-17. Cited in: Module 10.
  14. officialKubernetes SIG contributors. InferencePool (Kubernetes Gateway API Inference Extension docs). Kubernetes SIGs, InferencePool GA since v1.0.0; endpointPickerRef optional since v1.5.0, 2026. https://gateway-api-inference-extension.sigs.k8s.io/api-types/inferencepool/. Accessed 2026-09-17. Cited in: Module 10.
  15. officialKubernetes SIG contributors. LeaderWorkerSetSpec API Reference (LWS documentation). https://lws.sigs.k8s.io/docs/reference/leaderworkerset.v1/. Accessed 2026-09-17. Cited in: Module 14.
  16. officialLenovo. ThinkSystem AMD MI300X 192GB 750W 8-GPU Board (product guide, reproducing AMD's published MI300X specifications). https://lenovopress.lenovo.com/lp1943-thinksystem-amd-mi300x-192gb-750w-8-gpu-board. Accessed 2026-09-17. Cited in: Module 17.
  17. officialllm-d contributors. Architecture (llm-d docs). llm-d, live docs (v0.9.0 banner), 2026. https://llm-d.ai/docs/architecture. Accessed 2026-09-17. Cited in: Module 10.
  18. officialllm-d contributors. KEDA with EPP Metrics (llm-d docs). https://llm-d.ai/docs/dev/architecture/advanced/autoscaling/keda-epp. Accessed 2026-09-17. Cited in: Module 14.
  19. officialllm-d contributors. llm-d (homepage). llm-d, live site, 2026. https://llm-d.ai/. Accessed 2026-09-17. Cited in: Module 10.
  20. officialllm-d contributors. Prefix-Cache Aware Routing (llm-d docs). llm-d, docs/dev (live), 2026. https://llm-d.ai/docs/dev/architecture/advanced/kv-management/prefix-cache-aware-routing. Accessed 2026-09-17. Cited in: Module 2, Module 16.
  21. officialLMCache contributors. Example: Offload KV cache to CPU (LMCache Documentation). docs.lmcache.ai, 2026. https://docs.lmcache.ai/getting_started/quickstart/offload_kv_cache.html. Accessed 2026-09-17. Cited in: Module 8.
  22. officialNVIDIA (ai-dynamo). NIXL Architecture (nixl.md, main branch docs). 2026. https://github.com/ai-dynamo/nixl/blob/main/docs/nixl.md. Accessed 2026-09-17. Cited in: Module 1, Module 8, Module 13, Module 17.
  23. officialNVIDIA / TensorRT-LLM contributors. Disaggregated Serving (disagg-serving.md). TensorRT-LLM documentation, main branch. https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/features/disagg-serving.md. Accessed 2026-09-17. Cited in: Module 4, Module 8, Module 13, Module 16.
  24. officialNVIDIA / TensorRT-LLM team. Disaggregated Serving in TensorRT LLM. NVIDIA TensorRT-LLM Tech Blog, 2026. https://nvidia.github.io/TensorRT-LLM/blogs/tech_blog/blog5_Disaggregated_Serving_in_TensorRT-LLM.html. Accessed 2026-09-17. Cited in: Module 3, Module 13.
  25. officialNVIDIA. Disaggregated Serving (NVIDIA Dynamo Documentation). docs.nvidia.com/dynamo/knowledge-base/concepts/system-architecture/disaggregated-serving. https://docs.nvidia.com/dynamo/knowledge-base/concepts/system-architecture/disaggregated-serving. Accessed 2026-09-17. Cited in: Module 4, Module 8, Module 9, Module 13, Module 16.
  26. officialNVIDIA. GB200 NVL72 (product page). n/a, 2026. https://www.nvidia.com/en-us/data-center/gb200-nvl72/. Accessed 2026-09-17. Cited in: Module 1, Module 5, Module 13, Module 18.
  27. officialNVIDIA. H100 GPU (product page). n/a, 2026. https://www.nvidia.com/en-us/data-center/h100/. Accessed 2026-09-17. Cited in: Module 1, Module 17, Module 18.
  28. officialNVIDIA. HGX (product page). n/a, 2026. https://www.nvidia.com/en-us/data-center/hgx/. Accessed 2026-09-17. Cited in: Module 1.
  29. officialNVIDIA. KV-Aware Routing (NVIDIA Dynamo Documentation). docs.nvidia.com/dynamo/dev/knowledge-base/concepts/system-architecture/kv-aware-routing, 2026. https://docs.nvidia.com/dynamo/dev/knowledge-base/concepts/system-architecture/kv-aware-routing. Accessed 2026-09-17. Cited in: Module 9.
  30. officialNVIDIA. KVBM Architecture (NVIDIA Dynamo Documentation). v-0-8-1. https://docs.nvidia.com/dynamo/v-0-8-1/components/kvbm/architecture. Accessed 2026-09-17. Cited in: Module 9.
  31. officialNVIDIA. KVBM Design (NVIDIA Dynamo Design Docs). v1.3.0. https://docs.nvidia.com/dynamo/v1.3.0/design-docs/component-design/kvbm-design. Accessed 2026-09-17. Cited in: Module 9.
  32. officialNVIDIA. Metrics Catalog (NVIDIA Dynamo Documentation, reference). docs.nvidia.com/dynamo/reference/observability/metrics-catalog. https://docs.nvidia.com/dynamo/reference/observability/metrics-catalog. Accessed 2026-09-17. Cited in: Module 9.
  33. officialNVIDIA. NCCL-RDMA-SHARP Plugins (HPC-X Software Toolkit docs). 2026. https://networking-docs.nvidia.com/hpcxum/212/NCCL-RDMA-SHARP+Plugins. Accessed 2026-09-17. Cited in: Module 1.
  34. officialNVIDIA. NVIDIA ConnectX-7 400G Ethernet (datasheet). Apr21. https://www.nvidia.com/content/dam/en-zz/Solutions/networking/ethernet-adapters/connectx-7-datasheet-Final.pdf. Accessed 2026-09-17. Cited in: Module 1.
  35. officialNVIDIA. NVIDIA ConnectX-7 NDR 400G InfiniBand Adapter Card (datasheet). Apr21, 2021. https://www.nvidia.com/content/dam/en-zz/Solutions/networking/infiniband-adapters/infiniband-connectx7-data-sheet.pdf. Accessed 2026-09-17. Cited in: Module 1.
  36. officialNVIDIA. NVIDIA DGX B200 (datasheet). 2025. https://www.aspsys.com/wp-content/uploads/2025/05/nvidia-dgx-b200-datasheet.pdf. Accessed 2026-09-17. Cited in: Module 17.
  37. officialNVIDIA. NVIDIA Enters Production With Dynamo, the Broadly Adopted Inference Operating System for AI Factories. NVIDIA Newsroom, 2026-03-16. https://nvidianews.nvidia.com/news/dynamo-1-0. Accessed 2026-09-17. Cited in: Module 9.
  38. officialNVIDIA. NVIDIA Quantum-X800 InfiniBand Platform (product page). 2026. https://www.nvidia.com/en-us/networking/products/infiniband/quantum-x800/. Accessed 2026-09-17. Cited in: Module 1.
  39. officialNVIDIA. Planner Guide (NVIDIA Dynamo Documentation). v1.2.1. https://docs.nvidia.com/dynamo/v1.2.1/components/planner/planner-guide. Accessed 2026-09-17. Cited in: Module 9, Module 14, Module 16.
  40. officialNVIDIA. Router: Configuration and Tuning (NVIDIA Dynamo Documentation). v1.2.1. https://docs.nvidia.com/dynamo/v1.2.1/components/router/configuration-and-tuning. Accessed 2026-09-17. Cited in: Module 9.
  41. officialNVIDIA. Running Disaggregated Serving with Triton TensorRT-LLM Backend. Archived documentation (Triton Inference Server release 2580). https://docs.nvidia.com/deeplearning/triton-inference-server/archives/triton-inference-server-2580/user-guide/docs/tensorrtllm_backend/all_models/disaggregated_serving/README.html. Accessed 2026-09-17. Cited in: Module 13.
  42. officialNVIDIA. TensorRT-LLM Backend (README, Triton Inference Server user guide). Live docs, 2026. https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/tensorrtllm_backend/README.html. Accessed 2026-09-17. Cited in: Module 13.
  43. officialNVIDIA. Understanding Your Grace-Blackwell Systems (multi-node-nvlink-systems tuning guide docs). 2026. https://docs.nvidia.com/multi-node-nvlink-systems/multi-node-tuning-guide/system.html. Accessed 2026-09-17. Cited in: Module 1.
  44. officialRay / Anyscale contributors. Serving LLMs (Ray Serve Documentation). Ray 2.58.0. https://docs.ray.io/en/latest/serve/llm/index.html. Accessed 2026-09-17. Cited in: Module 14.
  45. officialSGLang contributors. PD Disaggregation. SGLang Documentation, docs.sglang.io/advanced_features/pd_disaggregation.html. https://docs.sglang.io/advanced_features/pd_disaggregation.html. Accessed 2026-09-17. Cited in: Module 4, Module 5, Module 7, Module 8, Module 12, Module 13, Module 16, Module 17.
  46. officialvLLM contributors. Disaggregated Prefilling. vLLM Documentation, docs.vllm.ai/en/latest/features/disagg_prefill/, package v0.29.0 (2026-09-09). https://docs.vllm.ai/en/latest/features/disagg_prefill/. Accessed 2026-09-17. Cited in: Module 4, Module 8, Module 11, Module 13, Module 17.
  47. officialvLLM contributors. LWS (vLLM Documentation, deployment/frameworks/lws). https://docs.vllm.ai/en/stable/deployment/frameworks/lws/. Accessed 2026-09-17. Cited in: Module 14.
  48. officialvLLM contributors. Metrics. vLLM Design Documentation, docs.vllm.ai/en/latest/design/metrics/ (v1 engine). https://docs.vllm.ai/en/latest/design/metrics/. Accessed 2026-09-17. Cited in: Module 11, Module 14.
  49. officialvLLM Project. Expert Parallel Deployment (vLLM Documentation). 2026. https://docs.vllm.ai/en/latest/serving/expert_parallel_deployment/. Accessed 2026-09-17. Cited in: Module 5, Module 11.
  50. standardInfiniBand Trade Association. IBTA Announces New RoCE Specification. n/a. https://www.infinibandta.org/ibta-announces-new-roce-specification/. Accessed 2026-09-17. Cited in: Module 1.
  51. paperBarham, P., Chowdhery, A., Dean, J., Ghemawat, S., Hand, S., Hurt, D., Isard, M., Lim, H., Pang, R., Roy, S., Saeta, B., Schuh, P., Sepassi, R., El Shafey, L., Thekkath, C. A., Wu, Y. Pathways: Asynchronous Distributed Dataflow for ML. MLSys 2022 / arXiv:2203.12533, 2022. https://arxiv.org/abs/2203.12533. Accessed 2026-09-17. Cited in: Module 18.
  52. paperDeepSeek-AI. DeepSeek-V3 Technical Report. arXiv:2412.19437, 2024-12-26. https://arxiv.org/pdf/2412.19437. Accessed 2026-09-17. Cited in: Module 6, Module 16.
  53. paperPatel, P., Choukse, E., Zhang, C., Shah, A., Goiri, I., Maleki, S., Bianchini, R. Splitwise: Efficient Generative LLM Inference Using Phase Splitting. ISCA 2024 / arXiv:2311.18677v2, 2023. https://arxiv.org/pdf/2311.18677. Accessed 2026-09-17. Cited in: Module 1, Module 3.
  54. paperQin, R., Li, Z., He, W., Zhang, M., Wu, Y., Zheng, W., Xu, X. Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving. USENIX FAST 2025 (Best Paper). arXiv:2407.00079v4. https://arxiv.org/pdf/2407.00079. Accessed 2026-09-17. Cited in: Module 3, Module 7, Module 8, Module 11, Module 15, Module 16.
  55. paperSun, B., Huang, Z., Zhao, H., Xiao, W., Zhang, X., Li, Y., Lin, W. Llumnix: Dynamic Scheduling for Large Language Model Serving. arXiv:2406.03243v1, USENIX OSDI 2024. https://arxiv.org/pdf/2406.03243. Accessed 2026-09-17. Cited in: Module 16.
  56. paperThe AIBrix Team. AIBrix: Towards Scalable, Cost-Effective Large Language Model Inference Infrastructure. arXiv:2504.03648, 2025. https://arxiv.org/abs/2504.03648. Accessed 2026-09-17. Cited in: Module 14.
  57. paperZhong, Y., Liu, S., Chen, J., Hu, J., Zhu, Y., Liu, X., Jin, X., Zhang, H. DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving. USENIX OSDI 2024. arXiv:2401.09670v3. https://arxiv.org/pdf/2401.09670. Accessed 2026-09-17. Cited in: Module 3, Module 11, Module 15, Module 16.
  58. repoAIBrix contributors. vllm-project/aibrix (README). v0.7.0 (2026-06-16). https://github.com/vllm-project/aibrix. Accessed 2026-09-17. Cited in: Module 14.
  59. repoDeepSeek-AI. Day 6 (One More Thing): DeepSeek-V3/R1 Inference System Overview. open-infra-index, #OpenSourceWeek, 2025-02. https://github.com/deepseek-ai/open-infra-index/blob/main/202502OpenSourceWeek/day_6_one_more_thing_deepseekV3R1_inference_system_overview.md. Accessed 2026-09-17. Cited in: Module 6, Module 15, Module 16.
  60. repoDeepSeek-AI. deepseek-ai/DeepEP (docs/legacy.md, V1 archived documentation). 2026. https://github.com/deepseek-ai/DeepEP/blob/main/docs/legacy.md. Accessed 2026-09-17. Cited in: Module 5.
  61. repoDeepSeek-AI. deepseek-ai/DeepEP (README, main branch, v2 kernels). 2026. https://github.com/deepseek-ai/DeepEP/blob/main/README.md. Accessed 2026-09-17. Cited in: Module 1, Module 5.
  62. repoDeepSeek-AI. deepseek-ai/DeepEP (README). GitHub, main branch. https://github.com/deepseek-ai/DeepEP. Accessed 2026-09-17. Cited in: Module 15.
  63. repoDeepSeek-AI. deepseek-ai/EPLB (README, main branch). 2026. https://github.com/deepseek-ai/EPLB/blob/main/README.md. Accessed 2026-09-17. Cited in: Module 5, Module 6, Module 16.
  64. repoGoogle (AI-Hypercomputer). JetStream (README, main branch). 2026. https://github.com/AI-Hypercomputer/JetStream/blob/main/README.md. Accessed 2026-09-17. Cited in: Module 18.
  65. repoHazyResearch contributors. ThunderKittens (GitHub repository README). Stanford University, Hazy Research Lab, main branch (v2.0, released 2026-01-11). https://github.com/HazyResearch/ThunderKittens. Accessed 2026-09-17. Cited in: Module 16.
  66. repoKubernetes SIG / Gateway API Inference Extension contributors. Endpoint Picker Protocol (004-endpoint-picker-protocol). Kubernetes SIGs, main branch proposal doc, 2026. https://github.com/kubernetes-sigs/gateway-api-inference-extension/tree/main/docs/proposals/004-endpoint-picker-protocol. Accessed 2026-09-17. Cited in: Module 10.
  67. repoKubernetes SIG / Gateway API Inference Extension contributors. EPP Architecture Proposal (0683-epp-architecture-proposal). Kubernetes SIGs, main branch proposal doc, 2026. https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/docs/proposals/0683-epp-architecture-proposal/README.md. Accessed 2026-09-17. Cited in: Module 2, Module 10.
  68. repoKubernetes SIG contributors. kubernetes-sigs/gateway-api-inference-extension (README). Kubernetes SIGs, GA since v1.0.0, 2026. https://github.com/kubernetes-sigs/gateway-api-inference-extension. Accessed 2026-09-17. Cited in: Module 2, Module 9, Module 10, Module 14.
  69. repoKubernetes SIG contributors. kubernetes-sigs/gateway-api-inference-extension Releases. Kubernetes SIGs, v1.6.1 (2026-09-11), 2026. https://github.com/kubernetes-sigs/gateway-api-inference-extension/releases. Accessed 2026-09-17. Cited in: Module 10.
  70. repoKubernetes SIG contributors. kubernetes-sigs/lws (LeaderWorkerSet README). Main branch. https://github.com/kubernetes-sigs/lws/blob/main/README.md. Accessed 2026-09-17. Cited in: Module 11, Module 14.
  71. repollm-d contributors. llm-d-inference-scheduler architecture.md. llm-d (CNCF Sandbox; Red Hat, Google Cloud, IBM Research, CoreWeave, NVIDIA), main branch, 2026. https://github.com/llm-d/llm-d-inference-scheduler/blob/main/docs/architecture.md. Accessed 2026-09-17. Cited in: Module 2, Module 10.
  72. repollm-d contributors. llm-d/llm-d (README). llm-d (CNCF Sandbox since 2026-03-12; Red Hat, Google Cloud, IBM Research, CoreWeave, NVIDIA), v0.7-v0.9, 2026. https://github.com/llm-d/llm-d. Accessed 2026-09-17. Cited in: Module 10.
  73. repollm-d contributors. llm-d/llm-d Releases. llm-d, v0.9.0 (2026-08-17), 2026. https://github.com/llm-d/llm-d/releases. Accessed 2026-09-17. Cited in: Module 10.
  74. repollm-d contributors. workload-autoscaling guide (llm-d/llm-d repository, main branch). https://github.com/llm-d/llm-d/blob/main/guides/workload-autoscaling/README.md. Accessed 2026-09-17. Cited in: Module 14.
  75. repoLMCache contributors. LMCache/LMCache (README). Main branch, 2026. https://github.com/LMCache/LMCache. Accessed 2026-09-17. Cited in: Module 8.
  76. repoMoonshot AI / MADSys contributors. kvcache-ai/Mooncake (README, main branch). 2026. https://github.com/kvcache-ai/Mooncake/blob/main/README.md. Accessed 2026-09-17. Cited in: Module 7, Module 8.
  77. repoNVIDIA / ai-dynamo contributors. ai-dynamo/dynamo (README). v1.4.2 container tags; v1.5.0-dev builds referenced, 2026. https://github.com/ai-dynamo/dynamo. Accessed 2026-09-17. Cited in: Module 9, Module 10, Module 12, Module 16.
  78. repoNVIDIA / ai-dynamo contributors. ai-dynamo/dynamo Releases. v1.4.2 (2026-08-29), v1.5.0-dev builds in progress. https://github.com/ai-dynamo/dynamo/releases. Accessed 2026-09-17. Cited in: Module 9, Module 16.
  79. repoNVIDIA / ai-dynamo contributors. ai-dynamo/grove (README). https://github.com/ai-dynamo/grove. Accessed 2026-09-17. Cited in: Module 9.
  80. repoNVIDIA / ai-dynamo contributors. ai-dynamo/nixl Releases. v1.4.1, 2026-09-01. https://github.com/ai-dynamo/nixl/releases. Accessed 2026-09-17. Cited in: Module 8, Module 16.
  81. repoopenucx/ucx. README.md. 2026. https://github.com/openucx/ucx/blob/master/README.md. Accessed 2026-09-17. Cited in: Module 1.
  82. repoROCm. mori (Modular RDMA Interface, README). 2026. https://github.com/ROCm/mori. Accessed 2026-09-17. Cited in: Module 17.
  83. repoROCm. rccl (README.md, develop branch). 2026. https://github.com/ROCm/rccl/blob/develop/README.md. Accessed 2026-09-17. Cited in: Module 17.
  84. reposgl-project/sglang GitHub. [Roadmap] Prefill-Decode Disaggregation Roadmap (2026 Q2). Issue #21703, opened 2026-03-30. https://github.com/sgl-project/sglang/issues/21703. Accessed 2026-09-17. Cited in: Module 12.
  85. reposgl-project/sglang GitHub. SGLang Router Architecture Improvement Proposal. Issue #7532, opened 2025-06-25. https://github.com/sgl-project/sglang/issues/7532. Accessed 2026-09-17. Cited in: Module 12.
  86. repoSGLang / RadixArk contributors. SGLang Model Gateway (README). sgl-project/sglang repository, main branch. https://github.com/sgl-project/sglang/blob/main/sgl-model-gateway/README.md. Accessed 2026-09-17. Cited in: Module 12.
  87. repoSGLang contributors. sglang-router (PyPI package page). SGLang / RadixArk, 0.3.2, 2026-01-15. https://pypi.org/project/sglang-router/. Accessed 2026-09-17. Cited in: Module 2, Module 12, Module 16.
  88. repovLLM Project contributors. vllm-project/production-stack (README). vLLM Project, released 2026-01-22. https://github.com/vllm-project/production-stack. Accessed 2026-09-17. Cited in: Module 2, Module 8, Module 11.
  89. repovLLM Project. tpu-inference (README, incl. live feature-support matrices). 2026. https://github.com/vllm-project/tpu-inference. Accessed 2026-09-17 (support-matrix snapshot dated 2026-08-27 in-repo). Cited in: Module 18.
  90. talkSmith, A., Alla, V. AMD Instinct MI300X Generative AI Accelerator and Platform Architecture. Hot Chips 2024, August 2024. https://hc2024.hotchips.org/assets/program/conference/day1/23_HC2024.AMD.MI300X.ASmith(MI300X).v1.Final.20240817.pdf. Accessed 2026-09-17. Cited in: Module 17.
  91. blogAMD ROCm Blogs. DI Series: Scaling GLM-5.1-FP8 to 64 MI300X GPUs. August 2026. https://rocm.blogs.amd.com/software-tools-optimization/di-glm-wideep/README.html. Accessed 2026-09-17. Cited in: Module 17.
  92. blogCNCF. Welcome llm-d to the CNCF: Evolving Kubernetes into SOTA AI Infrastructure. Cloud Native Computing Foundation, 2026-03-24. https://www.cncf.io/blog/2026/03/24/welcome-llm-d-to-the-cncf-evolving-kubernetes-into-sota-ai-infrastructure/. Accessed 2026-09-17. Cited in: Module 10.
  93. blogDatabricks Mosaic Research team. LLM Inference Performance Engineering: Best Practices. Databricks Blog, 2023-10-12. https://www.databricks.com/blog/llm-inference-performance-engineering-best-practices. Accessed 2026-09-17. Cited in: Module 15.
  94. blogGoogle Cloud. AI Hypercomputer inference updates for Google Cloud TPU and GPU. Google Cloud Blog, 2025. https://cloud.google.com/blog/products/compute/ai-hypercomputer-inference-updates-for-google-cloud-tpu-and-gpu. Accessed 2026-09-17. Cited in: Module 18.
  95. blogGoogle Cloud. Inside the Ironwood TPU codesigned AI stack. Google Cloud Blog, 2026. https://cloud.google.com/blog/products/compute/inside-the-ironwood-tpu-codesigned-ai-stack. Accessed 2026-09-17. Cited in: Module 18.
  96. blogHazy Research. ThunderMLA: FlashMLA, Faster and Fused-er! Stanford University, published 2025-03-04. https://hazyresearch.stanford.edu/blog/2025-03-04-thundermla. Accessed 2026-09-17. Cited in: Module 16.
  97. blogLMSYS Org / RadixArk. RadixArk Joins Forces with Google to Bring Full SGLang Features to TPUs. lmsys.org, 2026-07-30. https://www.lmsys.org/blog/2026-07-30-sglang-google-tpu/. Accessed 2026-09-17. Cited in: Module 12, Module 18.
  98. blogLMSYS Org. Deploying DeepSeek with PD Disaggregation and Large-Scale Expert Parallelism on 96 H100 GPUs. lmsys.org, 2025-05-05. https://www.lmsys.org/blog/2025-05-05-large-scale-ep/. Accessed 2026-09-17. Cited in: Module 6, Module 12, Module 15, Module 16.
  99. blogLMSYS Org. SGLang v0.4: Zero-Overhead Batch Scheduler, Cache-Aware Load Balancer, Faster Structured Outputs. lmsys.org, 2024-12-04. https://www.lmsys.org/blog/2024-12-04-sglang-v0-4/. Accessed 2026-09-17. Cited in: Module 12.
  100. blogLMSYS Org. Win on TCO: How AMD Instinct MI355X Achieves Cost-Competitive Distributed Inference Through SGLang with MoRI. May 28, 2026. https://www.lmsys.org/blog/2026-05-28-mori/. Accessed 2026-09-17. Cited in: Module 17.
  101. blogMarkthub, P., Dinan, J., Potluri, S., Howell, S. Improving Network Performance of HPC Systems Using NVIDIA Magnum IO NVSHMEM and GPUDirect Async. NVIDIA Developer Blog, 2022-11-22. https://developer.nvidia.com/blog/improving-network-performance-of-hpc-systems-using-nvidia-magnum-io-nvshmem-and-gpudirect-async. Accessed 2026-09-17. Cited in: Module 6.
  102. blogMeta Engineering. RoCE networks for distributed AI training at scale. Engineering at Meta, 2024. https://engineering.fb.com/2024/08/05/data-center-engineering/roce-network-distributed-ai-training-at-scale/. Accessed 2026-09-17. Cited in: Module 1.
  103. blogNVIDIA Developer Blog. NVIDIA NVLink and NVIDIA NVSwitch Supercharge Large Language Model Inference. 2026. https://developer.nvidia.com/blog/nvidia-nvlink-and-nvidia-nvswitch-supercharge-large-language-model-inference/. Accessed 2026-09-17. Cited in: Module 1.
  104. blogNVIDIA. Benchmarking GPUDirect RDMA on Modern Server Platforms. NVIDIA Developer Blog, 2014. https://developer.nvidia.com/blog/benchmarking-gpudirect-rdma-on-modern-server-platforms/. Accessed 2026-09-17. Cited in: Module 1.
  105. blogNVIDIA. How NVIDIA Dynamo 1.0 Powers Multi-Node Inference at Production Scale. NVIDIA Developer Blog. https://developer.nvidia.com/blog/nvidia-dynamo-1-production-ready/. Accessed 2026-09-17. Cited in: Module 9.
  106. blogNVIDIA. ModelExpress: Distributing Model Artifacts at the Speed of Light. NVIDIA Developer Blog. https://developer.nvidia.com/blog/modelexpress-distributing-model-artifacts-at-the-speed-of-light. Accessed 2026-09-17. Cited in: Module 9.
  107. blogNVIDIA. NVIDIA Dynamo 0.4 Delivers 4x Faster Performance, SLO-Based Autoscaling, and Real-Time Observability. NVIDIA Developer Blog, published 2025-08-13. https://developer.nvidia.com/blog/dynamo-0-4-delivers-4x-faster-performance-slo-based-autoscaling-and-real-time-observability. Accessed 2026-09-17. Cited in: Module 9.
  108. blogNVIDIA. NVIDIA Dynamo Accelerates llm-d Community Initiatives for Advancing Large-Scale Distributed Inference. NVIDIA Developer Blog, published 2025-05-21. https://developer.nvidia.com/blog/nvidia-dynamo-accelerates-llm-d-community-initiatives-for-advancing-large-scale-distributed-inference/. Accessed 2026-09-17. Cited in: Module 9.
  109. blogScaling Intelligence Lab (Hazy Research-adjacent group). Tokasaurus: An LLM Inference Engine for High-Throughput Workloads. Stanford University, published 2025. https://scalingintelligence.stanford.edu/blogs/tokasaurus/. Accessed 2026-09-17. Cited in: Module 16.
  110. blogvLLM Team. vLLM TPU: A New Unified Backend Supporting PyTorch and JAX on TPU. vLLM Blog, 2025-10-16. https://blog.vllm.ai/2025/10/16/vllm-tpu.html. Accessed 2026-09-17. Cited in: Module 18.
  111. analysisSemiAnalysis / InferenceX. AMD Instinct MI355X — Specs, Pricing & AI Inference Benchmarks. 2026. https://inferencex.semianalysis.com/chips/mi355x. Accessed 2026-09-17. Cited in: Module 17.
  112. analysisSemiAnalysis. InferenceMAX: Open Source Inference Benchmarking. InferenceMAX v1, 2026. https://newsletter.semianalysis.com/p/inferencemax-open-source-inference. Accessed 2026-09-17. Cited in: Module 3, Module 15, Module 16, Module 17.