Multi-Node LLM Inference: From One Node to N · Appendix
Bibliography
Every source cited across the 18 modules, deduplicated (112 entries), ordered by source type: official documentation and standards first, then papers, textbooks, repositories, talks, blogs, third-party analyses.
All sources
- officialAMD. Multi-node setup for AI workloads. ROCm Documentation, rocm.docs.amd.com. https://rocm.docs.amd.com/en/latest/how-to/rocm-for-ai/system-setup/multi-node-setup.html. Accessed 2026-09-17. Cited in: Module 17.
- officialAMD. vLLM V1 performance optimization. ROCm Documentation, rocm.docs.amd.com. https://rocm.docs.amd.com/en/latest/how-to/rocm-for-ai/inference-optimization/vllm-optimization.html. Accessed 2026-09-17. Cited in: Module 17.
- officialAnyscale. Understand LLM latency and throughput metrics. docs.anyscale.com, 2026. https://docs.anyscale.com/llm/serving/benchmarking/metrics. Accessed 2026-09-17. Cited in: Module 3, Module 15.
- officialDeepSeek-AI. DeepSeek-R1 Release (API pricing announcement). api-docs.deepseek.com/news, 2025-01-20. https://api-docs.deepseek.com/news/news250120. Accessed 2026-09-17. Cited in: Module 6, Module 15.
- officialEnvoy Proxy contributors. External Processing Filter (ext_proc). Envoy Proxy, latest docs, 2026. https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/ext_proc_filter. Accessed 2026-09-17. Cited in: Module 10.
- officialGoogle. Perform multihost inference using Pathways (AI Hypercomputer Documentation). 2026. https://docs.cloud.google.com/ai-hypercomputer/docs/workloads/pathways-on-cloud/multihost-inference. Accessed 2026-09-17. Cited in: Module 18.
- officialGoogle. TPU architecture (system-architecture-tpu-vm, Google Cloud Documentation). 2026. https://docs.cloud.google.com/tpu/docs/system-architecture-tpu-vm. Accessed 2026-09-17. Cited in: Module 18.
- officialGoogle. TPU v6e (Google Cloud Documentation). 2026. https://docs.cloud.google.com/tpu/docs/v6e. Accessed 2026-09-17. Cited in: Module 18.
- officialGoogle. TPU7x (Google Cloud Documentation). 2026. https://docs.cloud.google.com/tpu/docs/tpu7x. Accessed 2026-09-17. Cited in: Module 18.
- officialKServe contributors. Multi-node/Multi-GPU Inference (KServe docs). https://kserve.github.io/website/docs/model-serving/generative-inference/multi-node. Accessed 2026-09-17. Cited in: Module 14.
- officialKubernetes contributors. Safely Drain a Node (Kubernetes Documentation). https://kubernetes.io/docs/tasks/administer-cluster/safely-drain-node/. Accessed 2026-09-17. Cited in: Module 14.
- officialKubernetes SIG contributors. API Reference: InferencePool (Kubernetes Gateway API Inference Extension docs). Kubernetes SIGs, live docs, 2026. https://gateway-api-inference-extension.sigs.k8s.io/reference/spec/. Accessed 2026-09-17. Cited in: Module 10.
- officialKubernetes SIG contributors. FAQ (Kubernetes Gateway API Inference Extension docs). Kubernetes SIGs, live docs, 2026. https://gateway-api-inference-extension.sigs.k8s.io/faq/. Accessed 2026-09-17. Cited in: Module 10.
- officialKubernetes SIG contributors. InferencePool (Kubernetes Gateway API Inference Extension docs). Kubernetes SIGs, InferencePool GA since v1.0.0; endpointPickerRef optional since v1.5.0, 2026. https://gateway-api-inference-extension.sigs.k8s.io/api-types/inferencepool/. Accessed 2026-09-17. Cited in: Module 10.
- officialKubernetes SIG contributors. LeaderWorkerSetSpec API Reference (LWS documentation). https://lws.sigs.k8s.io/docs/reference/leaderworkerset.v1/. Accessed 2026-09-17. Cited in: Module 14.
- officialLenovo. ThinkSystem AMD MI300X 192GB 750W 8-GPU Board (product guide, reproducing AMD's published MI300X specifications). https://lenovopress.lenovo.com/lp1943-thinksystem-amd-mi300x-192gb-750w-8-gpu-board. Accessed 2026-09-17. Cited in: Module 17.
- officialllm-d contributors. Architecture (llm-d docs). llm-d, live docs (v0.9.0 banner), 2026. https://llm-d.ai/docs/architecture. Accessed 2026-09-17. Cited in: Module 10.
- officialllm-d contributors. KEDA with EPP Metrics (llm-d docs). https://llm-d.ai/docs/dev/architecture/advanced/autoscaling/keda-epp. Accessed 2026-09-17. Cited in: Module 14.
- officialllm-d contributors. llm-d (homepage). llm-d, live site, 2026. https://llm-d.ai/. Accessed 2026-09-17. Cited in: Module 10.
- officialllm-d contributors. Prefix-Cache Aware Routing (llm-d docs). llm-d, docs/dev (live), 2026. https://llm-d.ai/docs/dev/architecture/advanced/kv-management/prefix-cache-aware-routing. Accessed 2026-09-17. Cited in: Module 2, Module 16.
- officialLMCache contributors. Example: Offload KV cache to CPU (LMCache Documentation). docs.lmcache.ai, 2026. https://docs.lmcache.ai/getting_started/quickstart/offload_kv_cache.html. Accessed 2026-09-17. Cited in: Module 8.
- officialNVIDIA (ai-dynamo). NIXL Architecture (nixl.md, main branch docs). 2026. https://github.com/ai-dynamo/nixl/blob/main/docs/nixl.md. Accessed 2026-09-17. Cited in: Module 1, Module 8, Module 13, Module 17.
- officialNVIDIA / TensorRT-LLM contributors. Disaggregated Serving (disagg-serving.md). TensorRT-LLM documentation, main branch. https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/features/disagg-serving.md. Accessed 2026-09-17. Cited in: Module 4, Module 8, Module 13, Module 16.
- officialNVIDIA / TensorRT-LLM team. Disaggregated Serving in TensorRT LLM. NVIDIA TensorRT-LLM Tech Blog, 2026. https://nvidia.github.io/TensorRT-LLM/blogs/tech_blog/blog5_Disaggregated_Serving_in_TensorRT-LLM.html. Accessed 2026-09-17. Cited in: Module 3, Module 13.
- officialNVIDIA. Disaggregated Serving (NVIDIA Dynamo Documentation). docs.nvidia.com/dynamo/knowledge-base/concepts/system-architecture/disaggregated-serving. https://docs.nvidia.com/dynamo/knowledge-base/concepts/system-architecture/disaggregated-serving. Accessed 2026-09-17. Cited in: Module 4, Module 8, Module 9, Module 13, Module 16.
- officialNVIDIA. GB200 NVL72 (product page). n/a, 2026. https://www.nvidia.com/en-us/data-center/gb200-nvl72/. Accessed 2026-09-17. Cited in: Module 1, Module 5, Module 13, Module 18.
- officialNVIDIA. H100 GPU (product page). n/a, 2026. https://www.nvidia.com/en-us/data-center/h100/. Accessed 2026-09-17. Cited in: Module 1, Module 17, Module 18.
- officialNVIDIA. HGX (product page). n/a, 2026. https://www.nvidia.com/en-us/data-center/hgx/. Accessed 2026-09-17. Cited in: Module 1.
- officialNVIDIA. KV-Aware Routing (NVIDIA Dynamo Documentation). docs.nvidia.com/dynamo/dev/knowledge-base/concepts/system-architecture/kv-aware-routing, 2026. https://docs.nvidia.com/dynamo/dev/knowledge-base/concepts/system-architecture/kv-aware-routing. Accessed 2026-09-17. Cited in: Module 9.
- officialNVIDIA. KVBM Architecture (NVIDIA Dynamo Documentation). v-0-8-1. https://docs.nvidia.com/dynamo/v-0-8-1/components/kvbm/architecture. Accessed 2026-09-17. Cited in: Module 9.
- officialNVIDIA. KVBM Design (NVIDIA Dynamo Design Docs). v1.3.0. https://docs.nvidia.com/dynamo/v1.3.0/design-docs/component-design/kvbm-design. Accessed 2026-09-17. Cited in: Module 9.
- officialNVIDIA. Metrics Catalog (NVIDIA Dynamo Documentation, reference). docs.nvidia.com/dynamo/reference/observability/metrics-catalog. https://docs.nvidia.com/dynamo/reference/observability/metrics-catalog. Accessed 2026-09-17. Cited in: Module 9.
- officialNVIDIA. NCCL-RDMA-SHARP Plugins (HPC-X Software Toolkit docs). 2026. https://networking-docs.nvidia.com/hpcxum/212/NCCL-RDMA-SHARP+Plugins. Accessed 2026-09-17. Cited in: Module 1.
- officialNVIDIA. NVIDIA ConnectX-7 400G Ethernet (datasheet). Apr21. https://www.nvidia.com/content/dam/en-zz/Solutions/networking/ethernet-adapters/connectx-7-datasheet-Final.pdf. Accessed 2026-09-17. Cited in: Module 1.
- officialNVIDIA. NVIDIA ConnectX-7 NDR 400G InfiniBand Adapter Card (datasheet). Apr21, 2021. https://www.nvidia.com/content/dam/en-zz/Solutions/networking/infiniband-adapters/infiniband-connectx7-data-sheet.pdf. Accessed 2026-09-17. Cited in: Module 1.
- officialNVIDIA. NVIDIA DGX B200 (datasheet). 2025. https://www.aspsys.com/wp-content/uploads/2025/05/nvidia-dgx-b200-datasheet.pdf. Accessed 2026-09-17. Cited in: Module 17.
- officialNVIDIA. NVIDIA Enters Production With Dynamo, the Broadly Adopted Inference Operating System for AI Factories. NVIDIA Newsroom, 2026-03-16. https://nvidianews.nvidia.com/news/dynamo-1-0. Accessed 2026-09-17. Cited in: Module 9.
- officialNVIDIA. NVIDIA Quantum-X800 InfiniBand Platform (product page). 2026. https://www.nvidia.com/en-us/networking/products/infiniband/quantum-x800/. Accessed 2026-09-17. Cited in: Module 1.
- officialNVIDIA. Planner Guide (NVIDIA Dynamo Documentation). v1.2.1. https://docs.nvidia.com/dynamo/v1.2.1/components/planner/planner-guide. Accessed 2026-09-17. Cited in: Module 9, Module 14, Module 16.
- officialNVIDIA. Router: Configuration and Tuning (NVIDIA Dynamo Documentation). v1.2.1. https://docs.nvidia.com/dynamo/v1.2.1/components/router/configuration-and-tuning. Accessed 2026-09-17. Cited in: Module 9.
- officialNVIDIA. Running Disaggregated Serving with Triton TensorRT-LLM Backend. Archived documentation (Triton Inference Server release 2580). https://docs.nvidia.com/deeplearning/triton-inference-server/archives/triton-inference-server-2580/user-guide/docs/tensorrtllm_backend/all_models/disaggregated_serving/README.html. Accessed 2026-09-17. Cited in: Module 13.
- officialNVIDIA. TensorRT-LLM Backend (README, Triton Inference Server user guide). Live docs, 2026. https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/tensorrtllm_backend/README.html. Accessed 2026-09-17. Cited in: Module 13.
- officialNVIDIA. Understanding Your Grace-Blackwell Systems (multi-node-nvlink-systems tuning guide docs). 2026. https://docs.nvidia.com/multi-node-nvlink-systems/multi-node-tuning-guide/system.html. Accessed 2026-09-17. Cited in: Module 1.
- officialRay / Anyscale contributors. Serving LLMs (Ray Serve Documentation). Ray 2.58.0. https://docs.ray.io/en/latest/serve/llm/index.html. Accessed 2026-09-17. Cited in: Module 14.
- officialSGLang contributors. PD Disaggregation. SGLang Documentation, docs.sglang.io/advanced_features/pd_disaggregation.html. https://docs.sglang.io/advanced_features/pd_disaggregation.html. Accessed 2026-09-17. Cited in: Module 4, Module 5, Module 7, Module 8, Module 12, Module 13, Module 16, Module 17.
- officialvLLM contributors. Disaggregated Prefilling. vLLM Documentation, docs.vllm.ai/en/latest/features/disagg_prefill/, package v0.29.0 (2026-09-09). https://docs.vllm.ai/en/latest/features/disagg_prefill/. Accessed 2026-09-17. Cited in: Module 4, Module 8, Module 11, Module 13, Module 17.
- officialvLLM contributors. LWS (vLLM Documentation, deployment/frameworks/lws). https://docs.vllm.ai/en/stable/deployment/frameworks/lws/. Accessed 2026-09-17. Cited in: Module 14.
- officialvLLM contributors. Metrics. vLLM Design Documentation, docs.vllm.ai/en/latest/design/metrics/ (v1 engine). https://docs.vllm.ai/en/latest/design/metrics/. Accessed 2026-09-17. Cited in: Module 11, Module 14.
- officialvLLM Project. Expert Parallel Deployment (vLLM Documentation). 2026. https://docs.vllm.ai/en/latest/serving/expert_parallel_deployment/. Accessed 2026-09-17. Cited in: Module 5, Module 11.
- standardInfiniBand Trade Association. IBTA Announces New RoCE Specification. n/a. https://www.infinibandta.org/ibta-announces-new-roce-specification/. Accessed 2026-09-17. Cited in: Module 1.
- paperBarham, P., Chowdhery, A., Dean, J., Ghemawat, S., Hand, S., Hurt, D., Isard, M., Lim, H., Pang, R., Roy, S., Saeta, B., Schuh, P., Sepassi, R., El Shafey, L., Thekkath, C. A., Wu, Y. Pathways: Asynchronous Distributed Dataflow for ML. MLSys 2022 / arXiv:2203.12533, 2022. https://arxiv.org/abs/2203.12533. Accessed 2026-09-17. Cited in: Module 18.
- paperDeepSeek-AI. DeepSeek-V3 Technical Report. arXiv:2412.19437, 2024-12-26. https://arxiv.org/pdf/2412.19437. Accessed 2026-09-17. Cited in: Module 6, Module 16.
- paperPatel, P., Choukse, E., Zhang, C., Shah, A., Goiri, I., Maleki, S., Bianchini, R. Splitwise: Efficient Generative LLM Inference Using Phase Splitting. ISCA 2024 / arXiv:2311.18677v2, 2023. https://arxiv.org/pdf/2311.18677. Accessed 2026-09-17. Cited in: Module 1, Module 3.
- paperQin, R., Li, Z., He, W., Zhang, M., Wu, Y., Zheng, W., Xu, X. Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving. USENIX FAST 2025 (Best Paper). arXiv:2407.00079v4. https://arxiv.org/pdf/2407.00079. Accessed 2026-09-17. Cited in: Module 3, Module 7, Module 8, Module 11, Module 15, Module 16.
- paperSun, B., Huang, Z., Zhao, H., Xiao, W., Zhang, X., Li, Y., Lin, W. Llumnix: Dynamic Scheduling for Large Language Model Serving. arXiv:2406.03243v1, USENIX OSDI 2024. https://arxiv.org/pdf/2406.03243. Accessed 2026-09-17. Cited in: Module 16.
- paperThe AIBrix Team. AIBrix: Towards Scalable, Cost-Effective Large Language Model Inference Infrastructure. arXiv:2504.03648, 2025. https://arxiv.org/abs/2504.03648. Accessed 2026-09-17. Cited in: Module 14.
- paperZhong, Y., Liu, S., Chen, J., Hu, J., Zhu, Y., Liu, X., Jin, X., Zhang, H. DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving. USENIX OSDI 2024. arXiv:2401.09670v3. https://arxiv.org/pdf/2401.09670. Accessed 2026-09-17. Cited in: Module 3, Module 11, Module 15, Module 16.
- repoAIBrix contributors. vllm-project/aibrix (README). v0.7.0 (2026-06-16). https://github.com/vllm-project/aibrix. Accessed 2026-09-17. Cited in: Module 14.
- repoDeepSeek-AI. Day 6 (One More Thing): DeepSeek-V3/R1 Inference System Overview. open-infra-index, #OpenSourceWeek, 2025-02. https://github.com/deepseek-ai/open-infra-index/blob/main/202502OpenSourceWeek/day_6_one_more_thing_deepseekV3R1_inference_system_overview.md. Accessed 2026-09-17. Cited in: Module 6, Module 15, Module 16.
- repoDeepSeek-AI. deepseek-ai/DeepEP (docs/legacy.md, V1 archived documentation). 2026. https://github.com/deepseek-ai/DeepEP/blob/main/docs/legacy.md. Accessed 2026-09-17. Cited in: Module 5.
- repoDeepSeek-AI. deepseek-ai/DeepEP (README, main branch, v2 kernels). 2026. https://github.com/deepseek-ai/DeepEP/blob/main/README.md. Accessed 2026-09-17. Cited in: Module 1, Module 5.
- repoDeepSeek-AI. deepseek-ai/DeepEP (README). GitHub, main branch. https://github.com/deepseek-ai/DeepEP. Accessed 2026-09-17. Cited in: Module 15.
- repoDeepSeek-AI. deepseek-ai/EPLB (README, main branch). 2026. https://github.com/deepseek-ai/EPLB/blob/main/README.md. Accessed 2026-09-17. Cited in: Module 5, Module 6, Module 16.
- repoGoogle (AI-Hypercomputer). JetStream (README, main branch). 2026. https://github.com/AI-Hypercomputer/JetStream/blob/main/README.md. Accessed 2026-09-17. Cited in: Module 18.
- repoHazyResearch contributors. ThunderKittens (GitHub repository README). Stanford University, Hazy Research Lab, main branch (v2.0, released 2026-01-11). https://github.com/HazyResearch/ThunderKittens. Accessed 2026-09-17. Cited in: Module 16.
- repoKubernetes SIG / Gateway API Inference Extension contributors. Endpoint Picker Protocol (004-endpoint-picker-protocol). Kubernetes SIGs, main branch proposal doc, 2026. https://github.com/kubernetes-sigs/gateway-api-inference-extension/tree/main/docs/proposals/004-endpoint-picker-protocol. Accessed 2026-09-17. Cited in: Module 10.
- repoKubernetes SIG / Gateway API Inference Extension contributors. EPP Architecture Proposal (0683-epp-architecture-proposal). Kubernetes SIGs, main branch proposal doc, 2026. https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/docs/proposals/0683-epp-architecture-proposal/README.md. Accessed 2026-09-17. Cited in: Module 2, Module 10.
- repoKubernetes SIG contributors. kubernetes-sigs/gateway-api-inference-extension (README). Kubernetes SIGs, GA since v1.0.0, 2026. https://github.com/kubernetes-sigs/gateway-api-inference-extension. Accessed 2026-09-17. Cited in: Module 2, Module 9, Module 10, Module 14.
- repoKubernetes SIG contributors. kubernetes-sigs/gateway-api-inference-extension Releases. Kubernetes SIGs, v1.6.1 (2026-09-11), 2026. https://github.com/kubernetes-sigs/gateway-api-inference-extension/releases. Accessed 2026-09-17. Cited in: Module 10.
- repoKubernetes SIG contributors. kubernetes-sigs/lws (LeaderWorkerSet README). Main branch. https://github.com/kubernetes-sigs/lws/blob/main/README.md. Accessed 2026-09-17. Cited in: Module 11, Module 14.
- repollm-d contributors. llm-d-inference-scheduler architecture.md. llm-d (CNCF Sandbox; Red Hat, Google Cloud, IBM Research, CoreWeave, NVIDIA), main branch, 2026. https://github.com/llm-d/llm-d-inference-scheduler/blob/main/docs/architecture.md. Accessed 2026-09-17. Cited in: Module 2, Module 10.
- repollm-d contributors. llm-d/llm-d (README). llm-d (CNCF Sandbox since 2026-03-12; Red Hat, Google Cloud, IBM Research, CoreWeave, NVIDIA), v0.7-v0.9, 2026. https://github.com/llm-d/llm-d. Accessed 2026-09-17. Cited in: Module 10.
- repollm-d contributors. llm-d/llm-d Releases. llm-d, v0.9.0 (2026-08-17), 2026. https://github.com/llm-d/llm-d/releases. Accessed 2026-09-17. Cited in: Module 10.
- repollm-d contributors. workload-autoscaling guide (llm-d/llm-d repository, main branch). https://github.com/llm-d/llm-d/blob/main/guides/workload-autoscaling/README.md. Accessed 2026-09-17. Cited in: Module 14.
- repoLMCache contributors. LMCache/LMCache (README). Main branch, 2026. https://github.com/LMCache/LMCache. Accessed 2026-09-17. Cited in: Module 8.
- repoMoonshot AI / MADSys contributors. kvcache-ai/Mooncake (README, main branch). 2026. https://github.com/kvcache-ai/Mooncake/blob/main/README.md. Accessed 2026-09-17. Cited in: Module 7, Module 8.
- repoNVIDIA / ai-dynamo contributors. ai-dynamo/dynamo (README). v1.4.2 container tags; v1.5.0-dev builds referenced, 2026. https://github.com/ai-dynamo/dynamo. Accessed 2026-09-17. Cited in: Module 9, Module 10, Module 12, Module 16.
- repoNVIDIA / ai-dynamo contributors. ai-dynamo/dynamo Releases. v1.4.2 (2026-08-29), v1.5.0-dev builds in progress. https://github.com/ai-dynamo/dynamo/releases. Accessed 2026-09-17. Cited in: Module 9, Module 16.
- repoNVIDIA / ai-dynamo contributors. ai-dynamo/grove (README). https://github.com/ai-dynamo/grove. Accessed 2026-09-17. Cited in: Module 9.
- repoNVIDIA / ai-dynamo contributors. ai-dynamo/nixl Releases. v1.4.1, 2026-09-01. https://github.com/ai-dynamo/nixl/releases. Accessed 2026-09-17. Cited in: Module 8, Module 16.
- repoopenucx/ucx. README.md. 2026. https://github.com/openucx/ucx/blob/master/README.md. Accessed 2026-09-17. Cited in: Module 1.
- repoROCm. mori (Modular RDMA Interface, README). 2026. https://github.com/ROCm/mori. Accessed 2026-09-17. Cited in: Module 17.
- repoROCm. rccl (README.md, develop branch). 2026. https://github.com/ROCm/rccl/blob/develop/README.md. Accessed 2026-09-17. Cited in: Module 17.
- reposgl-project/sglang GitHub. [Roadmap] Prefill-Decode Disaggregation Roadmap (2026 Q2). Issue #21703, opened 2026-03-30. https://github.com/sgl-project/sglang/issues/21703. Accessed 2026-09-17. Cited in: Module 12.
- reposgl-project/sglang GitHub. SGLang Router Architecture Improvement Proposal. Issue #7532, opened 2025-06-25. https://github.com/sgl-project/sglang/issues/7532. Accessed 2026-09-17. Cited in: Module 12.
- repoSGLang / RadixArk contributors. SGLang Model Gateway (README). sgl-project/sglang repository, main branch. https://github.com/sgl-project/sglang/blob/main/sgl-model-gateway/README.md. Accessed 2026-09-17. Cited in: Module 12.
- repoSGLang contributors. sglang-router (PyPI package page). SGLang / RadixArk, 0.3.2, 2026-01-15. https://pypi.org/project/sglang-router/. Accessed 2026-09-17. Cited in: Module 2, Module 12, Module 16.
- repovLLM Project contributors. vllm-project/production-stack (README). vLLM Project, released 2026-01-22. https://github.com/vllm-project/production-stack. Accessed 2026-09-17. Cited in: Module 2, Module 8, Module 11.
- repovLLM Project. tpu-inference (README, incl. live feature-support matrices). 2026. https://github.com/vllm-project/tpu-inference. Accessed 2026-09-17 (support-matrix snapshot dated 2026-08-27 in-repo). Cited in: Module 18.
- talkSmith, A., Alla, V. AMD Instinct MI300X Generative AI Accelerator and Platform Architecture. Hot Chips 2024, August 2024. https://hc2024.hotchips.org/assets/program/conference/day1/23_HC2024.AMD.MI300X.ASmith(MI300X).v1.Final.20240817.pdf. Accessed 2026-09-17. Cited in: Module 17.
- blogAMD ROCm Blogs. DI Series: Scaling GLM-5.1-FP8 to 64 MI300X GPUs. August 2026. https://rocm.blogs.amd.com/software-tools-optimization/di-glm-wideep/README.html. Accessed 2026-09-17. Cited in: Module 17.
- blogCNCF. Welcome llm-d to the CNCF: Evolving Kubernetes into SOTA AI Infrastructure. Cloud Native Computing Foundation, 2026-03-24. https://www.cncf.io/blog/2026/03/24/welcome-llm-d-to-the-cncf-evolving-kubernetes-into-sota-ai-infrastructure/. Accessed 2026-09-17. Cited in: Module 10.
- blogDatabricks Mosaic Research team. LLM Inference Performance Engineering: Best Practices. Databricks Blog, 2023-10-12. https://www.databricks.com/blog/llm-inference-performance-engineering-best-practices. Accessed 2026-09-17. Cited in: Module 15.
- blogGoogle Cloud. AI Hypercomputer inference updates for Google Cloud TPU and GPU. Google Cloud Blog, 2025. https://cloud.google.com/blog/products/compute/ai-hypercomputer-inference-updates-for-google-cloud-tpu-and-gpu. Accessed 2026-09-17. Cited in: Module 18.
- blogGoogle Cloud. Inside the Ironwood TPU codesigned AI stack. Google Cloud Blog, 2026. https://cloud.google.com/blog/products/compute/inside-the-ironwood-tpu-codesigned-ai-stack. Accessed 2026-09-17. Cited in: Module 18.
- blogHazy Research. ThunderMLA: FlashMLA, Faster and Fused-er! Stanford University, published 2025-03-04. https://hazyresearch.stanford.edu/blog/2025-03-04-thundermla. Accessed 2026-09-17. Cited in: Module 16.
- blogLMSYS Org / RadixArk. RadixArk Joins Forces with Google to Bring Full SGLang Features to TPUs. lmsys.org, 2026-07-30. https://www.lmsys.org/blog/2026-07-30-sglang-google-tpu/. Accessed 2026-09-17. Cited in: Module 12, Module 18.
- blogLMSYS Org. Deploying DeepSeek with PD Disaggregation and Large-Scale Expert Parallelism on 96 H100 GPUs. lmsys.org, 2025-05-05. https://www.lmsys.org/blog/2025-05-05-large-scale-ep/. Accessed 2026-09-17. Cited in: Module 6, Module 12, Module 15, Module 16.
- blogLMSYS Org. SGLang v0.4: Zero-Overhead Batch Scheduler, Cache-Aware Load Balancer, Faster Structured Outputs. lmsys.org, 2024-12-04. https://www.lmsys.org/blog/2024-12-04-sglang-v0-4/. Accessed 2026-09-17. Cited in: Module 12.
- blogLMSYS Org. Win on TCO: How AMD Instinct MI355X Achieves Cost-Competitive Distributed Inference Through SGLang with MoRI. May 28, 2026. https://www.lmsys.org/blog/2026-05-28-mori/. Accessed 2026-09-17. Cited in: Module 17.
- blogMarkthub, P., Dinan, J., Potluri, S., Howell, S. Improving Network Performance of HPC Systems Using NVIDIA Magnum IO NVSHMEM and GPUDirect Async. NVIDIA Developer Blog, 2022-11-22. https://developer.nvidia.com/blog/improving-network-performance-of-hpc-systems-using-nvidia-magnum-io-nvshmem-and-gpudirect-async. Accessed 2026-09-17. Cited in: Module 6.
- blogMeta Engineering. RoCE networks for distributed AI training at scale. Engineering at Meta, 2024. https://engineering.fb.com/2024/08/05/data-center-engineering/roce-network-distributed-ai-training-at-scale/. Accessed 2026-09-17. Cited in: Module 1.
- blogNVIDIA Developer Blog. NVIDIA NVLink and NVIDIA NVSwitch Supercharge Large Language Model Inference. 2026. https://developer.nvidia.com/blog/nvidia-nvlink-and-nvidia-nvswitch-supercharge-large-language-model-inference/. Accessed 2026-09-17. Cited in: Module 1.
- blogNVIDIA. Benchmarking GPUDirect RDMA on Modern Server Platforms. NVIDIA Developer Blog, 2014. https://developer.nvidia.com/blog/benchmarking-gpudirect-rdma-on-modern-server-platforms/. Accessed 2026-09-17. Cited in: Module 1.
- blogNVIDIA. How NVIDIA Dynamo 1.0 Powers Multi-Node Inference at Production Scale. NVIDIA Developer Blog. https://developer.nvidia.com/blog/nvidia-dynamo-1-production-ready/. Accessed 2026-09-17. Cited in: Module 9.
- blogNVIDIA. ModelExpress: Distributing Model Artifacts at the Speed of Light. NVIDIA Developer Blog. https://developer.nvidia.com/blog/modelexpress-distributing-model-artifacts-at-the-speed-of-light. Accessed 2026-09-17. Cited in: Module 9.
- blogNVIDIA. NVIDIA Dynamo 0.4 Delivers 4x Faster Performance, SLO-Based Autoscaling, and Real-Time Observability. NVIDIA Developer Blog, published 2025-08-13. https://developer.nvidia.com/blog/dynamo-0-4-delivers-4x-faster-performance-slo-based-autoscaling-and-real-time-observability. Accessed 2026-09-17. Cited in: Module 9.
- blogNVIDIA. NVIDIA Dynamo Accelerates llm-d Community Initiatives for Advancing Large-Scale Distributed Inference. NVIDIA Developer Blog, published 2025-05-21. https://developer.nvidia.com/blog/nvidia-dynamo-accelerates-llm-d-community-initiatives-for-advancing-large-scale-distributed-inference/. Accessed 2026-09-17. Cited in: Module 9.
- blogScaling Intelligence Lab (Hazy Research-adjacent group). Tokasaurus: An LLM Inference Engine for High-Throughput Workloads. Stanford University, published 2025. https://scalingintelligence.stanford.edu/blogs/tokasaurus/. Accessed 2026-09-17. Cited in: Module 16.
- blogvLLM Team. vLLM TPU: A New Unified Backend Supporting PyTorch and JAX on TPU. vLLM Blog, 2025-10-16. https://blog.vllm.ai/2025/10/16/vllm-tpu.html. Accessed 2026-09-17. Cited in: Module 18.
- analysisSemiAnalysis / InferenceX. AMD Instinct MI355X — Specs, Pricing & AI Inference Benchmarks. 2026. https://inferencex.semianalysis.com/chips/mi355x. Accessed 2026-09-17. Cited in: Module 17.
- analysisSemiAnalysis. InferenceMAX: Open Source Inference Benchmarking. InferenceMAX v1, 2026. https://newsletter.semianalysis.com/p/inferencemax-open-source-inference. Accessed 2026-09-17. Cited in: Module 3, Module 15, Module 16, Module 17.