Course · Computer Science

System Design: From Classical Internet Systems to AI Inference and Training Platforms

Classical system design from first principles, then the same discipline applied to AI inference platforms and AI training infrastructure.

39 modules5 parts1 appendix≈ 23 h readingSources

Part 1

Foundations of System Design

The request-path, storage, coordination, and reliability mechanisms every later case study and platform design assumes.

01The Request Path: DNS, Load Balancers, Reverse Proxies, CDNs, and Edge Networks35 min02Caching at Every Layer: Placement, Coherence, and Invalidation35 min03Replication and Partitioning: Making One Dataset Look Like Many Machines35 min04Transactions, Indexes, and the Relational/NoSQL Split35 min05Consistency Models: Linearizability, CAP, and PACELC35 min06Consensus and Coordination: Paxos, Raft, Leases, and ZooKeeper/etcd40 min07Distributed File and Object Storage: GFS, HDFS, S3, and Tectonic40 min08Messaging and Streams: Logs, Queues, Exactly-Once, and Idempotency35 min09Batch and Stream Processing: The MapReduce to Spark/Flink Lineage35 min10Services, APIs, and the Trust Boundary: Microservices, Service Mesh, Backpressure, Rate Limiting, and Zero Trust40 min11Reliability Engineering: SLOs, Observability, and Resilience Patterns35 min
Part 2

Classical Case Studies, A to Z

Eleven internet-scale products read against their own companies' published numbers and architecture, not interview folklore.

12Google Search and Gmail: Crawl to Index to Serve, and the GFS/Bigtable/Spanner Lineage40 min13YouTube and Vitess: Sharded MySQL at Video Scale, and What Isn't Public30 min14Netflix Video Infrastructure: Open Connect CDN and Per-Title Encoding30 min15Netflix Resilience and Personalization: Chaos Engineering and the Offline/Nearline/Online Split30 min16Twitch: Live Ingest, Low-Latency HLS, and Chat at Scale35 min17WhatsApp, Twitter, and Instagram: Messaging and Feed Architectures at Scale35 min18Stripe: Ledgers, Idempotency Keys, and API Design for Payments30 min19Uber: Dispatch, Geo-Indexing with H3, and Real-Time ETA35 min20Airbnb: Search Ranking, Payment Idempotency, and the Monolith-to-SOA Migration40 min
Part 3

AI Inference Platform Design: Client to GPU

The complete system between a client and a GPU backend -- gateway, identity, billing, routing, streaming, agents, caching, safety, data, tenancy -- given full treatment as the part Aman builds against most directly.

21Edge and API Gateway for AI Inference Platforms35 min22Identity, Rate Limits, and Quotas: API Keys, Orgs, and Token-Bucket Enforcement40 min23Billing and Metering: Ledger Design, Idempotency, Holds, and Reconciliation35 min24Request Routing, Admission Control, and SLO Tiers: Fallbacks and Graceful Degradation35 min25The GPU Backend Handoff: Designing the Interface to Multi-Node Serving30 min26Streaming, Cancellation, and Resumption: SSE, WebSockets, and Long-Running Requests35 min27Tool Use, Agent Harnesses, and Sandboxes for Code Execution40 min28Platform Prompt/Prefix Caching: Tiers, TTLs, and Billing Implications30 min29Safety and Moderation Pipelines: Pre/Post Filters, Classifiers, and Policy Routing35 min30Data and Observability: Conversation Storage, Retention, Privacy, and Per-Token Tracing30 min31Multi-Tenancy, Isolation, and Capacity/Cost Management35 min
Part 4

AI Training Infrastructure: Systems Level

Pre-training, post-training, and RL infrastructure as systems -- data, storage, scheduling, fault tolerance, and framework architecture -- with distributed-training internals deliberately excluded.

32Training Data Pipelines: Crawl/Ingest, Filtering/Dedup, Tokenization, and Sharded Datasets30 min33Storage and Checkpointing at Scale: Cluster Network Topology35 min34Cluster Scheduling and Orchestration: Slurm, the Borg/Kubernetes Lineage, Ray, and MAST35 min35Fault Tolerance and Elasticity: Checkpoint/Restart, Stragglers, Node Health, and Incident Patterns30 min36Post-Training Pipelines: SFT/Preference Data Flows, Evaluation Gates, and Model Registries35 min37RL Training Infrastructure: Environments as Services, Rollout Generation, and Trainer/Rollout Decoupling35 min38Comparing the Open-Source RL Frameworks: verl, OpenRLHF, SkyRL, AReaL, slime, and NeMo-RL40 min
Part 5

Frontier and Synthesis

What remains genuinely unsettled across classical systems, AI inference platforms, and AI training infrastructure, as of this course's as-of date.

39Where the Field Still Disagrees and What Is Changing: Consistency, Multi-Tenancy, Async RL, and the Next Edition30 min

Appendices

ABibliography