Course · Computer Science

ML Systems Engineering

Production inference systems engineering — from fundamentals to specialized hardware deployment. Covers quantization, transformers, distributed inference, CPU/GPU/Apple Silicon/Edge AI, fine-tuning, voice AI, and RAG infrastructure.

39 modules10 parts5 appendices≈ 21 h reading

Part 1

Inference Optimization Fundamentals

Hardware-agnostic principles that apply everywhere

01The Inference Systems Mental Model25 min02Quantization — Complete Systems Perspective40 min03Sparsity — Structured & Unstructured35 min04Knowledge Distillation — Systems Perspective30 min05Graph Optimization & Compilation40 min
Part 2

Transformer Inference Architecture

The algorithms that make LLM inference fast

06Attention Mechanism — Systems Optimization45 min07KV-Cache Management Systems35 min08Inference Serving Systems Architecture40 min
Part 3

Distributed Inference

Multi-device, multi-node inference at scale

09Parallelism Strategies for Inference35 min10Communication Primitives & Collective Operations35 min
Part 4

Server-Grade CPU Inference

Intel Xeon & AMD EPYC inference optimization

11Why CPU Inference Matters & When It Wins25 min12CPU Inference Engine Internals40 min13GEMM Optimization for CPU Inference45 min14Attention Implementation for CPU35 min15Full LLM Inference Pipeline on CPU45 min
Part 5

Apple Silicon Inference

M-series unified memory architecture

16Apple Silicon Architecture for ML Engineers30 min17CoreML & ANE Programming30 min18Metal & GPU Inference on Apple Silicon30 min19llama.cpp on Apple Silicon30 min20Full Stack Inference on Apple Silicon30 min
Part 6

On-Device & Edge AI Inference

Mobile, embedded, microcontroller inference

21Edge AI Hardware Landscape30 min22TinyML — Inference on Microcontrollers25 min23Mobile Inference — Android & iOS30 min24Edge LLM Inference30 min
Part 7

NVIDIA GPU Inference

Modern GPU stack beyond Ampere

25Modern GPU Inference Stack35 min
Part 8

Fine-Tuning Systems

Infrastructure for adapting trained models

26Fine-Tuning Infrastructure35 min27LoRA & Parameter-Efficient Fine-Tuning30 min28RLHF & Alignment Systems35 min
Part 9

Fast Voice AI Systems

Streaming architecture, sub-100ms TTFA

29Voice AI Pipeline Architecture40 min30VAD (Voice Activity Detection) Systems25 min31Streaming ASR30 min32LLM Integration in Voice Pipeline30 min33TTS (Text-to-Speech) Systems Engineering30 min34Audio Processing & Codec Systems25 min35End-to-End Voice AI Optimization30 min
Part 10

RAG & AI Systems Infrastructure

Retrieval-augmented generation and production toolchain

36RAG Systems Architecture40 min37Deployment Toolchain30 min38Multimodal Inference30 min39Observability & Monitoring25 min

Appendices

AMLPerf Benchmarks ReferenceBModel Memory Sizing CalculatorCLatency Budget TemplatesDKey Paper Reading ListEHardware Comparison Matrix