Deep dive · Computer Science
AI Inference Hardware
A comparative deep dive into NVIDIA, AMD, Google TPU (v5-v7), and AWS Trainium/Inferentia chips for choosing, sizing, and serving models on AI inference hardware.
6 pages≈ 4.1 h readingSources
01NVIDIA Hopper, Blackwell, and the Rubin Roadmap: H100/H200/B200/GB200 NVL72 to Vera Rubin NVL7235 min02AMD Instinct CDNA3/CDNA4/CDNA5: MI300X to MI455X and the Helios Rack35 min03Google TPU v5e/v5p/v6e/v7: Pod Architecture from Trillium to Ironwood35 min04AWS Trainium and Inferentia: NeuronCore Architecture and Instance Topology35 min05Inference and Training Benchmarking: MLPerf and InferenceMAX Methodology, Batch-Scaling Mechanics50 min06Choosing Inference Hardware: Model-Size Fit, Cloud Pricing, and TCO Across Vendors55 min