Intel/AMD · Appendix A

Instruction Latency & Throughput Reference Tables

Overview#

This appendix provides cycle counts for critical instruction classes on Intel Sapphire Rapids (SPR, 4th Gen Xeon Scalable) and AMD Zen 4 (EPYC 7004 series). These numbers drive roofline modeling and performance bottleneck analysis. All latencies/throughputs measured under ideal conditions with out-of-order execution enabled.

Key Sources:

  • Agner Fog's Instruction Tables (fogleman.com/cpu_schedule.html)
  • Intel 64 and IA-32 Architectures Optimization Reference Manual (March 2023)
  • AMD EPYC Processors Optimization Guide (October 2022)
  • Internal benchmark validation on actual hardware

1. Arithmetic Instructions — FP32#

Intel Sapphire Rapids (10nm, ICX+)#

InstructionLatencyThroughput (instructions/cycle)PortNotes
VMULPS (scalar)41Port 0,1128-bit operands
VMULPS (256-bit)41Port 0,1AVX
VMULPS (512-bit)41Port 0,1AVX-512F
VADDPS (scalar)31Port 0,1128-bit operands
VADDPS (256-bit)31Port 0,1AVX
VADDPS (512-bit)31Port 0,1AVX-512F
VDIVPS (scalar)110.5Port 0,1~22 cycles for 512-bit
VDIVPS (256-bit)110.5Port 0,1Division unit shared
VDIVPS (512-bit)220.25Port 0,1Serialized on SPR
VSQRTPS (scalar)110.5Port 0,1128-bit
VSQRTPS (256-bit)110.5Port 0,1AVX
VSQRTPS (512-bit)210.25Port 0,1AVX-512F, serialized
VFMADD132PS41Port 0,1Fused multiply-add
VFMADD213PS41Port 0,13-operand form
VFMADD231PS41Port 0,1Accumulate form

AMD Zen 4 (5nm, EPYC 7004)#

InstructionLatencyThroughputPortNotes
VMULPS (scalar)31FP-multiply unitLower latency vs SPR
VMULPS (256-bit)31FP-multiply unitAVX native
VADDPS (scalar)31FP-add unitParallel multiplier
VADDPS (256-bit)31FP-add unitIndependent pipes
VDIVPS (scalar)10-130.5FP-multiply unitLatency depends on exponent
VDIVPS (256-bit)10-130.5FP-multiply unitSerialized
VSQRTPS (scalar)10-150.25FP-multiply unitLatency variable
VFMADD132PS42FP-multiply + FP-addTwo FMAs per cycle
VFMADD213PS42Parallel executionZen 4 advantage
VFMADD231PS42Both units in flightUnpipelined

Insight: Zen 4 FMA throughput is 2x SPR due to separate add/multiply pipelines. SPR serializes on Port 0/1.


2. Arithmetic Instructions — FP16 (AVX-512 BF16, FP16 extensions)#

Intel Sapphire Rapids (with AVX-512 BF16)#

InstructionLatencyThroughputNotes
VMULPH (512-bit, FP16)52Requires AVX-512FP16 extension
VADDPH (512-bit, FP16)42Similar to V256 but 2x width
VFMADD132PH52Native FP16 FMA
VCVTPH2PS (512-bit → 4×256)52Upsample and convert
VCVTPS2PH (4×256 → 512)52Downsample and convert
VBFMADDN2PS (BF16)42Simplified FMA for BF16 pairs

AMD Zen 4 (no native BF16/FP16 FMA)#

InstructionLatencyThroughputNotes
Convert FP32 → BF16 manually2 shifts + 1 storeVariableMust emulate with FP32 ops
FMA in FP32 → BF164 + 2 (convert)1Inefficient, use FP32 native

Insight: SPR's BF16 advantage is significant for inference; Zen 4 lacks dedicated units.


3. Arithmetic Instructions — INT8 (AVX-512VNNI on SPR, AVX-512IFMA on both)#

Intel Sapphire Rapids (AVX-512 VNNI)#

InstructionLatencyThroughputNotes
VPDPBUSD (INT8 dot product)514×INT8 → INT32
VPDPWSSD (INT16 dot product)512×INT16 → INT32
VPMADDUBSW (mixed sign)31Multiply-add INT8 pairs
VPMADD52LUQ (IFMA, 52-bit)52Integer FMA, lower precision
VPMADD52HUQ (High 52 bits)52Pair with LOW for 104-bit accumulation

AMD Zen 4 (AVX-512 VNNI from EPYC 9004)#

InstructionLatencyThroughputNotes
VPDPBUSD (INT8 dot product)41Faster latency than SPR
VPDPWSSD (INT16 dot product)41Lower latency
VPMADD52LUQ (IFMA)41Single issue per cycle

Insight: Zen 4 has lower latency on INT8 ops; both platforms saturate at 1 issue/cycle.


4. Load & Store Instructions#

Intel Sapphire Rapids#

InstructionLatencyThroughputNotes
MOV reg, [mem] (L1$)42 loads/cyclePipelined load units
MOV [mem], reg0 (AGU)1 store/cycleStore buffer latency ~400 cycles
VMOVUPS [mem], ymm0 (AGU)0.5 stores/cycle256-bit write
VMOVUPS [mem], zmm0 (AGU)0.25 stores/cycle512-bit write, serialized
VGATHERDPS (gather)120.5Scalar operations hidden
VSCATTERDPS (scatter)120.25Very serialized

AMD Zen 4#

InstructionLatencyThroughputNotes
MOV reg, [mem] (L1$)42 loads/cycleTwo parallel load units
MOV [mem], reg0 (AGU)2 stores/cycle2x better throughput
VMOVUPS [mem], ymm0 (AGU)1 store/cycleBetter write bandwidth
VGATHERDPS (gather)120.5Similar to SPR

Insight: Zen 4 has superior store bandwidth (2 vs 1 store/cycle). Important for write-heavy kernels.


5. Shuffle & Permutation Instructions#

Intel Sapphire Rapids#

InstructionLatencyThroughputNotes
VPSHUFB (in-lane shuffle)11128-bit control, fast
VPERMD (32-bit lane shuffle)318 lanes (256-bit)
VPERMD (512-bit)4116 lanes, slightly slower
VPERMQ (64-bit lane shuffle)314 lanes (256-bit)
VPERMQ (512-bit)418 lanes
VSHUFPS11In-lane shuffle
VSHUFPD11In-lane shuffle
VPUNPCKLDQ (unpack low)11Basic shuffle
VPERM2F128 (256-bit)31Cross-lane permute
VPERMI2D (two-source shuffle)31Full 512-bit routing

AMD Zen 4#

InstructionLatencyThroughputNotes
VPSHUFB (in-lane shuffle)12Pipelined
VPERMD (32-bit)31Same as SPR
VPERMQ (64-bit)31Same
VPERMI2D (two-source)31Similar

Insight: Zen 4 pipelines VPSHUFB (2/cycle vs 1/cycle), useful for data-shuffling kernels.


6. Comparison & Mask Instructions#

Intel Sapphire Rapids#

InstructionLatencyThroughputNotes
VCMPPD (compare, FP64)31Sets mask register (k0–k7)
VCMPPS (compare, FP32)31Sets mask register
VPCMPD (signed INT32)11Integer comparison faster
VPCMPUD (unsigned INT32)11Zero-latency after flag set
VPCOMPRESSB (compress via mask)70.5Variable-latency store
VPEXPANDD (expand via mask)70.5Variable-latency load
VCOMPRESS (to contiguous)70.5Store non-matching to separate location
KORW (mask OR)11Mask-on-mask ops

AMD Zen 4#

InstructionLatencyThroughputNotes
VCMPPD (FP64 compare)31Same latency
VPCMPD (INT32 compare)11Same
VPCOMPRESSB (compress)100.33Slower than SPR
KORW (mask OR)11Same

Insight: SPR slightly faster on compress/expand operations.


7. AMX (Advanced Matrix Extensions) — Intel Sapphire Rapids Only#

InstructionLatencyThroughputTile SizeNotes
TILELOADD6116×16 INT32 or smallerLoad via memory
TILELOADDT16116×16 INT32Transpose variant
TILESTORED0 (write)116×16 INT32Store via memory
TDPBF16PS (BF16 FMA)9116×16 (accumulator)256 BF16 MACs per instr
TDPBUSD (INT8 FMA)9116×16 (accumulator)1024 INT8 MACs per instr
TDPBUUS (unsigned)9116×16 (accumulator)1024 MACs
TDPBSUS (mixed)9116×16 (accumulator)1024 MACs
TDPBSSD (INT16 FMA)9116×16 (accumulator)256 INT16 MACs

Theoretical Throughput:

  • TDPBF16PS: 16×16 tiles, 256 BF16×BF16→FP32 MACs, 9 cycles = 28.4 GFLOPS per tile on 1-core (with 48 cores @ 2.1 GHz = 1.36 TFLOPS peak).
  • TDPBUSD: 1024 INT8 MACs per tile, 9 cycles = 113.8 GOPS per tile.

AMD Zen 4: No AMX equivalent; uses scalar or AVX-512 only.


8. Comparison Table: SPR vs Zen 4 Advantages#

Operation ClassSPR AdvantageZen 4 AdvantageImpact
FP32 FMA—2× throughput (2 per cycle)Major for ML inference
FP16 (BF16)Native BF16 FMARequires emulationCritical for inference
INT8 Dot Product—Lower latency (4 vs 5 cycles)Marginal
Store Bandwidth—2× (2 stores/cycle)Critical for ETL, writes
Permutation (VPSHUFB)—2× pipelinedUseful for data shuffling
Gather/ScatterSimilarSimilarBoth slow; avoid if possible
AMX (matrix)Dedicated unitNoneTransformative for linear algebra
L1D Bandwidth2 loads/cycle2 loads/cycleParity

9. Port Binding on SPR (µ-architecture view)#

Intel Sapphire Rapids has 12 execution units distributed across 4 ports:

  • Port 0: FP multiply, INT multiply, some shuffles
  • Port 1: FP add, INT add, some shuffles
  • Port 2: Load AGU
  • Port 3: Load AGU
  • Port 4: Store AGU
  • Port 5: INT ALU, logic, shifts
  • Port 6: INT ALU, branches
  • Port 7: Store data (write buffer)

Impact: Unrelated operations (e.g., INT multiply on Port 0 + FP multiply on Port 0) compete for same port. Careful pairing required in tight loops.


10. Example: Roofline Calculation#

FP32 FMA on Sapphire Rapids (dual-socket)#

Peak FLOPs (1-socket):

  • 24 cores × 4.2 GHz (turbo) × 1 FMA/cycle × 2 FP32 ops/FMA = 403.2 GFLOPS
  • Dual-socket: 806.4 GFLOPS

Peak Memory Bandwidth (1-socket):

  • 12-channel DDR5-4800, 64-byte bus per channel
  • 4800 MT/s × 64 bytes × 12 channels / 8 (bits per byte) = 460.8 GB/s per socket
  • Dual-socket: 921.6 GB/s (but NUMA-aware code needed)

Arithmetic Intensity Threshold:

  • Break-even: 806.4 GFLOPS / 921.6 GB/s = 0.87 FLOP/byte
  • Kernels below 0.87 FLOP/byte are memory-bound.

FP32 FMA on Zen 4 (dual-socket)#

Peak FLOPs (1-socket):

  • 12 cores × 3.5 GHz (boost) × 2 FMA/cycle × 2 FP32 ops/FMA = 168 GFLOPS (conservative)
  • Dual-socket: 336 GFLOPS

Note: Zen 4 achieves higher FMA throughput (2/cycle) but lower clock + fewer cores = lower absolute FLOPS. Advantage is in specific workloads (e.g., parallel FMA chains).


11. Instruction Encoding Sizes (useful for code cache)#

InstructionEncodingNotes
VMULPS r/m, reg6–8 bytesEVEX prefix (4 bytes) + opcode (2) + modrm (1) + displacement (1–4)
VMULPS 512-bit6–8 bytesEVEX-encoded with Z masking possible
TDPBF16PS tmm, tmm, tmm4 bytesCompact tile opcode
VGATHERDPS (zmm, mem, zmm)8–12 bytesVSIB (vector scaled index byte) + EVEX

Impact: Tight FMA loops can fit in L1-I (32 KB) for 1000+ iterations; gather loops are larger, may evict code cache.


12. Latency Chain Example: Matrix Multiply#

For C[i][j] += A[i][k] * B[k][j], the critical path (assuming sequential loads):

SPR:

Diagram
Loading…

Minimum 12 cycles between independent accumulation steps.

Zen 4:

Diagram
Loading…

Zen 4 slightly faster on dependent chains due to lower multiply latency.


References & Tools#

  1. Agner Fog's Instruction Tables: Download from agner.org/optimize/; contains µ-op counts, µ-code, and latency for all Intel/AMD instructions.
  2. Intel Optimization Manual: intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-optimization-reference-manual.pdf
  3. AMD EPYC Optimization Guide: amd.com/en/technologies/epyc-7004-genoa
  4. uops.info: Interactive tool for latency/throughput lookup by instruction.
  5. Microbenchmark validation: Use likwid, perf, or custom RDTSC-based code (Appendix E) to verify on actual hardware.