System Design · Module 14

Netflix Video Infrastructure: Open Connect CDN and Per-Title Encoding

Modules 1 and 2 already used Netflix's Open Connect as the counter-example to a general-purpose CDN — one tenant, one content type, purpose-built appliances. This module opens the appliance up: the specific hardware and software bottlenecks that had to be found and removed one at a time to take a single box from 10 Gbps to several hundred, the reason TLS encryption was the single hardest slice of bandwidth to add, and the encoding-side decision — one fixed bitrate ladder versus a ladder tuned per title or per shot — that determines how many bytes ever need to leave an appliance in the first place.

Course 30 min read 9 sources As of 2026-09-17 Prerequisite: Modules 1, 2
A Netflix playback request: client asks the control plane which appliance to use, the control plane picks one from telemetry the appliance already reported, and the appliance itself serves from memory or NVMe depending on the title's popularity Client TV / phone / browser Netflix control plane — AWS Playback apps Steering service (CODA) Cache Control Service (CCS) Open Connect Appliance Memory hottest 10–20% of titles NVMe remaining catalog 2 play request 4 URL(s) for the chosen OCA 1 OCA continuously reports health, routes, file inventory 5 GET segment over HTTPS 6 serve bytes from memory or NVMe
Figure 1. The six-step playback path Netflix's own Open Connect documentation describes: an appliance reports its health and file inventory to the Cache Control Service; a client's play request reaches the playback application, which asks the Steering service (internally called CODA) which appliance to use; the steering service returns a URL for the chosen appliance; the client fetches segments directly from that appliance, which serves them from its memory tier if the title is one of the 10–20% hottest, or from NVMe otherwise1. The rest of this module is about what happens inside the rightmost box, and how many bytes it has to move.

1. Why Netflix built its own CDN

Netflix started the Open Connect program in 2011, and its own account of why is direct about the two reasons: Netflix had grown into a large enough share of traffic on residential ISP networks that it needed a direct, collaborative relationship with those ISPs rather than an arm's-length one through a third-party CDN, and building a content-delivery system customized for exactly one workload — video, from one catalog, with known popularity statistics in advance — let Netflix run a proactive, directed caching system instead of the demand-driven caching a general CDN runs, which Netflix states reduces the overall demand placed on upstream network capacity "by several orders of magnitude"1. Module 1 already covers what a general-purpose, multi-tenant CDN's edge tier looks like and why Open Connect's single-tenant design lets it tune every layer for one workload where a shared platform cannot; this section does not repeat that argument, only the mechanics of how Open Connect is actually built and controlled.

An Open Connect Appliance (OCA) is deliberately narrow in what it does. It stores encoded video and image files and serves them over HTTP/HTTPS to whatever client device asks for them; it does not store any per-customer state — no viewing history, no DRM keys, no account data1. Netflix deploys OCAs two ways, and the difference is entirely about who owns the last-mile network the bytes cross:

  • Embedded, inside an ISP's own network. Netflix ships the server hardware to a qualifying ISP at no charge; the ISP supplies power, rack space, and connectivity, and decides which of its own customers get routed to that appliance. This is the deployment that turns a residential ISP's expensive upstream transit link into a link that mostly does not need to carry Netflix traffic at all, because the bytes never leave the ISP's own network.
  • At internet exchange points (IXPs), via settlement-free interconnect (SFI). Netflix installs OCAs at major exchange points in significant markets and peers with any ISP present at that exchange, at no cost to either side. This is the deployment for markets or ISPs too small, individually, to justify shipping embedded hardware.

Both deployment types run the identical appliance software, and both funnel their operational telemetry back to the same control plane in AWS1. That control plane is split into two named services worth knowing by name because Figure 1 uses them: the Cache Control Service (CCS) ingests each OCA's self-reported health, BGP-learned routes, and file inventory, and the Steering service (Netflix's internal name for it is CODA) uses that state, plus the requesting client's network proximity, to decide which specific appliance a given playback request should be handed a URL for1. Neither service ever routes bytes itself — once the client has a URL, the video bytes flow appliance-to-client directly, and the control plane's job is finished for that request until the next one.

An OCA reports back only two kinds of information, by design: its own health/routes/file inventory to the control plane, and the bytes it serves to whichever client asked for them1. Netflix's stated operational model treats a deployed appliance almost as a consumable: its Open Connect Operations team monitors every deployment continuously, "proactively fix[es] most issues remotely with minimal input required from our ISP partners," and if a server degrades below Netflix's own quality bar, "we simply replace it — at no cost to our partners"1. Across a global network Netflix describes as "thousands of OCAs" deployed across both the embedded and IXP models1, that operational posture — Netflix owns the hardware lifecycle end to end, regardless of whose building it sits in — is what makes the appliance-level tuning in the rest of this module possible: nobody has to negotiate with an ISP to change a kernel parameter, because the ISP never touches the software at all.

Appliance hardware itself is not static; Netflix states it is "involved in the development of all layers of the software stack" and publishes both hardware design and software as open source1, and the two current appliance families it documents publicly trade capacity for throughput in opposite directions:

Table 1. The two Open Connect Appliance families Netflix documents publicly, as of this course's as-of date2.
Appliance familyRaw storageNetworkOperational throughputPeak power
Storageup to 120 TB6×10 Gbps LAG or up to 2×100 Gbps≈200 Gbps≈400 W
Globalup to 60 TB10 Gbps and 100 Gbps connectivity≈80 Gbps≈250 W

The Storage family holds twice the catalog per box and pushes roughly twice the throughput; the Global family trades capacity for a lower power and footprint profile, which matters for ISP sites that cannot host a large rack. Both families ship with Mellanox and Broadcom network controllers2 — the same Mellanox lineage (ConnectX-6 Dx) that Section 3 shows carrying hardware TLS offload once encryption itself becomes the bottleneck being removed. Neither number in this table is the one this module spends most of its time on — that number is what a single appliance's throughput ceiling was, how it moved, and why, which is a story about CPU, PCIe, and TLS rather than about disk capacity.

Module 13 covers YouTube's own public disclosure and finds it deliberately narrow: Google has published Vitess, the MySQL-sharding layer underneath YouTube's databases, in real detail, but states nothing comparable about YouTube's transcoding pipeline or CDN. Netflix's Open Connect sits at the opposite end of that same spectrum — a company-run CDN with its hardware specifications, its FreeBSD kernel contributions, and its encoding research all published as primary engineering material, which is the only reason this module can go as deep into mechanism as it does. That contrast is itself informative: it says more about each company's disclosure culture than about which CDN is better engineered.

2. Pushing one appliance from 10 Gbps to 100+ Gbps

Open Connect's proof-of-concept, in 2012, ran on a single 10 Gbit/s network interface, vanilla FreeBSD 9.0-RELEASE, and stock nginx3. Every number in this section is the story of what had to change, in hardware and in the FreeBSD kernel, to turn that single 10 Gbps box into a fleet that a Netflix FreeBSD engineer could describe, a decade later, as "one of the biggest sources of traffic on the internet — sending terabits per second, powered by thousands of servers or appliances, all running FreeBSD"3. The reason this is worth a dedicated mechanism section rather than a single headline number is that the ceiling did not move once — it moved in a sequence of distinct bottlenecks, each one found and removed before the next one became visible, and Netflix has been explicit about why the effort was worth it at every step: "when deploying a CDN at a global scale, even a single percentage point increase in performance results in savings worth hundreds of thousands of dollars"3. At Netflix's fleet size, a bottleneck that costs 5% of throughput is not a rounding error to be tolerated until the next hardware refresh — it is a standing invoice, which is the reason the same team kept finding and removing bottlenecks for a decade instead of stopping once "good enough" was reached.

By the time Netflix published its own account of the climb to 100 Gbps in September 2017, the sequence looked like this. A single-socket Intel Xeon E5-2697v2 system was CPU-limited at 40 Gbps: the CPU ran out of cycles to move and encrypt bytes before the network or the storage did. Removing that ceiling exposed the next one — a PCIe Gen3 x8 slot limited the same box to 52 Gbps, because the bus between the NIC and the rest of the system, not the CPU, was now the tightest link. Moving to updated hardware with a wider PCIe path and a newer CPU generation pushed throughput into the 70 Gbps range on prototype systems, and eventually to "well over 90 Gbps for the first time." The last number in the sequence is the one that matters most for the rest of this section: Netflix reached the point where it could "serve 100% TLS traffic comfortably at 90 Gbps," en route to its stated 100 Gbps target4.

Bar chart of single-appliance throughput at each hardware/software generation from 2016 to 2017, labeled with the bottleneck that limited it 0 40 80 120 Gbps served by one appliance 40 CPU-limited single-socket Xeon E5-2697v2 52 bus-limited: PCIe Gen3 x8 to the NIC ≈80 wider PCIe, newer CPU generation 90+ exceeded 90 Gbps for the first time 100 stated goal, reached: 90 Gbps of that already under 100% TLS traffic
Figure 2. A single appliance's measured throughput ceiling across five successive hardware and software generations, each bar labeled with the bottleneck that had to be removed to reach the next one: CPU, then the PCIe bus, then a newer CPU/PCIe generation, then TLS handling itself. Netflix's account states 100 Gbps from one FreeBSD-based appliance as the stated goal and eventual result, with 90 of those Gbps already demonstrated under fully-encrypted traffic4. Section 3 covers the mechanism behind that last bar.

That last transition — from "over 90 Gbps" to "100% TLS traffic comfortably at 90 Gbps" — is easy to read as a footnote, but it is the one this module treats as the main event, because encryption is not a fixed tax that scales linearly with the rest of the pipeline the way disk I/O or network transfer roughly do. Section 3 works through why, and what specifically had to change in FreeBSD's kernel to make it affordable. The short version, confirmed by a Netflix FreeBSD engineer describing later work at the FreeBSD Vendor Summit in November 2023: "we had the first 100 gigabit per second production CDN server in the world... due to Kernel TLS"3 — and by 2023, the same lineage of kernel-TLS work plus NIC-based TLS offload had a single appliance sustaining roughly 375 Gbps at about 53% CPU utilization, on a machine budgeted for 400 Gb/s of raw serving capacity3. The publicly documented Storage-family appliance in Table 1, at roughly 200 Gbps, sits between the 2017 milestone and that 2023 demonstration — a reminder that every number in this section is a snapshot of hardware and kernel work that has kept moving well past the point any single blog post captured it.

3. Why encryption was the hard part: kernel TLS

Until 2015, the video and audio bytes an OCA served were not individually encrypted per playback session — the objects themselves carry Digital Rights Management encryption baked in at encode time, which protects the content, but the transport between an appliance and a client was not separately wrapped in TLS5. Netflix began rolling out per-session TLS for playback traffic in 2015, expecting the majority of streaming sessions to be TLS-encrypted by the end of 20165. That rollout is what turned "serve N Gbps" into "serve N Gbps while touching every byte with a cipher," and the OCA generation those numbers apply to — an Intel 64-bit Xeon CPU appliance running FreeBSD 10.2 and Nginx 1.9, holding 10–200 TB of objects, sized for 10,000–40,000 simultaneous long-lived TCP sessions, and designed for 10–40 Gbps of continuous bandwidth5 — is the same generation Figure 2's CPU-limited 40 Gbps bar describes. Encryption did not add a fixed percentage on top of that number; it took over the same CPU cycles the appliance needed for everything else, which is why it shows up in Figure 2 as its own distinct bottleneck rather than as background cost baked into the "40 Gbps" bar.

The first lever was cipher choice. AES-CBC, the traditional TLS bulk cipher, needs the plaintext processed twice — once to produce the ciphertext, once to compute the SHA hash that authenticates it. AES-GCM needs the data touched once: it is a combined encrypt-and-authenticate mode, decryption is cheap, and by TLS 1.2 it is supported everywhere that matters. Netflix moved its whole platform to prefer GCM and fall back to CBC only for clients that could not be upgraded5 — a straight halving of the cipher's own per-byte CPU cost before anything else changed.

The second lever was which implementation of GCM to run. Stock OpenSSL 1.0.1 in early 2015 had no AES-NI-optimized GCM path at all; Netflix first moved to BoringSSL's hand-optimized AES-NI implementation, then in mid-2015 to Intel's Intelligent Storage Acceleration Library (ISA-L), which is tuned per Intel CPU model family and outperformed BoringSSL in Netflix's own testing5 — at the price of a build-system friction cost that is easy to forget once the performance number is all anyone quotes later: ISA-L was written in the YASM assembler dialect, which the FreeBSD kernel's GCC/LLVM toolchain could not call directly, and Netflix had to patch the kernel build makefiles to invoke YASM as an external assembler just to link it in5. Neither of these levers, on its own, was the biggest one. The biggest one was moving the cipher out of user space entirely.

Running TLS in nginx, in user space, means every encrypted byte gets copied from the kernel's page cache into a user-space buffer, encrypted there, and copied back into the kernel to be sent — on top of whatever copies the AES-NI implementation itself required because it could only encrypt in place. Netflix's own profiling of that path found three separate, avoidable costs: the encrypt-in-place requirement of their AES-NI code forced extra copies; nginx's calls into the TLS code were not passing the correct flags to sendfile(2), so hot content was not staying cached the way it should; and encryption code kept walking a linked list of kernel mbuf structures to find the memory it needed to encrypt, polluting the CPU cache on every walk5. The fix for all three was the same idea applied three times: stop treating encryption as a detour the data takes on its way through the kernel, and make the kernel itself the place where the cipher runs. Netflix built an iovec array of pointers to encrypt from and to, filled in once during sendfile's own setup instead of re-derived from an mbuf chain on every pass, and redesigned mbuf allocation so that a page and its mbuf wrapper were allocated as a single unit with an embedded reference count, cutting a three-allocation path down to one5.

This is kernel TLS (kTLS): moving the bulk cipher itself into the kernel so it runs directly against the pages sendfile is already moving, instead of asking user space to copy those pages out, encrypt them, and hand them back. Scott Long first proposed the idea at Netflix; Randall Stewart and Long built the foundational software-kTLS transmission path together, and Drew Gallatin added the specific kernel machinery — external-page mbufs and M_NOTREADY mbufs — needed to hold encrypted data safely inside the kernel, plus a pluggable interface so different cipher backends could sit behind the same kTLS path3. Measured against the machine each configuration ran on, moving the cipher into the kernel did not just add throughput — on the newest hardware it changed which resource was the bottleneck at all:

Table 2. Three OCA hardware revisions, each run with encryption in user-space OpenSSL versus in the kernel (sendfile + kTLS) with an AES-NI-optimized cipher, measured against live production traffic during a busy hour5.
RevisionCPU / storageCipher pathCPU utilizationBandwidth
Rev HE5-2650Lv2, HDDuser-space OpenSSL60–65%12–12.5 Gbps
Rev H(same box)kernel sendfile + AES-NI55–70%15–16.5 Gbps
Rev FE5-2697v2, SSDuser-space OpenSSL80% (capped)22–23 Gbps
Rev F(same box)kernel sendfile + AES-NI80% (capped)25–25.5 Gbps
Rev NE5-2697v3, SSDuser-space OpenSSL70–75%, SSD-bound29–30 Gbps
Rev N(same box)kernel sendfile + AES-NI≈53%, NIC-bound35.5–36 Gbps

Rev N is the row worth pausing on. In user space, that box could not even reach its own 80% CPU target — its SSDs ran out of I/O capacity first, at 29–30 Gbps. Once encryption moved into the kernel, the SSD bottleneck disappeared from view (kTLS did nothing to the disks), CPU utilization dropped to about 53%, and the new limit was the network interface itself, at 35.5–36 Gbps. Moving a cost out of user space did not just make that cost cheaper — it revealed that the disk had been the binding constraint the whole time, and once the cipher stopped competing with disk I/O for CPU attention, the box could finally push enough bytes to find its actual ceiling.

The FreeBSD kernel-TLS lineage did not stop at software. For FreeBSD 13, Gallatin implemented TLS transmit offload to the network interface card itself, in partnership with Chelsio for its T6 adapters, with Hans Petter Selasky later extending the same NIC-offload path to Mellanox ConnectX-6 Dx cards3 — moving the cipher a second time, from the kernel's CPU-executed AES-NI instructions to dedicated crypto silicon on the NIC. That combination is what a Netflix FreeBSD engineer later credited for "the first 100 gigabit per second production CDN server in the world"3, and by late 2023 the same lineage was demonstrated sustaining roughly 375 Gbps at about 53% CPU utilization on a machine provisioned for 400 Gb/s of raw throughput3.

That 400 Gb/s figure is worth turning into a memory-bandwidth budget, because it is the same roofline reasoning that governs whether a GPU kernel is memory-bound: 400 Gb/s of network throughput is 50 GB/s of bytes per second, and a single byte on the serve path is not touched once. Netflix's own accounting of the 400 Gb/s design assigns roughly 50 GB/s to each of four hops — disk to memory, memory to the CPU for encryption, the CPU writing the encrypted result back to memory, and memory out to the network card — for a total of about 200 GB/s of internal memory bandwidth consumed to sustain 50 GB/s of externally visible throughput, over roughly 64 PCIe Gen4 lanes3:

$$ \text{memory bandwidth needed} \approx 4 \times (\text{external throughput}) = 4 \times 50\ \text{GB/s} = 200\ \text{GB/s} $$

The factor of four is the direct consequence of encryption forcing a read-then-write round trip through the CPU on top of the read-then-write round trip storage-to-NIC transfer already needed: without any cipher at all, the same bytes would only need to move disk→memory→NIC, a factor of two, not four. Kernel TLS's entire value, in this accounting, is refusing to add a fifth and sixth hop for a user-space copy in and back out — it keeps the multiplier at four instead of six.

4. The memory-vs-NVMe trade-off, and why Netflix does not just use more memory

Section 3 established that an OCA's throughput ceiling, once TLS is accounted for, is set by whichever of CPU, PCIe, storage I/O, or the NIC runs out first. Storage is where a result that looks backwards on first read belongs: Netflix measured that serving everything from memory, rather than from NVMe storage, made the same appliance slower. "The performance actually dropped from being CPU limited at 40 Gbps to being CPU limited at only 22 Gbps" when Netflix tried an all-memory configuration4 — the same box, the same CPU, roughly half the throughput, with the only change being where the bytes were read from before being sent.

Module 2, section 5, already covers this exact result as a caching-policy question — how much of a fast tier's budget to spend, and on what — and derives the general lesson that "put everything in the fastest medium" can lose to "put the popular fraction in the fastest medium" once a fixed per-byte CPU cost dominates over medium latency. This module adds the piece that actually makes that policy executable at Netflix's scale: Netflix does not discover popularity reactively the way an LRU cache does, because it does not have to. It knows, in advance, from its own catalog's viewing statistics, that 10–20% of the most popular titles account for the overwhelming majority of requests4, so the placement decision is made ahead of playback, not learned from it.

Same appliance, two storage configurations: all-memory serving is CPU-limited at 22 Gbps, while NVMe with a memory-resident popular fraction is CPU-limited at 40 Gbps 0 20 40 Gbps, CPU-limited 22 all-memory (100% of catalog in RAM) 40 memory + NVMe (hottest 10–20% in RAM, rest on NVMe) +82%
Figure 3. On identical hardware, serving the entire catalog from memory is CPU-limited at 22 Gbps; serving the hottest 10–20% of titles from memory and the rest from NVMe is CPU-limited at 40 Gbps — an 82% throughput increase from serving less of the catalog out of the supposedly faster medium4.
Schematic content-popularity long tail: cumulative share of requests versus title rank, with the memory/NVMe placement cutoff marked titles, ranked by popularity (highest first) → cumulative share of requests 10–20% hottest titles kept in memory remaining catalog — served from NVMe
Figure 4. Schematic long-tail popularity curve — the axes and curve shape are illustrative of a typical long-tail distribution, but the placement cutoff is the cited figure: the memory-resident fraction is set at 10–20% of titles by popularity, chosen because that fraction already captures most of the request volume4.

Knowing the cutoff in advance is only half the mechanism; the other half is how an appliance's file inventory gets updated without ever touching that popular fraction's residency decision reactively. Netflix calls this process fill: every OCA downloads new and re-ranked content during a configured, largely off-peak window, sized to each appliance's family — up to 12 hours starting around 02:00 local time (adjustable by roughly ±2 hours per site), capped at roughly 1.2 Gbps and 6.5 TB total for a Storage-family appliance, and at roughly 0.4 Gbps and 2.1 TB for a Global-family appliance6 — the same two families, and roughly the same 2:1 ratio, as Table 1's serving-throughput split. Fill itself is prioritized to avoid re-fetching the same bytes from Netflix's own origin storage more than once per site: an appliance first tries to peer fill from another OCA in the same cluster or subnet, falls back to tier fill from an OCA it can see in BGP but that is outside its own cluster, and only falls back to a direct cache fill from Netflix's S3-backed origin if neither peer succeeds6 — so a popularity re-ranking that changes which titles deserve the memory tier at one appliance in a cluster tends to reach the rest of that cluster's appliances by copying between appliances, not by every appliance separately re-downloading from the origin.

The same proactive-over-reactive preference runs through all three mechanisms this module covers, at three different points in a video's lifecycle. Before a title is ever encoded, Section 5's per-title and per-shot analysis decides its bitrate ladder from trial encodes, not from watching how it streams in production. Before a client ever asks for a byte, this section's fill process has already decided, appliance by appliance, which fraction of tonight's catalog sits in memory. And before an ISP ever signs up, Section 1's appliance-placement decision has already assumed Netflix can predict roughly where its viewers and their bandwidth needs will be. None of these are unusual algorithms in isolation — trial encoding, LRU-adjacent placement heuristics, and capacity planning all exist elsewhere. What is unusual is that Netflix can run all three ahead of the request at all, because it controls the content, the catalog, and the appliance simultaneously; a general-purpose CDN serving arbitrary third-party origins has no equivalent knowledge to be proactive with, which is the structural reason Module 1's purpose-built-versus-multi-tenant framing exists in the first place.

5. From one bitrate ladder to per-title to per-shot

Everything in Sections 2–4 is about moving a fixed number of bytes as fast and as cheaply as possible. This section is about the other lever entirely: making fewer bytes need to move in the first place, by choosing what to encode more carefully. Until 2015, Netflix encoded every title against the same fixed bitrate ladder — a set of resolution/bitrate pairs at 320×240, 384×288, 512×384, 720×480, 1280×720, and 1920×1080, with a maximum bitrate of 5800 kbps at the top rung7. A client device picks the highest rung its current bandwidth can sustain; the ladder itself does not know or care whether the title behind it is a static-camera dialogue scene or a fast-motion action sequence. Quality was measured with PSNR, with a rough guide of 45 dB or above as imperceptibly good, 38 dB as the acceptable floor, and 35 dB as visibly degraded7 — and Netflix's own catalog needed anywhere from 2500 to over 8000 kbps at 1080p to hit that floor, depending entirely on the title7. One fixed ladder cannot simultaneously avoid wasting bits on a title that needs 2500 kbps and avoid starving one that needs 8000.

Per-title encoding, published by Netflix's encoding team in December 2015, replaces the one-ladder-fits-all assumption with a per-title trial-encode process7. For a given title, Netflix runs trial encodes at each candidate resolution across a range of quantization parameters (QPs), producing a set of (bitrate, quality) points; plotting quality against bitrate for every resolution traces out a family of curves, and the convex hull — the Pareto-efficient upper boundary across all of those curves — identifies, at every bitrate, which resolution actually delivers the best achievable quality7. The bitrate ladder for that specific title is then built by walking along this convex hull and picking rungs spaced so that "the perceptual difference between two adjacent bitrates should fall just below one just-noticeable difference" (JND)7 — enough rungs that a client stepping down one rung under bandwidth pressure loses the least perceptible quality possible, and no more rungs than that.

The two examples Netflix itself published make the mechanism concrete, and they pull in opposite directions because the two titles have opposite complexity profiles:

Table 3. Two titles, same convex-hull method, opposite outcomes, at equivalent perceptual quality7.
TitleContent profileFixed-ladder rungPer-title rungEffect
BoJack Horsemanflat-color animation, low complexity480p @ 1750 kbps1080p @ 1540 kbpshigher resolution at a lower bitrate
Orange Is the New Blacklive-action, high complexity1080p @ 5800 kbps1080p @ 4640 kbpssame resolution, ~20% fewer bits

BoJack Horseman's animated content is cheap to encode well — large flat regions of a single color compress efficiently — so the convex hull finds that full 1080p is achievable for fewer bits than the fixed ladder was spending to deliver a blurrier 480p. Orange Is the New Black is exactly the opposite case: live-action detail and grain are expensive to encode, but the fixed ladder's one-size 5800 kbps ceiling was already conservative enough that the convex hull could still shave roughly 20% off it at the same resolution and quality7. Averaged across Netflix's whole catalog, the company's own estimate is that per-title encoding saves about 20% in bitrate, and doing the same convex-hull optimization on a per-scene basis instead of per-title pushes that to about 30%7.

Bar comparison of fixed-ladder versus per-title bitrate at the specific rung Netflix published for two example titles 0 2000 4000 6000 kbps at the published rung 1750 fixed, 480p 1540 per-title, 1080p BoJack Horseman — higher resolution, fewer bits 5800 fixed, 1080p 4640 per-title, 1080p Orange Is the New Black — same resolution, ~20% fewer bits
Figure 5. Netflix's own published before/after numbers for two titles: per-title encoding either spends the saved bits on a higher resolution (BoJack Horseman) or simply removes them (Orange Is the New Black), depending on which the convex hull for that specific title supports7.

PSNR was always a proxy, and Netflix's own later work says so directly: "although researchers and engineers in the field are well aware that PSNR does not consistently reflect human perception, it remains the de facto standard for codec comparisons"8. In June 2016, Netflix open-sourced VMAF (Video Multimethod Assessment Fusion), a metric trained on human quality ratings that combines several lower-level quality signals through a machine-learned regressor instead of relying on any single one of them8. VMAF became the optimization objective for encoding decisions that PSNR previously served, on the premise that any bitrate-saving decision is only as good as the metric used to confirm it did not cost perceptible quality8.

Per-title encoding treats an entire title as one complexity profile. A title is not uniform, though — a dialogue-heavy scene and a fast-action sequence three minutes later can have wildly different complexity, and per-title's single ladder for the whole title has to be conservative enough to cover its hardest scene. Netflix's March 2018 Dynamic Optimizer work applies the same convex-hull idea per shot instead of per title: a typical shot runs about 4 seconds, so a one-hour episode that a chunk-based pipeline would have split into roughly 20 chunks gets encoded as roughly 900 separate shots instead9, each with its own optimized quality/resolution trajectory chosen against VMAF as the objective. Measured against per-title encoding at the same VMAF=80 quality target, the shot-based optimized AVC-High profile needed less than half the bits per-title encoding did, and the shot-based VP9 profile needed less than a third9 — on top of, not instead of, the 20% per-title had already saved relative to the original fixed ladder. Against Netflix's mobile-specific baselines specifically, the same shot-based profiles saved a further 17% (AVC-High versus its prior mobile encode) and 30% (VP9 versus its prior mobile encode)9, and shipped to iOS, Android, PlayStation 4, and Xbox One clients, with the VP9 variant limited at first to select Android devices9 — a reminder that a bitrate saving on a server means nothing until the specific client decoder it targets can actually play the resulting stream.

Where experts disagree

Section 5 presented per-title and per-shot encoding as a one-directional improvement: each step down in granularity saved more bits. What that framing skips is that Netflix's own publications about the same work also documented, in equal detail, what each step down in granularity cost to build and operate. The disagreement below is not between two companies with opposite philosophies — it is between two defensible readings of the same evidence, about whether finer-grained, per-unit encoding optimization is a strategy that generalizes past a catalog and viewership pattern like Netflix's.

Finer granularity should win whenever compute is affordable relative to bandwidth

Netflix's own encoding-team trajectory, read as a single argument across three publications79

Each step down in the unit of optimization — whole ladder, to per-title, to per-scene, to per-shot — bought more savings against the same quality target: roughly 20% from per-title, roughly 30% from per-scene, and shot-based AVC-High/VP9 needing less than half and less than a third of per-title's bits respectively at equal VMAF79. Encoding compute is a one-time cost paid once per title at ingest; bandwidth is a recurring cost paid on every one of millions of subsequent views. For a catalog that gets watched repeatedly at Netflix's volume, almost any amount of additional trial-encode compute is cheap relative to the egress bytes it removes forever.

  • Savings compound: bits removed at encode time are removed for every future view, with no additional marginal cost.
  • The quality metric side (VMAF) and the optimization-unit side (per-shot) reinforce each other — a better metric makes finer-grained decisions trustworthy instead of just more numerous.
  • The argument assumes a view-count-per-title distribution high enough that the one-time encode cost amortizes; it says nothing about whether that holds for a smaller or less-rewatched catalog.

Granularity has a real, Netflix-acknowledged operational cost that a smaller operator may not be able to absorb

Netflix's own account of productionizing shot-based encoding9

Moving from per-title's roughly 20 chunks per hour-long episode to per-shot's roughly 900 shots is not a free refinement — Netflix's own description of the "primary challenge" in shipping it was "retrofitting our parallel encoding pipeline to process significantly more encode units," which required "an order of magnitude more complexity" in the trial-encoding analysis step and "exposed system bottlenecks related to the number of messages passed between compute instances"9. That is a real distributed-systems cost — more encode jobs, more messages, more places for the pipeline to bottleneck — paid by an already large encoding-infrastructure team, at a company whose catalog size and rewatch volume make the trade obviously worth it. A streaming operator with a smaller catalog, fewer views per title, or a smaller encoding-infrastructure team faces the same ~45× increase in encode units without the same volume of repeated views to amortize it against.

  • A fixed or per-title ladder is a bounded, predictable amount of encoding work per title, with none of the message-volume scaling problems shot-based encoding introduced.
  • Simplicity is itself an operational property: fewer encode units means fewer places a pipeline can silently fail on a given title.
  • Leaves real, demonstrated bandwidth savings on the table if the operator's actual view-count-per-title distribution would have supported the finer-grained approach.
Assessment

Both positions describe the same underlying trade correctly; the disagreement is about where a given operator sits on the view-count-per-title axis, not about whether the physics of the trade-off is real. Netflix's own sequencing — ship per-title first, only add per-scene and then per-shot once the simpler win was already in production and paying for itself — is itself evidence that granularity should be adopted in the order of its cost, not all at once: per-title's convex-hull optimization is a modest, single-team encoding change with no distributed-systems consequences, while per-shot is a pipeline-architecture change that "exposed system bottlenecks" even for Netflix. A smaller operator can and should adopt per-title encoding almost unconditionally, since the technique itself does not multiply encode-unit count, but should treat per-shot as conditional on having Netflix-scale rewatch volume to amortize a real increase in encoding-pipeline complexity against. The number that would settle this for any specific operator is its own average views-per-title multiplied by the per-view bandwidth saved, compared against its own fully-loaded cost of the additional trial encodes and pipeline messaging — a calculation neither Netflix nor any other publisher of this technique has released in absolute dollar terms.

References

  1. officialNetflix, Inc. Open Connect Overview. PDF, © 2016, 2019, 2026. https://openconnect.netflix.com/Open-Connect-Overview.pdf. Accessed 2026-09-17. Used for: 2011 founding rationale, embedded-vs-IXP/SFI deployment models, the Cache Control Service / Steering (CODA) control-plane split, the six-step playback sequence in Figure 1, the "several orders of magnitude" demand-reduction claim.
  2. officialNetflix, Inc. Open Connect Appliances. Live documentation. https://openconnect.netflix.com/en/appliances/. Accessed 2026-09-17. Used for: Table 1, current-generation Storage vs. Global appliance hardware specifications.
  3. officialFreeBSD Foundation. Maintaining the World's Fastest Content Delivery Network at Netflix on FreeBSD. Case study, 2024, quoting Gleb Smirnoff (FreeBSD Vendor Summit, November 2023) and Drew Gallatin (OpenFest Bulgaria, November 2023). https://freebsdfoundation.org/wp-content/uploads/2024/05/netflix-CASE-STUDY-1.pdf. Accessed 2026-09-17. Used for: the 2012 10 Gbps proof-of-concept, the kTLS development history (Scott Long, Randall Stewart, Drew Gallatin), FreeBSD 13 NIC TLS offload (Chelsio T6, Mellanox ConnectX-6 Dx), the "first 100 gigabit per second production CDN server" quote, and the 400 Gb/s / ~200 GB/s memory-bandwidth / ~64 PCIe Gen4 lane figures used in the Section 3 derivation.
  4. blogNetflix Technology Blog. Serving 100 Gbps from an Open Connect Appliance. 2017-09-29. https://netflixtechblog.com/serving-100-gbps-from-an-open-connect-appliance-cdb51dda3b99. Accessed 2026-09-17. Used for: the 40→52→≈80→90+ Gbps throughput progression in Figure 2, the 100 Gbps stated goal, the memory-vs-NVMe 40→22 Gbps result in Figure 3, the 10–20% memory-resident popularity cutoff in Figure 4, and the TCP-telemetry-sampling detail.
  5. paperStewart, R., Long, S. (Netflix, Inc.). Improving High–Bandwidth TLS in the FreeBSD Kernel. AsiaBSDCon 2016. https://openconnect.netflix.com/publications/asiabsd_tls_improved.pdf. Accessed 2026-09-17. Used for: the 2015–16 OCA hardware generation's specs, the AES-GCM-over-CBC cipher decision, the BoringSSL/ISA-L bulk-cipher comparison, the sendfile/mbuf/iovec data-path fixes, and Table 2's Rev H/F/N cpu%/Gbps results.
  6. officialNetflix Open Connect Partner Help Center. Fill patterns. https://openconnect.zendesk.com/hc/en-us/articles/360035618071-Fill-patterns. Accessed 2026-09-17. Used for: fill window timing, the peer-fill/tier-fill/cache-fill priority order.
  7. blogAaron, A., Li, Z., Manohara, M., De Cock, J., Ronca, D. (Netflix Technology Blog). Per-Title Encode Optimization. 2015-12-14. https://netflixtechblog.com/per-title-encode-optimization-7e99442b62a2. Accessed 2026-09-17. Used for: the fixed bitrate ladder's resolutions and 5800 kbps ceiling, PSNR thresholds, the convex-hull/JND-spacing method, the BoJack Horseman and Orange Is the New Black worked examples in Table 3 and Figure 5, and the 20%/30% average savings estimates.
  8. blogLi, Z., Aaron, A., Katsavounidis, I., Moorthy, A., Manohara, M. (Netflix Technology Blog). Toward a Practical Perceptual Video Quality Metric. 2016-06-06. https://netflixtechblog.com/toward-a-practical-perceptual-video-quality-metric-653f208b9652. Accessed 2026-09-17. Used for: VMAF's open-source release date and its role replacing PSNR as the encoding-optimization objective.
  9. blogManohara, M., Moorthy, A., De Cock, J., Katsavounidis, I., Aaron, A. (Netflix Technology Blog). Optimized Shot-Based Encodes: Now Streaming!. 2018-03-09. https://netflixtechblog.com/optimized-shot-based-encodes-now-streaming-4b9464204830. Accessed 2026-09-17. Used for: the Dynamic Optimizer per-shot method, the 900-shots-vs-20-chunks figure, the AVC-High/VP9 bit-savings numbers, and the "Where experts disagree" section's production-cost quotes.