System Design · Module 13
YouTube and Vitess: Sharded MySQL at Video Scale, and What Isn't Public
Vitess is the one part of YouTube's stack that Google has actually documented at engineering depth: a MySQL-sharding proxy that served 100% of YouTube's database traffic for over five years1. This module goes deep on Vitess's VTGate/VTTablet/topology-service mechanism and the connection-memory arithmetic that motivated it, then draws an explicit line around everything else — transcoding, recommendations, CDN delivery — that Google/YouTube has not published at comparable depth.
1. Why YouTube needed something like Vitess
Vitess's own history page tells the origin story plainly, and it is worth reading as a case study in how a scaling problem migrates from one layer of a system to another until something has to absorb it permanently. YouTube's MySQL database reached a point where peak traffic would soon exceed its serving capacity. The first fix was the standard one: split the single instance into a primary for writes and a replica for reads2. That bought time, but read traffic — the page's own phrasing is "demand for cat videos at an all-time high" — grew enough to overload the replica, so YouTube added more replicas. That, too, was temporary: write traffic eventually became too large for the primary to keep up with, "requiring YouTube to shard data to handle incoming traffic"2. Sharding would have become necessary on size grounds alone even if write throughput hadn't forced it, since a single MySQL instance has a practical ceiling on how much data it can hold and still serve efficiently.
The part of this story that matters architecturally is what sharding did to the application layer. Once the data lived on multiple MySQL instances, every piece of code that touched the database had to first work out which shard held the row it wanted, then connect to the right instance. That routing logic got embedded directly in application source code2 — which means every service, every script, every one-off analysis job had to carry (and keep correctly synchronized) the same shard-mapping knowledge. Vitess's contribution was to delete that requirement: "Vitess let YouTube remove that logic from the application source code, introducing a proxy between the application and the database to route and manage database interactions"2. This is the general pattern this course keeps returning to — Module 10 named it explicitly for Vitess's VTGate as "the same 'one logical endpoint, many physical backends' shape a service-mesh sidecar provides for general RPC" — and it is worth being precise about what changed: the sharding decision (how many shards, which key ranges) didn't go away, it moved from being re-implemented in every client into being centralized in one piece of infrastructure that every client talks to identically.
The scale that made this worth doing is not fully quantified in Vitess's own documentation, but two adjacent facts anchor it. First, the payoff: "since then, YouTube has scaled its user base by a factor of more than 50, greatly increasing its capacity to serve pages, process newly uploaded videos, and more"2 — a 50x user-base multiplier is the kind of growth that a single, unsharded MySQL primary has no plausible path through. Second, current scale gives a feel for the traffic Vitess's descendant infrastructure has to sustain today: YouTube's own press materials report "over 20 million videos uploaded daily"15 as of this module's as-of date, each of which needs metadata rows written, indexed, and later served on every watch.
Vitess did not stay a YouTube-internal tool. In February 2018 the Cloud Native Computing Foundation's Technical Oversight Committee voted to accept it as a CNCF incubation project, and it became "the eighth CNCF project to graduate in November 2019, joining Kubernetes, Prometheus, Envoy, CoreDNS, containerd, Fluentd, and Jaeger"2. Companies including Slack, Square, and JD.com now run it in production1, independent of YouTube.
- Created at YouTube
Built to remove shard-routing logic from application code after primary/replica scaling ran out of headroom2.
- CNCF incubation
The CNCF Technical Oversight Committee accepts Vitess as an incubating project2.
- CNCF graduation
Eighth project to graduate, alongside Kubernetes, Prometheus, Envoy, and others2.
Sharded MySQL behind a proxy was not the only path available in 2010, and Vitess's own documentation states, in its comparison against NoSQL stores, exactly what the alternative would have cost: "if you're considering a NoSQL solution primarily because of concerns about the scalability of MySQL, Vitess might be a more appropriate choice," because "NoSQL databases do not define relationships between database tables, and only support a subset of the SQL language," "NoSQL datastores do not usually support transactions," and "NoSQL solutions have custom APIs, leading to custom architectures, applications, and tools"1. Read against §1's history, this is the same choice stated from the other direction: YouTube's problem was connection and shard-routing overhead, not a data-modeling mismatch with SQL, so discarding SQL semantics (joins, WHERE clauses, transactions, the query language every engineer on the team already knew) to fix a purely operational problem would have been solving the wrong layer. Module 3's Bigtable and Dynamo case studies are the other side of that trade: both accepted a restricted data model in exchange for scaling properties Vitess gets by other means (proxying and sharding an unmodified engine) rather than by giving up SQL.
2. The architecture: VTGate, VTTablet, and how a query finds its shard
Module 10 introduced Vitess's two moving parts at the level a general services module needs: VTGate as a MySQL-protocol-speaking proxy, VTTablet as a per-instance agent, and a topology service holding the map between them. This section goes one level deeper, into the pieces that actually decide which shard a given query belongs to — keyspaces, shards, VSchema, and Vindexes — because that decision logic is the part a general service-mesh treatment has no equivalent for.
2.1 VTGate: one endpoint, protocol-compatible, routing-aware
VTGate is described in Vitess's own component documentation as "a lightweight proxy server that routes traffic to the correct VTTablet servers and returns consolidated results back to the client"9. It speaks both the MySQL wire protocol and a Vitess-specific gRPC protocol, so "your applications can connect to VTGate as if it is a MySQL Server"9 — no driver change, no application-level awareness that the database behind the wire is actually dozens or hundreds of separate MySQL instances. When it decides where to send a query, VTGate takes into account "the sharding scheme, required latency and the availability of tables and their underlying MySQL instances"9: the sharding scheme tells it which shard a row lives on, the latency requirement (a transactional write versus a tolerant analytics read) tells it whether it may route to a lagging replica, and instance availability lets it avoid a MySQL instance that is currently down or failing over.
2.2 VTTablet and the tablet types that encode a MySQL instance's job
Vitess's documentation defines a tablet plainly: "a combination of a mysqld process and a corresponding vttablet process, usually running on the same machine"7. VTTablet is the half of that pair doing the work: it "manages queries, connections, and replication"1 for its one MySQL instance. Every tablet is assigned a tablet type, and that type is what tells VTGate whether it is safe to route a given query there. Vitess documents six types7:
| Type | Role | Conventional use |
|---|---|---|
| primary | The replica currently serving as the MySQL primary for its shard | All writes; consistent reads |
| replica | Eligible for promotion to primary | Live, user-facing read traffic (website frontend) |
| rdonly | Not eligible for promotion | Backups, dumps, heavy analytical queries, MapReduce |
| backup | Stopped replication at a consistent snapshot to upload a backup | Returns to its prior type once the backup finishes |
| restore | Started with no data, restoring from the latest backup | Becomes replica or rdonly once caught up |
| drained | Reserved by a Vitess background process | e.g. held out of serving during resharding |
This is the mechanism behind the latency/consistency trade-off VTGate makes when routing: sending a query to a replica or rdonly tablet buys horizontal read scaling at the cost of possible replication lag, while a primary read is always current but does not scale by adding more tablets. §5 returns to exactly what guarantee each choice buys.
2.3 Keyspace, shard, VSchema, and Vindex: the sharding-scheme layer VTGate consults
A keyspace is Vitess's name for a logical database: "if you're using sharding, a keyspace maps to multiple MySQL databases; if you're not using sharding, a keyspace maps directly to a MySQL database name. In either case, a keyspace appears as a single database from the standpoint of the application"5. A shard is "a subset of a keyspace"6 — a contiguous range of an unsigned integer keyspace-id space, written in hexadecimal. Vitess's own notation examples make the range arithmetic explicit: "-80 == 00-80 == 0000-8000 == ... a plain - denotes the full keyrange," and because the ranges are left-justified, "80- is not the same as 80-FF. This is why: 80-FF == 8000-FF00. Therefore FFFF will be out of the 80-FF range"6. An unsharded keyspace has exactly one shard, conventionally named 0 or -6; a sharded keyspace can have "hundreds of shards" for some Vitess users6.
What maps an actual row to a position in that keyspace-id range is a Vindex ("vitess index"), configured per table in a VSchema: "a VSchema allows you to describe how data is organized within keyspaces and shards. This information is used for routing queries, and also during resharding operations. For a keyspace, you can specify if it's sharded or not. For sharded keyspaces, you can specify the list of vindexes for each table"8. Vitess's own documentation gives a concrete example of the arithmetic: "a xxhash type vindex produces an unsigned 64-bit integer as output... all integers less than 0x8000000000000000 will fall in shard -80. Any number with the highest bit set will be >= 0x8000000000000000, and will therefore belong to shard 80-"6.
This is a deliberately different design from the consistent-hash ring in Module 3: a Dynamo-style ring places a key by hashing it directly onto a circular space that virtual nodes divide, with placement and replica selection fused into one mechanism. Vitess separates them — the Vindex is a pluggable function the schema author chooses per table (hash-based like xxhash, or a direct pass-through for an already-well-distributed binary column6), and the resulting id is then matched against explicit, human-assigned hexadecimal shard boundaries rather than an automatically-rebalanced ring. The trade-off is explicitness for automation: an operator can reason about exactly which shard owns which id range and can resize deliberately, at the cost of not getting the ring's near-automatic rebalancing on membership change.
Putting the three layers together gives the full request path: VTGate accepts a MySQL-protocol query, consults the VSchema to find the table's Vindex, computes the keyspace id, matches it against the keyspace's shard boundaries, and forwards the query to a VTTablet of an appropriate tablet type fronting that shard's MySQL instance.
3. The topology service: the map every VTGate consults
Everything in §2 assumed VTGate already knows which shards exist, which VTTablet fronts which MySQL instance, and which tablet is currently the primary for a shard. That knowledge lives in the topology service, which Vitess describes as "a set of backend processes running on different servers. Those servers store topology data and provide a distributed locking service"4. Vitess treats the actual storage backend as pluggable behind a common interface — "Vitess uses a plug-in system to support various backends for storing topology data, which are assumed to provide a distributed, consistent key-value store. The default topology service plugin is etcd2"4 — with etcd, ZooKeeper, or Consul all supported1. Any of the three would be recognizable from Module 6: they are exactly the class of consensus-backed coordination kernels that module derived from Raft and Zab, repurposed here to hold one specific kind of state — cluster topology — rather than general application locks.
The topology service is explicitly two-tiered, and the split matters for both fault tolerance and performance:
- Global topology. Stores data that "does not change frequently... data about keyspaces and shards as well as the primary tablet alias for each shard," used for operations like reparenting and resharding4. Because losing it would be catastrophic and it is read rarely, Vitess recommends spreading it across multiple cells: "in order to survive any single cell going down, the global topology service should have nodes in multiple cells, with enough to maintain quorum in the event of a cell failure"4 — the same majority-quorum reasoning Module 6 derived for Raft, applied here to placement rather than log replication.
- Local topology. One instance per cell, holding "data about tablets in the cell, the keyspace graph for that cell, and the replication graph for that cell"4. Vitess is explicit that this must stay available for day-to-day operation: "the local topology service must be available for Vitess to discover tablets and adjust routing as tablets come and go."4
The detail worth pulling out for a reader used to reasoning about tail latency is what the topology service is not on the hook for: "no calls to the topology service are made in the critical path of serving a query at steady state. That means queries are still served during temporary unavailability of topology"4. VTGate caches the routing information it needs and only re-consults the topology service when that cached view might be stale — a tablet failing over, a shard splitting — rather than on every single query. This is the same principle ZooKeeper's own paper (Module 6) states for its read path: serve the common case from a local, possibly-slightly-stale cache, and reserve the strongly-consistent coordination service for the events that actually change the world rather than for steady-state traffic.
4. The problem Vitess actually solves: MySQL's per-connection memory cost
Vitess's own comparison table states the constraint in one sentence: "every MySQL connection has a memory overhead that ranges between 256KB and almost 3MB, depending on which MySQL release you're using. As your user base grows, you need to add RAM to support additional connections, but the RAM does not contribute to faster queries. In addition, there is a significant CPU cost associated with obtaining the connections"1. Two things about that sentence matter for a systems reader. First, the overhead is per connection, not per query — an idle connection held open "just in case" costs the same memory as a busy one. Second, that memory buys nothing: it is bookkeeping (thread stacks, per-connection buffers, session state), not buffer-pool space that would make a query faster. Every additional connection is pure tax.
That range is not arbitrary, and MySQL's own reference manual documents exactly what it is spent on. "Each thread that the server uses to manage client connections requires some thread-specific space," specifically "a stack (thread_stack)... a connection buffer (net_buffer_length)... a result buffer (net_buffer_length)," where the connection and result buffers each "begin with a size equal to net_buffer_length bytes, but are dynamically enlarged up to max_allowed_packet bytes as needed"11. The stack term alone explains Vitess's stated 256KB floor: thread_stack defaults to 262,144 bytes — 256 KiB — on a 64-bit build12, and MySQL's classic connection handler runs one OS thread per connection, so that stack is paid once per open connection whether or not the connection is doing anything. The upper end of Vitess's stated range comes from the same buffers growing under load: a connection that has just run a large sort or scan can be holding an enlarged sort buffer (sort_buffer_size) or read buffer (read_buffer_size) on top of its fixed stack, and the manual notes a copy of the current statement string is kept for the duration of a running statement as well11 — which is how a connection sitting idle can cost barely more than a stack frame, while one that just finished a heavy query can be holding several megabytes it has not yet released.
The arithmetic below is illustrative, not a disclosed YouTube number — Vitess's documentation states the per-connection overhead range but not YouTube's actual fleet size or per-process connection count, and this course's source map has neither. Take a hypothetical fleet of 4,000 application-server processes, each configured to hold 25 concurrent connections open to a MySQL primary — a plausible shape for a web tier serving a high-traffic product, chosen only to make the units concrete:
Lower bound (256 KiB/connection)
$$ 4{,}000 \times 25 \times 256\ \text{KiB} = 100{,}000 \times 256\ \text{KiB} \approx 24.4\ \text{GiB} $$Upper bound (3 MiB/connection)
$$ 4{,}000 \times 25 \times 3\ \text{MiB} = 100{,}000 \times 3\ \text{MiB} \approx 293\ \text{GiB} $$That is 24 to 293 gibibytes of RAM on the database server spent purely on holding connections open, before a single byte of that memory has cached a row or served a query — and it scales linearly with the number of application processes, which is exactly the dimension that grows fastest as a product adds traffic. Vitess's fix is to stop the application from holding raw MySQL connections at all: "Vitess creates very lightweight connections. Vitess' connection pooling feature uses Go's concurrency support to map these lightweight connections to a small pool of MySQL connections. As such, Vitess can easily handle thousands of connections"1. The documentation does not publish an exact pool size — only that it is "small" relative to the number of logical application connections it serves — so the figure below marks that bar honestly as an order-of-magnitude claim rather than inventing a number Vitess itself does not give.
Raw MySQL connection memory (illustrative fleet)
\( M = N \times C \times B \), with \(B\) fixed to the two bounds Vitess's own documentation states1; \(N\) and \(C\) are yours to set — the point is that \(M\) grows linearly in both and buys zero query throughput.
Connection pooling is the mechanism that motivated Vitess, but Vitess's own feature list names three more protections that ride along with it, all documented on the same page: "query de-duping — reuse results of an in-flight query for any identical requests received while the in-flight query was still executing," a "transaction manager" that "limits the number of concurrent transactions and manages timeouts to optimize overall throughput," and query rewriting that "uses a configurable set of rules to rewrite queries that might hurt database performance," such as adding a missing LIMIT1. All three exploit the same architectural fact that made pooling possible: because every query already passes through VTGate and VTTablet rather than going straight to mysqld, there is a natural place to intercept it before it becomes load on the database.
5. Operating a sharded fleet: durability, consistency, and resharding
Routing is only half of what a sharding layer has to get right; the other half is what happens when a shard's primary dies, or when the number of shards itself needs to change. Vitess's own scalability-philosophy documentation states positions on both that are worth reading against Module 3's general replication treatment and Module 4/5's isolation-level and two-phase-commit vocabulary, because Vitess makes concrete, named choices inside that space rather than leaving them abstract.
On durability, Vitess explicitly rejects relying on a single disk flush: "the new approach to durability is achieved by copying the data to multiple machines, and even geographical locations"10 — the same replication-over-durable-write logic Module 3 derived generally, operationalized here as a specific recommendation to enable semi-synchronous replication, because "this allows Vitess to failover to a new replica when a primary goes down, with no data loss"10. The corollary recommendation is unusual enough to flag: "avoid recovering a crashed database. Instead, create a fresh one from a recent backup and let it catch up"10 — treating a crashed MySQL instance as disposable rather than something to repair in place, which only works because the semi-sync replicas already hold a durable copy. Leaning on replication this way also lets an operator relax MySQL's own disk-durability settings — Vitess recommends disabling sync_binlog, since doing so "greatly reduces the number of IOPS to the disk thereby increasing effective throughput"10, trading a per-write fsync for reliance on the replica set.
On consistency, Vitess is explicit that sharding is not free of new anomalies and names exactly which ones an application must tolerate: "cross-shard reads may not be consistent with each other," and in what it calls best-effort mode, "cross-shard transactions can fail in the middle and result in partial commits"10 — an availability/consistency trade every reader of Module 5's PACELC framing will recognize immediately. The alternative Vitess offers is 2PC mode, which "give[s] you distributed atomic guarantees," but at a stated cost: "choosing this option increases the write cost by approximately 50%"10. A single-shard transaction pays none of this: it "continue[s] to remain ACID, just like MySQL supports it"10, because it never needs coordination across tablets. This is exactly the incentive structure that makes shard-key choice a first-order design decision rather than an implementation detail — a schema whose shard key keeps related rows on the same shard avoids the 50% tax entirely, while a poorly chosen key pays it on every multi-row transaction.
Read consistency is tunable per query through tablet type, formalizing what §2.2's table only implied: a REPLICA/RDONLY read can be "stale depending on replica lag," and Vitess mitigates the risk operationally rather than by blocking — "VTGate servers are capable of monitoring replica lag and can be configured to avoid serving data from instances that are lagging beyond X seconds"10. A read from the PRIMARY tablet, by contrast, gives read-after-write consistency at isolation level READ_COMMITTED, and a primary transaction upgrades that to REPEATABLE_READ with full ACID writes for a single shard10 — vocabulary straight out of Module 4's isolation-level ladder, just applied per-tablet instead of per-database.
One replication topology Vitess deliberately does not support is active-active (multi-master): "Vitess doesn't support active-active replication... since all writes have to eventually be applied to all writable servers, it's not a sustainable strategy. Vitess addresses this problem through sharding, which can scale indefinitely"10. In its place, Vitess documents automatic primary-failover — "native capability for detecting primary failures and performing a failover to a new primary within seconds of failure detection"10 — as sufficient high availability for most applications, while conceding one gap outright: "low-latency geographically distributed writes... is not addressed by Vitess. The current recommendation is to absorb the latency cost," or shard by geography so that most primaries end up local to their own write traffic10.
Finally, resharding — changing the number of shards on a running cluster — is documented as an explicit, verified data-migration workflow rather than an instantaneous operation: "the data in the source shards is copied into the destination shards, allowed to catch up on replication, and then compared against the original to ensure data integrity. Then the live serving infrastructure is shifted to the destination shards, and the source shards are deleted"6. Vitess's own summary of the user-facing cost of that whole workflow is "just a few seconds of read-only downtime"1 per migration — a small, bounded, and specifically stated cost for an operation that, done by hand against unsharded application code, is exactly the kind of project that pushed YouTube toward Vitess in the first place (§1).
Operators are also steered toward smaller MySQL instances than the one-per-host default the industry tends to run: Vitess recommends "instances be broken up into manageable chunks (250GB per MySQL server), and not to shy away from running multiple instances per host," because "there are fewer lock contentions to worry about, replication is a lot happier, production impact of outages is smaller, backups and restores run faster"10 — smaller blast radius per instance, at the (stated as roughly neutral) cost of the same aggregate resource usage.
6. What is actually public about the rest of YouTube
Figure 1 drew the boundary; this section states it precisely enough to act on, the same way Module 12 stated the same kind of boundary for Gmail's storage architecture. Vitess sits on the well-documented side because it has a dedicated project, a public docs site, and a named history going back to 2010. The rest of what makes YouTube work as a video product — encoding, ranking, and delivery — has no source in this course's map that documents it at comparable depth, and each of the three gaps is a slightly different shape.
Transcoding. No source available to this module describes how an uploaded video is split into encode jobs, how those jobs are distributed across machines, what codec-selection or bitrate-ladder logic runs, or how the pipeline handles retries and partial failures. This is not a claim that YouTube lacks such a system — obviously it has one, since every uploaded video ends up available in multiple resolutions — only that Google has not published an engineering account of it comparable to the Vitess documentation this module has spent five sections on. Nothing in §1 through §5 should be read as describing transcoding in any way; it does not.
Recommendations. Here the gap is not total silence but a single, dated data point. Google published "Deep Neural Networks for YouTube Recommendations" at ACM RecSys in 2016, describing a two-stage system: a candidate-generation deep network that narrows hundreds of millions of videos to a few hundred candidates, followed by a separate ranking network that orders those candidates for a specific user and context14. That paper is a real, citable, Google-authored primary source — but it is a decade old at this module's as-of date, predates the transformer architectures that reshaped ranking and retrieval systems industry-wide since, and Google has not published a comparably detailed account of whatever runs in its place today. Citing it as "how YouTube's recommendation system works" in 2026 would overstate what one nine-page conference paper from 2016 can tell you about a system that has had a decade to change underneath it.
CDN and delivery. This is the gap with the most nuance, because Google does publish something here — just not at the level a systems reader wants. Google's own peering documentation describes the Google Global Cache (GGC) program: "Google Global Cache (GGC) allows ISPs to serve certain Google content from within their own networks," with caches placed as physical machines "inside ISP networks," and a stated hit rate where "typically between 70-90% of cacheable traffic can be served from GGC"16. Google also documents a failover property: "GGC provides resilience through multiple levels of redundancy, which are transparent to users. Google ensures that users are always served from the best available data location, even in the event of a temporary outage"16 — a real, if generic, statement that the caching layer degrades gracefully rather than failing a viewer's request outright. A public 2017 talk by a Google Edge peering lead independently confirms that YouTube's traffic rides on this same infrastructure, listing "Youtube, Play" among the products Google serves alongside Search, Maps, and Gmail on the same slide that introduces GGC18. None of this is engineering depth — it is peering-integration documentation aimed at ISPs deciding whether to host a cache box, not an engineering account: it says nothing about which videos get cached where, how per-title or per-region popularity is estimated, what storage medium the boxes use, or how a specific video stream reaches a specific viewer's player. Whatever generic video-CDN mechanism actually applies — the tiered, edge-to-origin caching pattern Module 1 already covered for CDNs in general, or the adaptive-bitrate-switching mechanism the IETF's own HTTP Live Streaming specification documents generically ("clients should switch between different Variant Streams to adapt to network conditions," where "a Master Playlist provides a set of Variant Streams, each of which describes a different version of the same content" at different bitrates and resolutions13) — neither source is YouTube's own account of what YouTube's players and edge infrastructure actually do; both are stated here explicitly as generic industry mechanism, not YouTube fact. Module 14's treatment of Netflix's Open Connect is the contrast case: Netflix has published exactly the kind of throughput-progression, memory-versus-disk, and popularity-tail engineering detail about its CDN that no source in this module's map has found for YouTube's.
The honest summary is a gradient, not a binary: Vitess is documented at the depth this whole module has used; GGC is documented at the depth of an ISP integration guide; recommendations have one aging academic paper; transcoding has nothing. Treating any of the last three as "documented" because a company blog post or a generic streaming-industry explainer exists for the general problem would be filling a gap with something that is not actually about YouTube — precisely the mistake this module's scope was written to avoid.
Where experts disagree
Vitess is one answer to a general question: once a relational workload outgrows one machine, do you keep the existing database engine and put a sharding-aware proxy in front of it, or do you move to a database that was built from the start to be distributed? YouTube's own history (§1) is a real data point for the first path; Spanner, already established in Module 4/12 of this course, is a real data point for the second. Neither source in this module's map states the comparison as a debate directly, but the architectural trade-off is explicit in each system's own documented design choices, and it recurs constantly in practice — most consumer sharding-middleware projects (Vitess, ProxySQL-style layers, Citus's Postgres extension) are entries in the first camp; most NewSQL systems (Spanner, CockroachDB, YugabyteDB, TiDB) are entries in the second.
Middleware sharding over an unmodified relational engine
Vitess's own architecture and documented history1210
Keep MySQL exactly as it is — same storage engine, same replication, same operational tooling, same query language — and add a proxy layer (VTGate/VTTablet) that handles routing, pooling, and failover on top. YouTube's actual 2010 choice was this path: it had a large, already-working MySQL-based application and needed to remove shard-routing logic from that application without rewriting it onto a different database.
- Incremental: an existing application can adopt it without a storage-engine migration.
- Reuses MySQL's mature operational ecosystem (backup tools, replication, monitoring) rather than replacing it.
- Single-shard transactions stay fully ACID with zero added cost10.
- Cross-shard transactions either risk partial commits in best-effort mode or pay a stated ~50% write-cost premium in 2PC mode10.
- Shard-key design becomes a permanent, consequential decision the application must get right up front.
- No default external consistency across shards the way TrueTime provides for Spanner17 (Spanner itself is established in Module 4/12; cited here only for this comparison).
Natively-distributed SQL, built for sharding from the start
Spanner's own design, as established in Module 4 and Module 1217
Replace the single-machine relational engine with a database whose storage layer was designed around Paxos-replicated shards (directories) from day one. Spanner "shards data across many sets of Paxos state machines in datacenters spread all over the world"17, moves a 50MB directory between Paxos groups in "a few seconds"17, and is "the first system to distribute data at global scale and support externally-consistent distributed transactions"17 via the TrueTime API, with no separate proxy layer for an application to operate.
- Resharding and rebalancing are automatic and built into the storage layer, not a scheduled migration workflow.
- Cross-shard transactions get the same external consistency guarantee as single-shard ones, without a stated write-cost penalty comparable to Vitess's 2PC mode.
- No separate proxy tier (VTGate-equivalent) for an operator to run and reason about as its own failure domain.
- Requires migrating off the existing relational engine entirely — a multi-year, high-risk project for an application already running at YouTube's 2010 scale.
- The externally-consistent design depends on infrastructure (GPS and atomic clock references for TrueTime) that only a handful of operators build in-house, though Google now sells the result as a managed product.
Which path wins is mostly a function of what already exists. For an application already running at scale on an unmodified relational engine — YouTube's actual 2010 position — the middleware path is close to strictly better: it removes the immediate scaling pain (connection cost, shard-routing logic scattered across every client) without betting the product on a multi-year storage-engine migration, and it lets an operator defer the harder cross-shard-transaction question until the application's actual transaction pattern is known well enough to choose best-effort or 2PC mode deliberately. For a system built from scratch, or one whose workload is dominated by transactions that routinely span many shards, the calculus flips: paying Vitess's stated ~50% write-cost premium on every such transaction, indefinitely, is a worse trade than accepting a natively-distributed engine's steeper adoption cost once, up front. Neither source in this module's map publishes a head-to-head benchmark of the two approaches on the same workload; that comparison — Vitess in 2PC mode versus a Spanner-class system, same schema, same cross-shard transaction mix — is the evidence that would actually settle which one is faster rather than merely which one is architecturally cleaner.
References
- officialVitess contributors. What Is Vitess. Vitess (CNCF graduated project) documentation, v24.0, 2026. https://vitess.io/docs/overview/whatisvitess/. Accessed 2026-09-17. Used for: VTGate/VTTablet roles, topology service one-liner, YouTube's 5+ years of 100% database traffic, the 256KB-3MB per-connection memory overhead and connection-pooling quote, resharding's "few seconds of read-only downtime," and the query de-duping/transaction manager/query rewriting feature list.
- officialVitess contributors. History. Vitess documentation, v24.0, 2026. https://vitess.io/docs/24.0/overview/history/. Accessed 2026-09-17. Used for: the 2010 origin story (primary/replica escalation, forced sharding, app-embedded routing logic), YouTube's 50x user-base growth since, and the Feb 2018 CNCF incubation / Nov 2019 CNCF graduation dates.
- paperPang, R., Caceres, R., Burrows, M., et al. Zanzibar: Google's Consistent, Global Authorization System. USENIX Annual Technical Conference, 2019. https://www.usenix.org/system/files/atc19-pang.pdf. Accessed 2026-09-17. Used for: the abstract's statement that Zanzibar is used by "hundreds of client services... including Calendar, Cloud, Drive, Maps, Photos, and YouTube," the one other Google-authored primary source in this course's map that names YouTube directly.
- officialVitess contributors. Concepts: Topology Service. Vitess documentation, v24.0, 2026. https://vitess.io/docs/24.0/concepts/topology-service/. Accessed 2026-09-17. Used for: the global-vs-local topology split, the default etcd2 plugin, multi-cell quorum survivability for global topology, and the "no calls to the topology service are made in the critical path... at steady state" claim.
- officialVitess contributors. Concepts: Keyspace. Vitess documentation, v24.0, 2026. https://vitess.io/docs/24.0/concepts/keyspace/. Accessed 2026-09-17. Used for: the definition of a keyspace as a logical database that can map to one or many MySQL databases.
- officialVitess contributors. Concepts: Shard. Vitess documentation, v24.0, 2026. https://vitess.io/docs/24.0/concepts/shard/. Accessed 2026-09-17. Used for: the shard definition, hexadecimal keyrange notation (-80/80- worked examples), the xxhash-vindex example, and the resharding data-copy/catch-up/verify/cutover workflow.
- officialVitess contributors. Concepts: Tablet. Vitess documentation, v24.0, 2026. https://vitess.io/docs/24.0/concepts/tablet/. Accessed 2026-09-17. Used for: the tablet definition (mysqld + vttablet pair) and the six documented tablet types (primary, replica, rdonly, backup, restore, drained) in Table 1.
- officialVitess contributors. Concepts: VSchema. Vitess documentation, v24.0, 2026. https://vitess.io/docs/24.0/concepts/vschema/. Accessed 2026-09-17. Used for: the VSchema's role in describing per-table Vindex assignment and sequence generators for auto-increment-like ids.
- officialVitess contributors. Concepts: VTGate. Vitess documentation, v24.0, 2026. https://vitess.io/docs/24.0/concepts/vtgate/. Accessed 2026-09-17. Used for: VTGate's protocol support (MySQL protocol and Vitess gRPC) and its routing inputs (sharding scheme, latency requirement, table/instance availability).
- officialVitess contributors. Overview: Scalability Philosophy. Vitess documentation, v24.0, 2026. https://vitess.io/docs/24.0/overview/scalability-philosophy/. Accessed 2026-09-17. Used for: the 250GB-per-instance sizing recommendation, semi-sync replication and sync_binlog durability guidance, the best-effort-vs-2PC cross-shard transaction model and its ~50% write-cost premium, REPLICA/RDONLY/PRIMARY read-consistency levels, replica-lag-aware routing, and the no-active-active-replication design decision.
- officialOracle / MySQL. MySQL 8.0 Reference Manual: 10.12.3.1 How MySQL Uses Memory. 2026. https://dev.mysql.com/doc/refman/8.0/en/memory-use.html. Accessed 2026-09-17. Used for: the per-thread memory breakdown (thread stack, connection buffer, result buffer growing to max_allowed_packet, sort/read buffers) that explains what Vitess's stated 256KB-3MB per-connection range is actually spent on.
- officialOracle / MySQL. MySQL Bugs: Bug #71045, wrong default value displayed for "thread_stack" option on 64-bit OS. bugs.mysql.com, 2026. https://bugs.mysql.com/bug.php?id=71045. Accessed 2026-09-17. Used for: confirming thread_stack's documented default of 262,144 bytes (256 KiB) on a 64-bit build, matching Vitess's stated per-connection memory floor.
- standardPantos, R., May, W. RFC 8216: HTTP Live Streaming. IETF, August 2017. https://www.rfc-editor.org/rfc/rfc8216. Accessed 2026-09-17. Used for: the generic adaptive-bitrate mechanism (Master Playlist / Variant Streams / client-side switching on network conditions), cited explicitly as a generic industry mechanism, not a description of YouTube's own implementation.
- paperCovington, P., Adams, J., Sargin, E. Deep Neural Networks for YouTube Recommendations. Proceedings of the 10th ACM Conference on Recommender Systems (RecSys '16), 2016. https://doi.org/10.1145/2959100.2959190. Accessed 2026-09-17. Used for: the one public, Google-authored, but decade-old account of YouTube's recommendation architecture (candidate generation followed by ranking), flagged as dated and not a description of the current production system.
- officialYouTube. Press. blog.youtube, 2026. https://blog.youtube/press/. Accessed 2026-09-17. Used for: the "over 20 million videos uploaded daily" scale statistic.
- officialGoogle. Introduction to GGC. Google Interconnect Help, support.google.com, 2026. https://support.google.com/interconnect/answer/9058809. Accessed 2026-09-17. Used for: the Google Global Cache program's placement of cache machines inside ISP networks and the stated 70-90% cacheable-traffic-served-locally figure, as the one official but peering-level (not engineering-level) source on YouTube's CDN delivery.
- paperCorbett, J.C., Dean, J., Epstein, M., et al. Spanner: Google's Globally-Distributed Database. OSDI 2012. https://research.google.com/archive/spanner-osdi2012.pdf. Accessed 2026-09-17. Used for: the Paxos-sharded architecture, the 50MB-directory-in-a-few-seconds resharding claim, and the externally-consistent-transactions-via-TrueTime claim, as the natively-distributed-SQL comparison point in "Where experts disagree" (Spanner's full mechanism is established in Module 4/12).
- talkVolmer, T. Google Global Cache - Enabling Content. Google Edge lead, EMEA. AfPIF (Africa Peering and Interconnection Forum) 2017, Abidjan. https://www.afpif.org/wp-content/uploads/2017/10/06_Google-Global-Cache-Enabler-2017.pdf. Accessed 2026-09-17. Used for: slide 2's listing of "Youtube, Play" among the Google products served by the infrastructure GGC is built on, confirming YouTube traffic rides on GGC.