Sharded URL Shortener: How a Request Finds Its Shard

In this chapter

Sharding in real life: a URL shortener, request by request

More on Scaling out

• Before Sharding: When One Database Is Not Enough
• Database Sharding Explained: Keys, Hash Rings, MySQL & Postgres
• Shard Keys, Consistent Hashing, Rebalancing & Routing
• Sharding MySQL & PostgreSQL, Names Across Databases, Interview

Keep learning

• Previous: Before Sharding: When One Database Is Not Enough
• Next: Database Sharding Explained: Keys, Hash Rings, MySQL & Postgres
• All SQL tutorial chapters

Overview

chapters · scaling out Sharding in real life: a URL shortener, request by request Once one server is not enough (before sharding), the data is split across servers. Before the theory, a real system. Shorten a link, then open it, and follow every hop from the browser to the one database server that holds the row. The sharding chapter then explains each idea you meet here.

1. The product

A URL shortener like bit.ly or TinyURL turns a long link into a short one (sho.rt/aB3x9Qz). When someone opens the short link, it redirects them to the long one. The data is one table, links: a short code and the long_url it points to. long URL → new short code one row in links short code → long URL the browser goes there Two operations: a write when a link is created, and a read every time it is opened. Reads outnumber writes about 100 to 1.

2. Why one database is not enough

At bit.ly scale there are billions of links, and thousands of new ones every second at peak. That is more data and more writes than one Postgres server should hold, so the links table is split across three database servers, each with the same table and a third of the rows. Each slice is a shard.

3. Try it: write a link, then read it

Press Shorten and step through: the browser, the load balancer, a unique ID, the short code, the hash ring, and the INSERT on exactly one shard. Then press Open on any row and follow the read: the cache, the same hash, the SELECT, the redirect.

4. Why it is built this way

The short code is the shard key Every read starts with the code, so every read goes to exactly one shard. It is unique, spreads evenly after hashing, and is in every query. Counter blocks, not hash(url) Hashing the long URL to make the code causes collisions you must check for. Unique numbers from reserved blocks never collide and need no extra lookup. A ring, not hash % 3 When a fourth shard is added, only about a quarter of the links move instead of three quarters. Cache in front of the shards 302, not 301 A 301 is cached by the browser, so later clicks never reach you and cannot be counted. Pick 301 if you want less traffic and no analytics. Analytics off the hot path Clicks go to a queue and are counted later, so a slow analytics system never slows a redirect. The one query this design makes expensive is “all links created by one user”: it has no code in it, so it would ask every shard. A real shortener keeps a second small table sharded by user_id for the “my links” page.

5. Now the theory behind it

Every idea above has its own section in the next chapter: Why split at all, and when not to: when the numbers say shard Why the code makes a good key: the shard key: three tests Why a ring and not % 3: hash % N vs a hash ring What “my links” costs: queries without the shard key Adding a fourth shard: rebalancing → Next: Sharding: one table split across many database servers

More courses

• Jobs, linked to lessons — Browse jobs where every requirement links to the chapter that teaches it. See how ready you are, how long the gap takes to close, and which listings look stale.
• Match a job post to courses — Paste a job post. Each skill is matched to a track here and put in learning order, foundations first.
• React from open source with patterns — Patterns extracted from Material UI, Radix, React Aria, React Hook Form, Redux Toolkit, TanStack, and Next.js — not slides about hooks.
• System Design — Core concepts first (SQL vs NoSQL, CAP, indexing, replication, sharding), then designs grouped by what carries them: the database, Redis, Kafka, or many services.
• Build a real API with NestJS — One ticket-booking API, built in order: a plain Node server first, then NestJS modules, validation, a real database, and the features real clients lean on.
• Microservices — Microservice patterns one diagram at a time: how to split a monolith, how services talk, and how data stays consistent without distributed transactions.
• Kafka — A beginner-to-senior path through event logs, reliable delivery, production operations, and disaster recovery.
• AI engineering for software engineers — Eight free reads, in order, for software engineers moving into AI: when to build an agent, how to build one, how to test it, and how the model works underneath.
• Frontend system design — How a UI is sliced when it is a product, not a page: ownership of state, cache, routes, and what happens when the network lies.
• TypeScript in depth — The type system as a design tool. Inference, narrowing, generics, and the errors that mean the model is wrong — not a catalog of utility types.
• CSS in depth — See the box, then the page. Visual chapters from the box model through Flexbox surprises, Grid, selectors, and units, then backgrounds, shapes, and visual effects, each with a playground.
• Accessibility — Foundations first (WCAG in plain words, semantic HTML, lists, alt text), then hands-on HTML labs, a full pass of WAI-ARIA 1.3, and a final fix-the-page challenge.
• Docker — What an image is, why layers cache, and how containers find each other on a network.
• Redis — Fifteen Redis patterns, one picture each: cache, session, lock, counter, rate limiter, queue, and when Redis is the wrong tool.
• Kubernetes — Kubernetes from zero to operators: why it exists, Pods, Deployments and Services, then configuration, networking, storage, scheduling, scaling and security.