Design S3-like Object Storage (System Design Interview)

Separate metadata from data, the write and read paths, packing small objects, replication vs erasure coding with a failure lab, multipart upload, LIST by…

Designing object storage: common questions

Is erasure coding better than replication?

It uses far less space for the same protection (4+2 costs 1.5x versus 3x), but reads and repairs touch more disks. Systems usually replicate hot new data and erasure code it once sealed.

How does S3 give read-after-write consistency?

Bytes are written first and the metadata row is committed last. That commit is the moment the object becomes visible, so any read after a successful PUT finds it.

Related chapters

Keep learning

Previous: Design a Digital Wallet (System Design Interview)
Next: Games in System Design
All System design chapters

Overview

3 copies Erasure 4+2 Erasure 8+4 Lab · store one 1 GB object, then fail disks Disks: click to fail or restore Click disks to fail them. Each chunk sits on a different disk, ideally in a different rack or zone. disk failures it survives 1 disk read from on a normal GET This is why repair speed matters: durability depends on rebuilding a lost chunk before more failures pile up. Read any surviving copy. Repair copies 1 GB from a healthy copy to a new disk. Simple and fast to read, but you pay for three full copies. system design · database-heavy designs · object storage Design S3-like object storage Store any number of files of any size, keep them for years without losing a single byte, and serve them over HTTP. The trick is that object storage is really two systems: a database that knows about each object, and a large pool of disks that hold its bytes.

1. Requirements and numbers

Objects are immutable You replace an object, you never edit bytes in the middle. This one fact makes caching, replication and erasure coding much simpler. Designed for 99.999999999% (“eleven nines”) a year: lose about one object in 100 billion. Availability around 99.99%. 100 PB, tens of billions of objects, from 1 KB thumbnails to 5 TB backups. user data raw disk with 3 copies raw disk with 4+2 erasure coding metadata at ~200 bytes for 50 billion objects

2. Separate metadata from data

Metadata service Bucket, key, version, size, checksum, owner, and where the bytes live. Small rows, lots of lookups, needs transactions and listing: a sharded database. Data service Huge, immutable blobs addressed by an internal object ID. Needs cheap capacity and durability, not queries: data nodes full of disks. Each side scales on its own: add metadata shards when object count or request rate grows, add data nodes when bytes grow. The public key (photos/2026/cat.jpg) never appears in the data layer.

3. The write and read paths

API gateway Data service Metadata DB PUT /photos/cat.jpg with Content-MD5 auth, quota, bucket policy pick a placement group disks in 3+ zones write chunks, verify checksum returns object id 8c1e… INSERT (bucket, key, version) → 8c1e… the moment it becomes visible 200 OK + ETag Bytes first, metadata last. A crash in between leaves unreferenced bytes, which garbage collection removes, but never metadata pointing at missing bytes. A GET is the reverse: look up the metadata row to get the object ID and placement, then stream bytes from the data nodes. Because the metadata commit is the visibility point, a read right after a successful PUT sees the new object: strong read-after-write consistency.

4. Storing bytes on a data node

One file per object fails for billions of tiny objects: the filesystem runs out of inodes and spends its time on directory lookups. Instead, append objects to large write-ahead files (say 1–4 GB). An object’s location is (file id, offset, size). When a file is full, it becomes read-only and is erasure coded as a whole. Append to open file sequential write File reaches 2 GB seal it Erasure code sealed file spread chunks across zones Compact later rewrite files with many deleted objects

5. Durability: copies or erasure codes

Spread across failure domains Chunks of one object go to different disks, servers, racks and zones. Two copies on one server are one copy. Checksums everywhere Checksum each chunk on write, verify on read, and run background scrubbing to catch silent bit rot before a second failure does.

6. Big files: multipart upload

POST ?uploads returns an upload ID. POST ?uploadId=… with the ordered list of parts. The service writes one metadata row pointing at all parts.

7. Sharding metadata, and the LIST problem

Even load LIST by prefix Hot prefix risk hash(bucket, key) range on (bucket, key) range + automatic splits Range sharding keeps a prefix on one shard so LIST is a scan, but keys like logs/2026-10-22/… pile onto the newest range. Split busy ranges automatically. LIST returns keys in sorted order, a page at a time, with a continuation token. It reads only metadata, never data nodes.

8. Deletes, versions and garbage collection

Each PUT adds a version row. DELETE adds a delete marker, so “undelete” is removing the marker. Lifecycle rules Move objects to cheaper tiers after 30 days, expire after a year, clean up old versions.

9. What to say in the interview

Pack small objects into large append-only files addressed by file, offset and size. Replicate hot data, erasure code sealed data, spread chunks across zones, checksum and scrub. Multipart upload for large files, with cleanup of abandoned parts. Related: numbers to know · key-value store · sharding

More courses

Jobs, linked to lessons — Browse jobs where every requirement links to the chapter that teaches it. See how ready you are, how long the gap takes to close, and which listings look stale.
Match a job post to courses — Paste a job post. Each skill is matched to a track here and put in learning order, foundations first.
React from open source with patterns — Patterns extracted from Material UI, Radix, React Aria, React Hook Form, Redux Toolkit, TanStack, and Next.js — not slides about hooks.
Database in depth — How a query becomes work. Indexes, isolation, locks, and the plans that explain why a page is slow — not an ORM tour.
Build a real API with NestJS — One ticket-booking API, built in order: a plain Node server first, then NestJS modules, validation, a real database, and the features real clients lean on.
Microservices — Microservice patterns one diagram at a time: how to split a monolith, how services talk, and how data stays consistent without distributed transactions.
Kafka — A beginner-to-senior path through event logs, reliable delivery, production operations, and disaster recovery.
AI engineering for software engineers — Eight free reads, in order, for software engineers moving into AI: when to build an agent, how to build one, how to test it, and how the model works underneath.
Frontend system design — How a UI is sliced when it is a product, not a page: ownership of state, cache, routes, and what happens when the network lies.
TypeScript in depth — The type system as a design tool. Inference, narrowing, generics, and the errors that mean the model is wrong — not a catalog of utility types.
CSS in depth — See the box, then the page. Visual chapters from the box model through Flexbox surprises, Grid, selectors, and units, then backgrounds, shapes, and visual effects, each with a playground.
Accessibility — Foundations first (WCAG in plain words, semantic HTML, lists, alt text), then hands-on HTML labs, a full pass of WAI-ARIA 1.3, and a final fix-the-page challenge.
Docker — What an image is, why layers cache, and how containers find each other on a network.
Redis — Fifteen Redis patterns, one picture each: cache, session, lock, counter, rate limiter, queue, and when Redis is the wrong tool.
Kubernetes — Kubernetes from zero to operators: why it exists, Pods, Deployments and Services, then configuration, networking, storage, scheduling, scaling and security.