What is database sharding and what strategies are used to partition data?
Take a moment.
Say your answer out loud or write it down first — retrieval is what makes it stick. Reveal when you're ready.
Short answer
Sharding splits a database into smaller partitions (shards) distributed across multiple servers so no single machine holds all the data. Strategies: range-based (contiguous key ranges), hash-based (hash the shard key for even distribution), and directory-based (a lookup table maps keys to shards). Sharding trades single-server simplicity for horizontal scale, at the cost of cross-shard queries, rebalancing, and operational complexity.
Detailed explanation
When a single database server hits its storage or throughput ceiling, you shard — splitting rows across servers so each holds a subset. The shard key determines which row goes where. Range-based sharding assigns contiguous ranges of the shard key (e.g., user IDs 1–1M on shard A, 1M–2M on shard B). Simple but risks hot shards if access is skewed toward one range. Hash-based sharding applies a hash function to the shard key and maps the result to a server. Distributes evenly but makes range queries hard and adding servers requires rehashing (consistent hashing helps). Directory-based sharding uses a lookup table that maps each key to its shard; flexible but adds a single point of failure and an extra hop. Cross-shard operations are the real pain: a query that spans shards must fan out, merge, and deduplicate results (scatter-gather); transactions across shards need distributed coordination (2PC or sagas). Joins across shards are impractical, so you denormalize or redesign access patterns. Rebalancing (adding/removing shards) is disruptive without consistent hashing. In practice, many teams avoid application-level sharding by using managed solutions: Vitess for MySQL, Citus for PostgreSQL, or cloud-native sharded stores like DynamoDB and Spanner that hide the mechanics.
Example
A messaging app shards conversations by conversation_id using hash-based sharding:
hash(conversation_id) % num_shards → shard number.
All messages for one conversation live on one shard, so fetching a chat history is a single-shard query.
A time-series metrics DB shards by time range: 2024-Q1 on shard-1, 2024-Q2 on shard-2. (Risk: latest shard is hottest.)Interview tip
Name all three strategies and the trade-off each optimizes. Then name the real cost: cross-shard queries and transactions. Mentioning a managed sharding solution (Vitess, DynamoDB) shows you know when to avoid building it yourself.
Common mistakes
- Describing sharding without choosing a shard key.
- Not mentioning cross-shard query difficulty.
- Forgetting that adding shards requires rebalancing.