The #1 Scaling Answer Candidates Fumble
42sTaps into the fear of failing interviews with a common yet tricky question, offering a clear 'aha' moment.
▶ Play ClipAI Summary
This video explains the concept of sharding for scaling database writes to handle millions of concurrent requests, a common topic in system design interviews. It covers the importance of choosing a good partition key, the trade-offs between read and write scaling, and the need to consider how many shards a request touches.
Chapters
[00:02]
Sharding as the answer
Sharding is the expected answer for scaling to millions of concurrent writes. It involves splitting a large database into multiple servers, where each server handles a portion of the writes.
[00:16]
Linear scaling example
If one server handles 1,000 writes per second, 10 servers can handle 10,000 writes per second, demonstrating linear scaling.
[00:31]
Partition key importance
The partition key is the field hashed to determine which shard a row lands on. A good key spreads writes evenly, while a poor key (e.g., partitioning by country) can lead to hotspots like a busy China shard and idle New Zealand shard.
[00:57]
Read vs. write trade-off
Sharding for writes may complicate reads. If writes spread perfectly but every read must hit all 100 shards, a new problem is created. Caching can help with read scaling, but it's crucial to ask how many shards a request touches and how often.
[01:11]
Conclusion and next steps
If you choose a partition key well and consider consequences, you're in good shape. This is part 2 of a series on scaling writes; more content is available on hellointerview.com.
Effective sharding requires careful partition key selection and awareness of its impact on read patterns. This video provides a concise overview for system design interview preparation.
Mentioned in this Video
Study Flashcards (4)
What is the expected answer for scaling to millions of concurrent writes?
easy
Click to reveal answer
What is the expected answer for scaling to millions of concurrent writes?
easy
Click to reveal answer
Sharding
00:02
If one server handles 1,000 writes per second, how many writes per second can 10 servers handle?
easy
Click to reveal answer
If one server handles 1,000 writes per second, how many writes per second can 10 servers handle?
easy
Click to reveal answer
10,000 writes per second
00:16
What is the partition key used for?
medium
Click to reveal answer
What is the partition key used for?
medium
Click to reveal answer
It is the field hashed to decide which shard a row lands on.
00:31
What is a potential downside of sharding for writes?
medium
Click to reveal answer
What is a potential downside of sharding for writes?
medium
Click to reveal answer
It may make reads harder if every read has to hit all shards.
00:57
💡 Key Takeaways
💡
Sharding is the key
Establishes the core concept that sharding is the standard solution for scaling writes.
00:02
🔧
Partition key pitfalls
Illustrates with a concrete example how a poor partition key can cause uneven load.
00:31
⚖️
Read/write trade-off
Highlights the often-overlooked consequence of sharding on read performance.
00:57Full Transcript
[00:02] you to scale to millions of concurrent rights, there's one answer almost every interviewer will expect, but most candidates fumble. Sharding. Picture a social app where every post, like, and view is written to one big database.
[00:16] That one box can't keep up anymore, so we go to split it into multiple servers. If one server handles a thousand rights a second, 10 servers handles 10,000. immediately expect another question from your interviewer. How do you pick the
[00:31] partition key? That's the field you hash to decide which shard a row lands on. If you pick this well, the rights spread evenly, which means higher utilization and better scaling. If you pick poorly, like
[00:44] partitioning by country, and your China shard is on fire while New Zealand sits idle. Sharding is not a free lunch. If you sharded this all for a write bottleneck, you may have made your read problem
[00:57] harder. If writes spread perfectly, but every read has to hit all 100 shards, you've created a new problem. That problem might be easier to solve. Caching is a great answer for most read scaling, but you need to always ask
[01:11] yourself how many shards does one request touch, and how often. evenly, and look ahead at the consequences, you're in good shape. This is part two, and we've got a lot more on scaling
[01:24] rights. Follow for the rest, and view the whole scaling rights pattern breakdown on our website, hellointerview.com.
