---
title: 'Object Storage in System Design Interviews w/ Ex-Meta Staff Engineer'
source: 'https://youtube.com/watch?v=RvaMHMxHjp4'
video_id: 'RvaMHMxHjp4'
date: 2026-08-04
duration_sec: 756
---

# Object Storage in System Design Interviews w/ Ex-Meta Staff Engineer

> Source: [Object Storage in System Design Interviews w/ Ex-Meta Staff Engineer](https://youtube.com/watch?v=RvaMHMxHjp4)

## Summary

This video explains object storage (blob storage) in the context of system design interviews, covering why traditional databases are unsuitable for large files, how object storage works under the hood, and best practices for using it in system design.

### Key Points

- **Definition of Object Storage** [00:02] — Object storage is a database designed for large files, commonly referred to as binary large objects (blobs), such as videos, photos, music files, JSON files, and large text files. These are large collections of bytes, often megabytes in size.
- **Why Not Store Blobs in Traditional Databases?** [00:46] — Relational or OLTP databases are built for small, frequently changing records that need rich queries and joins. They are not designed to handle large, mostly static files, leading to performance issues, memory pressure, and slow queries.
- **Example: Storing Images in PostgreSQL** [01:13] — PostgreSQL packs rows into 8KB pages. A 4MB image would span 500 pages, causing overhead. Simple queries like fetching top 50 users become slow due to large blobs, and replication and backups are negatively impacted.
- **Backup and Restoration Issues** [02:36] — Large blobs in backups bloat them, making restoration take hours instead of minutes, which is critical during downtime.
- **How Object Storage Works** [03:28] — Files are stored on cheap storage nodes. A metadata service uses an index to locate files. When a client requests a file, it queries the metadata service, which returns the server location, and the server streams the file back directly.
- **Flat Namespaces** [05:05] — Object storage uses flat namespaces, not folder trees. Buckets and folder-like patterns are UI sugar; under the hood, files are stored with a single string key, enabling fast direct lookup.
- **Immutable Writes** [05:46] — Files in object storage cannot be modified in place; you can only create new versions or overwrite entirely. This eliminates locks and race conditions, making operations simpler, faster, and cheaper.
- **Redundancy and Durability** [06:11] — Every object is replicated or erasure-coded across multiple servers, racks, and often data centers to achieve 11 nines of durability (99.999999999%). Losing a node is not a problem; data is fetched from another node automatically.
- **Best Practice: Store Metadata in Database** [07:03] — Large files should be stored in object storage, but metadata should be in a traditional database. This allows quick queries for metadata and then fetching the large files from object storage. Example: social media posts store text in DB and photo URL in object storage.
- **Pre-signed URLs for Direct Upload/Download** [08:52] — Clients can upload/download directly to/from object storage using pre-signed URLs, avoiding routing through your server. This saves bandwidth and avoids server bottlenecks. The client requests a URL with permissions for a limited time.
- **Multipart Upload** [10:14] — Large files are uploaded in chunks (e.g., 5MB each) and stitched together by object storage. This is called multipart upload. It overcomes HTTP size limits and allows parallel uploads.
- **Popular Object Storage Services** [11:33] — Amazon S3 is the most popular, followed by Google Cloud Storage and Azure Blob Storage. They all support pre-signed URLs and similar features.

### Conclusion

Object storage is essential for handling large files in system design, offering scalability, durability, and cost-effectiveness. Key takeaways: store blobs in object storage, keep metadata in a database, use pre-signed URLs for direct transfers, and leverage multipart upload for large files.

## Transcript

definitely think of it as one. Think of it as a database that's designed specifically for large files. And these large files are commonly referred to as binary large objects or blobs. But in reality, all that they are are things
like videos, photos, uh music files, JSON files, large text files. And each of these, if you were to actually open your terminal and look at, for example, a photo, what you would find is that they're just large collections of bytes,
there's some oftent times in the order of megabytes amount of bytes that need to be stored somewhere. And that's where object storage comes into play. Now, the somebody's mind when they're learning about object storage for the first time
is why don't we just store this in a quote unquote normal database? Well, the answer is that normal databases, where normal here means a relational or an OOLTP database, they're built for these small frequently changing records that
need rich queries and things like joins. And so, they're not made to handle large, mostly static files like the ones that we looked at above. So, let me show you with this example cuz I think that this will be pretty illustrative.
photos right alongside all of the other user information that's in your user to you. You have users, you're storing their roles, their names, all of these different things, and then additionally their profile image. But Postgress, we
can use as our example of a relational database here, packs all of these rows internally into 8 kilobyte pages. And so image like we have here, that's going to span 500 whole pages. And now you might
why is that so bad? Why is having 500 pages for an image bad? Well, I want you to imagine a simple query like this where you just want to pull in the top 50 users. Well, even though you only want a small piece of the data, having
these large images stored alongside all of the user data is going to create a ton of extra overhead. The database has to manage and potentially access these huge files, these many megabytes of pages that we have now. And this is
going to impact things like performance, increase memory pressure, slow down what increase memory pressure, slow down what should be uh fast and simple queries. need to consider replication as another issue. When you replicate data across
multiple database servers for either backups or scaling, that four megabyte blob has to be copied to every single replica with each write, which is going to create a ton of lag and consume massive bandwidth. Then there's the
notion of backups more specifically where these 4 megabyte images are going to be included in those maybe nightly backups that you run. And so you're going to turn then in a restoration process which should only take maybe a
so you can picture yourself, you're working, you're up late, your database has gone down, gone down, your users are frustrated, and you're trying to restore from a backup. But your backup is huge because it's bloated with all of these
large static files. And so the restoration of your database is going to take many hours. So the bottom line here is pretty straightforward. traditional databases, these relational or OOLTP databases, they choke on blobs both in
cost. And so hopefully this has made it these directly in your traditional database, this is where object storage comes into play. You should be storing them strictly speaking in your object or
ahead and take a quick look at what's happening under the hood of this diagram is a obviously a severe accurate and incredibly useful I think for for considering how this works but
it is of course simple. The key is that all of your files are stored just on these cheap storage nodes. This is meaning that they are stored on disk in servers in some racks in some data warehouse somewhere. Now the question
becomes how do we know where our file is? And so when a client wants to get or storage the first thing that they're going to do is that they're going to make a request to a metadata service. And this metadata service which is part
of blob storage is part of object storage is responsible for using an index to look up where that file is located. And so in our case it's located one is. The metadata service will say oh server A holds file one. Now in reality
servers. This is important for redundancy which we'll talk about in a second. But at least one copy of file one exists in server A. And so that's what we're going to do here. Now, server A's responsibility then is to stream
this file, those bytes of this file back to the client. And this might be via a pre-signed URL or via the gateway. But in either case, we're going to stream directly from the server right back uh to the user. It's really that
straightforward. It's a simple lookup, right? A connection and then a stream of the file back to the user. But the important bit is what makes this so And there's three things that I want to call out. The first is these flat name
spaces. And so unlike your your local machines where you have to navigate through these these folder trees to find your files, object storage uses just a thinking to yourself, wait a minute, I've used S3 and there's buckets and
there's kind of these folder slash patterns. That's all just UI sugar in order to make it easier for us as humans to use. In reality, under the hood, it's just storing that single string, which allows us to directly locate the file by
name, uh, making it incredibly fast to look up. We don't have to navigate a tree. The second thing is is these immutable rights. And so, unlike databases where you're constantly updating records, uh, in this case, you
can't modify files in object storage. You can't change bytes in the middle of the file. You can only create new versions, which is really common, or overwrite the file entirely. And so this is important because it eliminates the
needs within the the database engine for locks and race conditions which is going to make things a lot simpler, faster, and ultimately cheaper. And then the last one to call out here is redundancy. And so I talked about how file one in
our case doesn't only exist on server A, right? It might also exist on server B or server C. And the reality is that every single object, every single file is either fully replicated or something called eraser coded across multiple
different servers, multiple different racks, oftentimes even across multiple different data centers in cases. And we this is in order to achieve what we call this is in order to achieve what we call in the industry as 11 9 of durability.
in the industry as 11 9 of durability. And so it means that this is durable 99.999 whatever percent durability. If you lose one node, no big deal. The data
is still there. We can go fetch it from another node automatically and heal up in the background. That ultimately is the short and sweet of the high level here. So, let's keep moving and let's zone in on three things in particular
know in the context of your system design interview. And so, the first one might seem really obvious to you at this point. It's that large files should be stored in object storage, but your metadata should be stored in your
might seem obvious, but you'd be surprised. I've had quite a few junior candidates or me mid-level candidates in interviews keeping their metadata also in object storage. Um, strictly speaking, this isn't um, you know,
impossible, but it's against the industry standard for for a number of here. You want the ability to quickly query for your metadata and then pull in those larger files just from object storage. So a really simple example here
site. You have posts on that social media site. A user creates a post and text. Think of it like Twitter. And so we're going to store all of that in our database along with the link, the URL um
to our file, the photo that's on this post, which is stored in object storage. return all this metadata and then they Talk more about that in a second. And so some examples here, photos, videos like
I just mentioned for a messaging or a social app. Um user uploading files for Dropbox. You're going to store those files in object storage where the files metadata will be in the database. We also store static assets like um CSS and
JavaScript that are needed in order to render our website. This gets stored in object storage. it ends up being fronted by a CDN as well, but don't worry too much about that if that's confusing. Uh, we store logs, log files, output log
files from our application in S3 machine learning training data. This is a that you're going to use to train an LLM. Perfect thing to store in object Now, the second one here, and I kind of
little bit, is that you can actually upload and download directly from S3 via what are called pre-signed URLs. And so, at least in the upload case, if you want to upload a file, you might think, I'm going to upload that file to my server,
correct permissions, is going to upload directly to object storage. But the extra bandwidth. You have to go through your server first and then go to object store. And if your server um you know isn't particularly well scaled, it might
not be able to handle incredibly large files, especially if there's an influx of them. It's wasteful. And so what typically happens in the industry is request from your object store what's called a pre-signed URL. And this is
just going to be a URL that your client can use to put HTTP put basically to upload that file directly to the object store. So we're going to ask permission. We're going to say, I want to upload a file of this size, of this type. Can you
give me a URL that's available for the next 30 minutes or an hour or so that I permission to upload directly to you. This way, the client gets a direct line waste the bandwidth going through here. The same story on downloads. So on
downloads, we can get the S3 URL. Whether it's signed or not depends on your permissions, but in any case, you can just download or stream that file having to go through your server. This comes up a lot in system design
interviews. And then the third one here is that large files are going to be uploaded in chunks and then we can stitch them back together. And so this is something that's called multi-art upload. Traditionally, at least S3 calls
it this. But the important thing to note here is that there's limitations on the here is that there's limitations on the size of a file that can be posted on the internet, right? An HTTP post or put. And this is limitations sometimes in the
browser, in gateways, in the servers themselves, they exist all over. Um, in S3's case, the most popular object storage, which we'll talk about here in a second, their limit is 5 megabytes, I believe. Don't quote me on that. I'm
sure folks in the comments will tell me if I made a mistake there, but about 5 megabytes. And so what that means is that if we wanted to upload a, you know, 10 gigabyte file, even a 1 gigabyte file, we're going to have to chunk it
into these 5 megabyte chunks and then upload each of those chunks. It could be in parallel, they could be sequentially, but the client's going to take the file, chunks, upload each of those chunks to the object store, and then our object
storage is going to stitch those chunks together back into a full file once it has everything. And this is called multi-art upload again. Now, maybe the last thing to show here, what are some examples of popular object
or blob stores that exist in the industry? By far, these are the three main ones. Amazon S3, this is the most popular in your interview. If you've S3. If you want to learn a little bit about one of them, S3 is the most
into. But, of course, Google Cloud has their own. Google Cloud Storage, uh, Microsoft Azure has Azure Blob Storage. They all work very similarly. They all have these notions of pre-signed URLs, signed URLs for downloading. They all
to know. Awesome. Okay. Um, tried to make that quick. It's a highle overview. This is the quick nitty-gritty of what you interviews. Hopefully, this was useful. If you enjoyed this, we have a lot more
content like this that is high level, as well as tons of content that goes really deep on specific technologies and specific system designs over at hellin.com. If you haven't been over there, check it out. The overwhelming
majority of the content is free and I think you'll really enjoy it. Thanks for watching and best of luck with your upcoming interviews. Till next time.