Video Tqaqdfxi-J4

[00:00] Caching is an idea that developers learn about early in their careers. And at first it sounds simple, but when you sit down to really think about it, caching turns out to be a complex and fascinating topic that's worth discussing in some detail.

[00:13] So before my cache gets busted, let's get started. So what is caching?

[00:25] According to our old friend Wikipedia, a cache is a hardware or software component that stores data so that future requests for that data can be served faster. The data stored in a cache might be the result of an earlier computation or a copy of data stored elsewhere.

[00:40] As an example, when you log into Twitter, you're greeted with a long list of tweets from the users you follow. The list you see has been created beforehand and cached, rather than gathered and assembled on the spot. This ensures a great user experience. Essentially, when your backend

[00:56] systems aren't as fast as you'd like them to be, a cache gives you a holding pen for quick retrieval and thus a faster response time. Response time refers to the amount of time, usually in milliseconds, that an application takes to respond to a user interaction. When

[01:10] caching is part of your architecture, you can often reduce response times from hundreds of milliseconds to less than a millisecond. At the same time, you're reducing the load on your backend services and getting more from your existing hardware. And most importantly,

[01:25] users will benefit from faster interactions with your application So now that you know what caching is and why it important let look how a cache fits in your system architecture and let also review a basic caching algorithm The backend receives a request and then queries a disk data

[01:43] store to retrieve the necessary records. Let's say that each request takes around 75 milliseconds to fulfill. This might not seem like a long time, but remember, this is just one backend request.

[01:55] It's often necessary to make multiple requests to render a given view in an application. 75 milliseconds doesn't give us much wiggle room, especially if we're under load. If we ever need to scale, or if we're lucky enough to go viral,

[02:09] the extra load will add up, creating bottlenecks, blocking, and high latency. And remember, latency is the new outage. Enter the cache. In this case, our cache is a Redis database.

[02:21] Redis stores its entire data set in memory, which makes it an ideal choice for caching. Imagine taking that initial 75 milliseconds response time and reducing it to less than a millisecond. That's quite an improvement.

[02:34] Now when our application receives a request, it first quickly checks the cache to see if that data is already at hand. If so, great. We call this a cache hit. We can send the data back to the user right away

[02:46] rather than accessing the original datastore. If the data is not in the cache, we call this a cache miss. We then process the request to the original backend datastore as normal. This is a great opportunity, though. After adding the result of this request to our cache,

[03:00] subsequent requests result in a cache hit and a much faster response time So our example covered an application requesting data from a database But caches are used for all kinds of services not just legacy relational databases but also microservices and third APIs

[03:17] Why is that? Well, it comes down to speed and cost. Disk reads are comparatively slow when compared to RAM. Even though SSDs are fast, they're still an order of magnitude slower than RAM.

[03:29] So if you perform a query and the data you're looking for isn't in the disk or page cache, then you're going to incur the hit of going to disk. Remember, Redis stores all data and memory, so you'll never incur the extra cost of a lookup having to go to disk.

[03:43] Complex computations, such as those used in machine learning inferences and full text searches, take a lot of time and compute resources, making them good candidates for caching. Certain types of relational database queries, notably joins,

[03:56] are computationally expensive and benefit from caching as well. Let's also not forget slow network connections between your back-end services. This can add up to a considerable delay in response. Lastly, caching responses from third-party APIs

[04:11] reduces costs where API usage is billed on a per-request basis. So caching is useful in a number of situations, and it dramatically decreases your application response times. What's the downside? Well, one challenge of caching is staleness.

[04:25] At some point, your cache is no longer going to represent the system of record. This is known as cache staleness. The strategies to prevent a stale cache depend on your use case, but in simple terms they boil down to passive versus active cache expiration We can set a TTL on each cache entry to passively clear out cache TTL means time to live and it usually is measured in seconds minutes or hours

[04:51] It's like setting an expiration date. After a defined amount of time, the data will be automatically deleted, ensuring we do not store outdated information forever. We could also actively update the cache by replacing existing entries with newer, fresh

[05:05] versions taken from the system of record. This is known as cache busting. It's worth considering whether you really need to add a cache. For certain services, it might make sense to make Redis itself the system of record.

[05:17] This way, the data is always served from memory. And if you take advantage of Redis's persistence and high availability features, you can ensure that your data is still safe and available. We've covered quite a lot of ground here, so let's do a quick review.

[05:31] A cache is a data store that's faster to access in the system of record. Whether that's a relational database, an array of microservices, or a third-party API. Caches store a copy of frequently requested data in memory, saving your application from

[05:44] repeating the same calls to your backend. Caches become stale when they get out of sync with a system of record, so you need to make sure that you employ an exploration strategy to ensure that relevant, fresh information

[05:56] is always in the cache. Thanks for joining me in a quick exploration of caching. Stay tuned for future segments where we cover different caching strategies, geodistribution, replication, and persistence.

[06:08] Keep learning and stay fast!