L1/L2 Cache Sync in .NET — Step-by-Step Guide & Transcript

[00:02] engineering. And I think it's safe to assume that distributed caching is even more complicated. Now, what do you think happens when we introduce two-level caching and caching validation into the mix? Let me explain. So, let's first

[00:17] understand the problem that we typically use caching for inside of our net applications. Let's say we have an application server and some sort of database where we are storing our data. 90% of all of the applications that we

[00:30] make are some form of CRUD where we try to access the database to get some data. Now depending on how much data we are dealing with, this operation could be relatively expensive. So in order to make this faster and not have to incur

[00:44] the cost of a database query, we introduce another component that we call a cache. So now our flow turns into is this data in the cache? And if it is, we can just return it from the cache.

[00:57] Otherwise we get some data from the database plus we store it in the cache so that the next time when we try to retrieve this it's available inside of the cache. Now on a high level you could have two types of caches what I'm going

[01:10] to call L1 and L2. An L1 cache is typically going to work in memory whereas an L2 cache is going to be distributed. Now inside of net for the in-memory version we have a couple of abstractions usually an immemory cache

[01:25] it comes to a distributed cache we have many implementations of the I distributed cache interface and usually deserving to use some sort of external storage typically radius or you could even use posgress or any other database

[01:39] that implements this interface. Now, what's also interesting is that starting from .NET 9, we have a new abstraction called a hybrid cache that implements both an L1 and an L2 cache inside of the same abstraction. Now, if your

[01:53] application has just one application server and a single database and cache, you probably won't run into too many issues. But where this becomes problematic is when you have, let's say, multiple application servers and you're

[02:07] still using one database and one cache behind the scenes. Each of our application servers is going to have a local in-memory cache, what we would call the L1 cache. And we have to note that the L1 caches are separate from the

[02:22] something like Reddius. Now if you are using something like hybrid cache, you access something on the first application server, it doesn't find it

[02:34] in the cache. So it reaches out to the database to fetch this value and then it's going to store it inside of the L1 cache, but it's also going to store it inside of the L2 cache. So now what happens is you have this on your first

[02:48] server, but not the other three when it comes to the level one cache. Now, if your next request lands on the level one cache, you're going to run into a cache miss on the L1, but it's going to find a cache hit on the L2 and it won't reach

[03:00] out to the database to fetch this data. And the same happens when you reach out to the other application servers. So, eventually these values are going to sync up. Everything is going to work fine for a given period of time

[03:12] depending on how long your cache duration is. Now, let's say we get some update in the database and this forces us to also invalidate the cache. Let's us to also invalidate the cache. Let's say again we have an invalidation from

[03:25] write some value into the database and then it's going to attempt to invalidate the L1 cache and the L2 cache. Invalidating the distributed cache is going to work fine. So is invalidating the L1 cache. But the problem that we

[03:39] also have to solve in a distributed system is invalidating the level one cache in all of the application servers regardless of how many of them we have. So to do this, you need some sort of communication channel between your

[03:52] application servers. This is typically called a back plane and all of our application servers are going to connect to it to facilitate the communication between them. Now to make this a bit easier to digest, let me move the back

[04:05] plane over here. And what we want to happen in case of a cache invalidation event is our first application server sending some sort of message to the back plane. and then the back plane has to take care of notifying the other

[04:18] application servers that they should also clear their level one caches. Now, as of recording this video, the official hybrid cache library does not implement this. So, I'm going to show you a simple way of how you could implement a

[04:30] backplane yourself using just radius and the stack exchange radius library. So, here's my setup in the Aspire app host. I'm adding a radius resource and I'm configuring my API with five replicas. This is going to help us test out

[04:43] invalidation across a multi-node scenario where we have multiple level one caches and one level two cache inside of my net application. Things aren't going to be too complicated. We have a server registration adding the

[04:56] expiration times for the local and local expiration time is going to be 1 minute and the distributed one is going to be 5 minutes. So, this can be helpful if you don't want to use a back plane,

[05:09] but it kind of just masks the overall problem without solving anything. You can make this shorter if you're concerned with the local cache running scenario, this can be just a couple of seconds, let's say five or 10 seconds.

[05:23] glorified level two cache where the level one cache gets rarely used. So, how you really want to solve this is with a back plane. And for this we can use Reddus and the channel communication mechanism that Reddus implements. It's

[05:36] basically a publish subscribe mechanism where you can fire off a message and it gets fanned out to any existing subscribers. And a simple way to implement this is using a background service. Now as far as my public API

[05:48] surface goes, I've got two endpoints. I'm using the hybrid cache abstraction which exposes a get or create async method. And in case of a cache miss, I'm to fetch the information that I'm

[06:00] this value will be cached for a given period of time. I also have a delete endpoint allowing me to test out cache invalidation. And this is going to get my subscriber using a connection multiplexer. And it's going to publish a

[06:13] message to a reddish channel that I called cache invalidation. The value going to be the key that we want to invalidate. Now, this is a little inefficient. I'll explain why in a moment. Now, let's take a look at our

[06:27] background service. So here we again need the connection multiplexer which and then use throughout your application. And what's going on in the execute async method is we are subscribing to any messages that come

[06:40] from this cache invalidation channel. Our callback gives us access to the going to use for invalidation. And in this case I'm using hybrid cache to remove this value asynchronously which I'm also doing in the delete endpoint

[06:54] which you can see here. And this is why I mentioned a slight inefficiency in this implementation for two reasons. First is we're going to call remove async on the level two cache twice where we are really only concerned with

[07:07] if you want to ensure that you only remove this from the local cache, you can inject an instance of the immemory cache in here. You can comment out this code and you can say memory cache remove and just pass in the key that you want

[07:21] to invalidate. This is going to achieve the same result. And the second inefficiency appears because of how Reddish channels work. When you publish a message to a channel, it's going to be broadcast to all existing subscribers,

[07:35] which means the same service that published this message is also going to consume it. So what you could do is assign some sort of unique ID to each publisher. And when you consume this, you can check if you are consuming your

[07:47] own message. And in that case, you can just ignore it. Now I'm going to go back to the previous implementation because it's going to generate some distributed traces for us and let's run this. We're going to see the Aspire dashboard spin

[07:59] up and we have our Reddis instance and five backend instances of our movies API. I'm going to jump over to the distributed traces view and on my second curl requests. I'll start by sending a get request to fetch a movie from this

[08:15] API and you can see the share with a trace appear and a few moments later a couple of additional traces which represents storing the result of this API call in our radius instance. We know this was a cache miss because there's a

[08:29] trace here reaching out to the external service to fetch the data. Now if I call this again you can see that we don't have a trace for the external service because we can satisfy this endpoint from our cache value. So if I call this

[08:41] additional traces show up and occasionally we're going to also see a request to Reddis because we landed on a different API instance and we have to populate the L1 cache. Now if I go ahead and invalidate the cache key for this

[08:56] traces show up here and this is our background service running behind the scenes consuming this publish here and processing the message from a reddish channel where we execute a remove operation in the L1 and L2 caches. If we

[09:10] you can also see the logs from our background service on each of our API instances stating that we are clearing a cache for a given key. Now, if you don't want to write this code yourself, you can always use an existing solution and

[09:23] one of those that I recommend is fusion cache, which also has a hybrid cache implementation. So we can still use the hybrid cache API surface while implementation which is more feature complete in my opinion and also supports

[09:36] a back plane that you can plug in using Reddus and it's going to implement more you're interested in learning more about building distributed systems, I recommend checking out this video next. If you enjoyed the video right now about

[09:49] cash invalidation in a distributed scenario, leave this video a like. Thanks a lot for watching and until next time, stay awesome.