Distributed Rate Limiting with Redis — Step-by-Step Guide & Transcript

[00:02] work correctly is a hard problem to solve. You realize once you reach a certain scale that some of the assumptions you had no longer hold. And a great example of this is rate limiting. Let's say you have one API

[00:15] instance, you can probably get away with storing your rate limits in memory provided you have enough of it. But once you scale out to multiple API instances, this is where you run into a problem where multiple services need to agree on

[00:29] what is the correct value for a rate limit. And this is where a distributed video, I'm going to show you how you can solve this problem very easily using Redis. The first question I want to answer is why is rate limiting

[00:43] important? Why should you spend your developer time on this? And what makes this an interesting problem to solve in a distributed system? Well, a rate limiter typically serves a couple of purposes. First of all, it protects your

[00:56] API from potentially malicious users. For example, someone could do a denial of service attack against your API by simply sending a large number of requests, overloading your systems, and preventing your API from serving

[01:09] legitimate users. Then we want to ensure fair usage between our users, and rate limiters also solve this problem by distributing the allowed number of requests over some time window. We're going to talk about some of the rate

[01:21] limiting algorithms a bit later into the video, but essentially a rate limiter is going to guard the resources on your API server and distribute them as evenly as possible between your users. And then the last point, and I'm also noting that

[01:34] are more use cases and benefits to rate limiters, but let's say your API is offering some sort of paid service. Let's say you've got subscription tiers or you're offering some sort of credits or tokens, a rate limiter is a good way

[01:48] for enforcing this paid usage by applying a certain rate limit based on the user subscription level or the number of credits or tokens that they have to use with your API. Now let's spend a moment to understand why rate

[02:01] limiting isn't a trivial problem once you scale out to multiple service instances. So let's say we have an application server. I'll place it here. Then we've got our users, and of course our users are going to send some sort of

[02:14] requests to our API server. Now let's say we want to enforce a limit or for example, let's say 100 requests per minute. This is something we consider fair usage for our application server. If we have a single application server,

[02:28] we can definitely get away with using the built-in rate limiting support that we have inside of .NET. And how it works is by storing the available rate limits inside of your application server's memory. So let me add a block to

[02:41] represent this. I'm going to color it green for example, and then this is going to represent our memory. And inside of here, among other things, we are also storing our rate limits. Let me add a box around these just to make it

[02:53] obvious that this is all part of one instance. Now what happens if we scale out to multiple instances? So now our users can send requests to both of our API servers, but we still want to enforce this global rate limit of 100

[03:08] requests per minute. But now we have two application servers, and both of them their respective memory. And this is where our initial assumption comes crumbling down as we can no longer reliably enforce that requirement with

[03:20] the design that we currently have in place. So what can we do? Well, we need something where we can store our rate limits. And a great option for this is using Redis because it has a couple of interesting advantages. So what Redis

[03:34] gives you, among other things, is fast per key reads because it stores the data inside of memory, and this is very efficient to query from. It also gives you built-in expiration support where you can set a time to live value for

[03:48] implementing the various rate limiting algorithms where most of them rely on some sort of time window to enforce a given rate. And then another important aspect is that we can store our rate limits inside of Redis, and this lets us

[04:03] remove them from the memory of our two application servers, which now need to be able to talk to our Redis instance so that they can enforce this requirement. Now there are a number of popular rate limiting algorithms. The simplest one

[04:17] being the fixed window rate limiter where we only allow a given number of requests in a given time window. Let's say 100 requests every 1 minute is a simple example of this. The disadvantage of this algorithm is that it can allow

[04:29] bursts on the edges between two time windows. So a user could potentially send 100 requests at the end of one time window and at the beginning of another, effectively sending double the requests in a short amount of time. The sliding

[04:42] windows solves this by constantly sliding the time window and ensuring a more consistent number of requests in a given time window. Then there's also the token bucket algorithm, the leaky bucket algorithm. Another popular one is the

[04:56] concurrency rate limiter. And as I said, we have some of these available natively inside of .NET. However, the limitation is that they only work in memory. So can implement these rate limiting algorithms to make them work in a

[05:09] distributed system using Redis as our database and some simple .NET code. To make our implementation more realistic, I've partnered with Redis, who is the sponsor of today's video, and I'll be using their Redis Cloud offering to

[05:22] create a fully managed Redis instance in a matter of minutes, and we're going to use it from inside of our .NET application. So let me jump into the Redis Cloud console. And if it's your first time logging in, you should see

[05:34] something like this where you can click on new database to create your first Redis instance. So I'm going to do that. And if you also want to code along with me, you can get started with Redis Cloud by clicking the link that's going to be

[05:46] in the pinned comment right below. So we have a couple of options to choose from here. The Essentials and Pro plan are the prominent options. And the between if you want to use shared infrastructure for your managed Redis

[05:59] instance or dedicated infrastructure, which of course give you more control. However, if you just want to try this out, you can also create a free Redis instance that comes with 30 megabytes of memory, which is the option that I'm

[06:12] going to choose here. Let's give our database a name. I'm going to call it TB version of Redis you want to use. I'll be using the latest one. And you also get to pick between which cloud vendor you want to use for your managed

[06:27] instance. I'll use AWS and just leave US East as the region of choice. And then we get an overview of what we get inside of our instance. So we have 30 megabytes of RAM, up to 30 connections, and 100 Redis operations per second. Now if you

[06:41] want more, you can explore either the Essentials or the Pro plan. And here I very interesting. And if we scroll down to the plan details, you can obviously let's say you need more memory, you can go up to 100 gigabytes of RAM. But

[06:56] notice how the plan changes from RAM only to RAM plus SSD. This is their Flex plan, which is something I was really excited to show you as it's an excellent trade-off between more memory versus slightly lower performance. So what you

[07:11] slightly lower performance. So what you get with the Flex plan is a mix of RAM plus SSD for storing your overall data. You can keep your hot keys inside of RAM for fast access, and then offload the lesser used keys into SSD, which is a

[07:26] little slower. If you ever need those keys, your managed Redis Cloud instance is going to automatically find the respective key even if it's on the SSD without you having to change anything in your code. Now let's say we want 1

[07:41] gigabyte of memory. You can see that the Flex plan for this starts at just $6 a month. And then we get to choose if we want to have high availability with a primary and a read replica. You can also choose just a single managed instance,

[07:53] single instance. Or you can choose the RAM only plan, which starts at 20 bucks a month for 1 gigabyte of memory. Now going back to our free plan, I'm going to go ahead and click create database,

[08:06] my instance, which completes surprisingly quickly. This should be minute. And in a couple of moments, our Redis instance is up and running, and we can connect with from our .NET application and use it to implement our

[08:21] distributed rate limiter. Now I wanted to show you how you can connect to your Redis instance. So there's the option here to connect to your Redis database. If you click connect, you can choose what you actually want to use to connect

[08:33] to your instance. The options are Redis Insight, Redis CLI, and an SDK client. And the default option, which I like very much, is .NET using the StackExchange.Redis library where you get a nice little code snippet showing

[08:46] you how you can set up your Redis connection using StackExchange.Redis. So let's finally move into the code, and let me show you how we can implement our rate limiter using Redis. So I've got a fresh .NET 10 API, and I want to get

[09:01] started by installing the StackExchange.Redis library. So let me look for this NuGet package. I'll browse for Redis, and I want to install the latest version of StackExchange.Redis. Let me go back to my program file now,

[09:15] and what we want to do is to register a singleton instance of an IConnectionMultiplexer. Now I don't want to be hard coding our connection values, so I'll create a folder called rate limiting. And inside

[09:30] of it, I'm going to drop in an options object, and it's going to contain our configuration values, which are going to be our host, port, and the username and password to authenticate with our Redis instance. We'll provide these values

[09:42] through our app settings file, but let's see how we can use this. We have an overload here where we have access to the service provider, and this allows us to get our Redis options. So I can say service provider, get required service,

[09:56] and we're looking for I options of Redis options. We're interested in the value, and this lets us initialize the configuration options. If you remember the code snippet from our Redis Cloud dashboard, it contains much of the same

[10:11] things. And here we can provide our Redis endpoints if we have multiple of them, or in our case we just want to set the host and the port where our single Redis instance lives. Then we can provide our username and password. And

[10:26] finally, we can return an instance of the connection multiplexer by calling the connect method and passing in our configuration options. Don't forget to register your Redis options as an I options implementation, and you can do

[10:39] so by saying builder services configure, passing in Redis options as the argument our configuration section, where we're going to use the constant that we have on the Redis options class for the section name. Then we need to set these

[10:54] values inside of app settings, and I went ahead and dropped them in just to save us some time. And these are the values from our Redis Cloud dashboard. So I've got the host address, the port, the username, which is default, and then

[11:06] I'm going to set the password inside of my user secrets because I don't want this to be part of source control. Realistically, you also want to keep the host name out of here. I'm just placing it here to make this demo simpler. If

[11:18] you right-click your project and go to manage user secrets, you can add an option here with Redis and then password. And then this is where you're going to enter your real password. I'm going to do this off the screen and then

[11:31] save this. Now inside of our app settings, we also may want to control our rate limiter options. And let's say we want to have a limit of five requests in a 10-second time window. Now we can enforce this with different rate

[11:46] limiting algorithms, and I'm going to start by implementing the simpler one, which is going to be a fixed window rate limiter. So what do we need to implement connection multiplexer, which we're going to use to get our database

[12:00] And then I'm also going to drop in our rate limiter options class, which we're going to inject using I options. And this is going to fetch our limit, representing the number of requests in a given time window, which we're going to

[12:14] represent with a time span. So then we need our database instance, which is available as I database, and this represents our Redis instance. And we multiplexer get database. Then I'm also going to store the limit, and we can get

[12:29] this from our options instance. And I need one more field for the time span, representing our time window. And we're going to get this also from our rate limiter options. And then we need to implement our actual rate limiting

[12:43] method, which can simply return a boolean value saying if the request for a given key is allowed or not. Now in order to also make this reusable, I'm interface that's going to contain a single method

[12:59] with the signature. And we're going to make our fixed window rate limiter to let me switch out my implementations later when I introduce an additional rate limiter implementation. So how do we implement a fixed window rate

[13:14] limiter? Remember that this should only allow a given number of requests, represented by our limit, for the duration of a single time window. So we window, and we can do so by creating a

[13:27] unique key. We should assume that the value provided here is globally unique, and typically this is going to be something like a user identifier, maybe recommend that if you have access to the user's identifier. So let's create a key

[13:41] for our Redis instance, and I'm going to prefix this with rate limit then fixed, and then I'm going to append our unique key. And the simplest way we can implement this is using a counter inside of Redis. So Redis actually has a bunch

[13:56] of data structures that you can use for implementing a lot of various scenarios, rate limiting in this example, but you can also do things like leaderboards. default use case. What's also interesting is that Redis has a lot of

[14:10] capabilities for implementing AI features, but this is definitely a topic for a future video. So back to our counter, we can call the string increment async method, where we can pass in our key, and this is going to

[14:22] increment our counter in Redis by one, and also return the current value of the counter. Now if it doesn't exist, the default value will be zero, and if we get back a count equal to one, this means that this is a new key. So in this

[14:37] case, we want to set a time to live for our key, and for this we can call key expire async, pass in the Redis key, and then I can pass in the time window, which is going to determine when the current time window expires. Now if the

[14:53] count is greater than our limit or the allowed number of requests, we want to return false, as we don't want to allow this request. And in the end, we return true, which means this request is allowed as it's within the permitted

[15:07] rate limit. So you can see the fixed window rate limiter implementation is fairly straightforward, and we can just use a counter and setting a time to live value for our unique key. Now let me also enhance our implementation with

[15:19] some logging just to make it more production ready. And then our next step is to actually use this for enforcing our rate limit for let's say our weather forecast endpoint. And I'm going to do this using middleware. So I'll add a new

[15:33] type here that's going to contain a simple extension method, which I'm going to call use Redis rate limiting. And this will allow us to provide which implementation of the I rate limiter we want to use for enforcing our rate

[15:46] limits. We could make this more flexible by introducing some configurable values, same algorithms. Let's say 100 requests per minute for this endpoint and then endpoint. And let me walk you through

[16:00] the implementation here. So we're going to say app.use, which is one of the simplest way to define a middleware, and you provide a delegate with an HTTP context and the next middleware in the chain. Now we want to resolve our rate

[16:12] limiter from our services, and we can use a generic argument here because we have our generic constraint forcing the implementation to be an I rate limiter. For the client key, I'm going to use an IP address. But as noted, in a

[16:26] to be using something like a user identifier, which means your user first has to authenticate with your app before they can be rate limited. I'm also going to resolve the rate limiter options, and then we're just going to call our rate

[16:39] limiter and let it decide if we want to serve the current request. If not, we return a 429 response, which means too many requests. We can also set the retry after header, and here I'm using rate limiter options to set the correct value

[16:52] for the retry after header. And then we're just going to return from our middleware. Otherwise, we're going to serve the request until the user reaches the rate limit. So let me set this up. We have to register our rate limiter

[17:05] implementation. So I'll say builder services.add singleton, and we have our fixed window rate limiter. And then we can include our middleware, so I'll say app.use Redis rate limiting, and we're going to use our fixed window rate

[17:20] limiter implementation to enforce the rate limit. So let's start our API, and Postman. And you can see we hit a breakpoint inside of our middleware. So we're going to resolve our fixed window limiter from our application services.

[17:36] We'll set the client key, which is just the localhost IP address, and we want to call our is allowed async method. So we're going to define our unique key, counter for this key. Now because it doesn't exist, it's going to be zero.

[17:50] get back the value of one. And this means that this is a new key, so we want to set a time to live value for this key value, and it's going to be equal to our fixed window, and then we're going to allow this request and return true. So

[18:05] Postman. Now if I send this request a couple more times very quickly, we're going to exceed our rate limit and hit the next breakpoint I set, where the count value is equal to six, and our limit is five. So we're going to return

[18:19] false, and this is going to result in a 429 response saying that we sent too interesting, if we look at the response headers, you can see that the retry after header is present, and it has a

[18:32] value of 10, which means you should wait for 10 seconds before trying to send another request. Now while I was babbling this, 10 seconds elapsed, so if I send another request, we get a 200 OK response. Now as I said, the fixed

[18:44] window rate limiter is the simplest one to implement and understand how rate limiting with Redis works. Let me show you a more advanced example using the sliding window rate limiter. Now you'll see our dependencies are much of the

[18:56] same. We're getting a Redis instance, our limit, and our time window value. implementation? We still create our unique key. This time I'm using sliding as the rate limiter algorithm, but we have to use a different approach here.

[19:10] Now Redis has a very interesting data structure called a sorted set, where for a given key, you can store a set of values in a sorted array. And we can use this to implement a sliding window rate limiter by storing the timestamp for the

[19:24] current request under our unique key, and then based on the current timestamp, we can calculate the beginning of the time window, which is this value here represented by the window start variable. Then we can call this method

[19:36] sorted set remove range by score async. And what it does is simply remove all of the values that are outside our current time window and are no longer relevant for rate limiting. Then we check how many requests are there in the current

[19:50] time window, and if that's equal to our rate limit, then we can't satisfy this request and we return false. Otherwise, we have to add some unique value, so we can generate it using a GUID, and we want to add this to our sorted set using

[20:03] our unique key. Then we pass in the request ID as our value, and then this is the critical part. We add the timestamp as the score value for our sorted set entry. This means that our request ID is going to be sorted in the

[20:17] set according to this timestamp, which means we basically only append it at the end, and we continue doing so with every subsequent request. We also want to refresh the expiration for our key, and then we return true. Now, while both of

[20:29] our implementations are effectively functional, and they're going to achieve our desired results in most scenario, they suffer from one problem, and that's the fact that these operations aren't atomic. So, in a very high concurrency

[20:42] scenario, we may run into some problems where a user could potentially exceed their allowed rate limit. Now, the way to solve this is using Lua scripting on your Redis server, and you can also do this using the Stack Exchange library,

[20:55] implementation, I'm going to leave some documentation links below this video in the description where you can check them out. Now, let's register this as a rate limiter, so I'm going to say sliding window rate limiter. Let's replace our

[21:09] current rate limiter with the new one, and then let me show you how this works. If I send a request from Postman, we're going to hit our breakpoint, and then we can calculate the current timestamp and when our current time window started,

[21:22] and we want to first remove any entries in our sorted set that are outside this window. Now, you can see that we currently don't have any entries in our set, so we can allow this request, but we have to make sure to add the request

[21:36] ID for the current request into the set, also update the expiration time for our key before returning true. Now, if I send this request very quickly a couple of times, we're going to run into our rate limit and get back a 429 too many

[21:50] request response. The retry after header will have a value of 10, and then we can wait for this to expire before trying again. So, that's how you can implement a sliding window rate limiter and a fixed window rate limiter using Redis

[22:02] supports. For some more advanced examples, make sure to take a look at description of this video. If you want to get started with Redis Cloud and get a managed Redis instance in a matter of minutes, go ahead and click the link

[22:17] right below this video. And if you want to see how you can use Redis to implement distributed messaging, go ahead and watch this video next. Consider gently smashing the like button if you enjoyed this video. Thanks a lot

[22:30] for watching, and until next time, stay awesome.