Idempotency - What it is and How to Implement it

[00:01] channel. So, when designing an API, it's easy to think about the happy path of your application. But, if you want to build a robust application and keep your users happy, then you need to consider the unhappy path as well. What happens

[00:13] if your users' internet connection cuts out just after making a request? Or maybe your server's struggling under load and the client times out before it are handled by the client by doing retries. If there's an error or the

[00:26] request times out, then the client will retry a number of times, either automatically or by a frustrated user hitting that submit button. If the user and more than one of them gets through, then you'll likely end up with

[00:39] application, this might not be a big deal. If the user is just writing a product, then it's not the end of the world if they end up with multiple posts. But, what if they're sending an email? Is it okay for the recipients to

[00:52] email? What if they were buying something online? Would the customer be happy if they got charged twice even 10 times for potency comes in. But, what does it actually mean? If we have a look at the

[01:06] internet standards, it says the following. A request method is considered item potent if the intended effect on the server of multiple the same as the effect for a single such request. So, if we call an API multiple

[01:20] times with the same request, then it's considered item potent if it has the same effect on the server as if we'd only called it once. Now, let's have a whether they're item potent or not. The get method is read-only. It doesn't

[01:33] affect the server in any way. Nothing gets created or updated. So, we can call the get method multiple times without it having any adverse effects. So, a get request, we're going to be performing updates. So, let's say we're updating a

[01:48] times we call this method, we should always get the same response. And the updated to the same thing. So, a put request is always item potent as well. With a post request, we're creating

[02:01] this method multiple times, then we should expect to get multiple copies of might get an error if we're trying to create something that can only be one of. For example, if we're trying to create a new user with a post request,

[02:14] but a second request for the same username will fail because the user still item potent, but it's not a very when they make their first request, they then have an internet connection

[02:27] failure, and therefore don't get a response. When they try and retry, it exists, but in fact, it was their first request that created that user. If we take the example of making a payment, then generally there aren't any unique

[02:40] identifiers on a payment like there are with creating a user. This is what we typically might send through on a platform like Stripe if someone's buying something for $20. However, there is nothing unique about this request that

[02:52] links it to a particular purchase. If a payment platform receives two of these, then it'll just think it's two separate payments and charge the customer twice, which is not what we want. So, post requests are generally not item potent.

[03:04] With patch requests, it depends on how you use them. If you're just using it to update an email address of a user, then that could be considered item potent. However, you can also use a patch request to do things like copy, move,

[03:16] add, and remove, and therefore it's not considered item potent. Finally, we have object on the server. If you run delete multiple times, it's server as if you'd only run it once. Sure, the second time you might run it,

[03:31] server doesn't change its state. Therefore, delete is also considered to be item potent. In the cases where you're making a payment or some other exactly once, then you need to make those other methods such as post and

[03:45] patch item potent as well. The only way that you can really do this is to have some form of unique identifier. That way, you can distinguish that request from some otherwise identical requests. You may have something like this already

[03:58] Let's go back to the payment analogy. Let's say you have a saved basket of items and that basket has an ID. That ID is therefore unique to that order and you could use that ID for the item potency key. Another option is to hash

[04:12] you're doing this, you need to make sure that there's no reason for that person to ever send exactly the same request. There should be something about the that makes it unique. For example, if someone orders a SpongeBob beach towel

[04:26] and then 30 seconds later orders another SpongeBob beach towel, you don't know that that's a duplicate order. They might just really like SpongeBob. is to add an item potency key to a header. Stripe, for example, have a

[04:39] header called item potency-key, but really this could be anything. So, you now have some form of key that makes this request unique. However, with a user-supplied key, you can't guarantee that it's going to be globally unique.

[04:52] The user could send you the same key as another user or even the same key from multiple endpoints. So, it's important you always combine that user-supplied key with something like a user ID as well as the API path that's being used.

[05:05] To implement item potency, you need to store your combined item potency key along with the successful response that you send to the user. If a request with the same item potency key comes in, instead of actually performing the

[05:17] operation, you just send them back the response that you sent out before. As operation was successful and they don't need to know that you're sending out a previous response. Typically, this is done using some form of key-value

[05:29] storage such as Redis or DynamoDB. Technically, you could store your item potency keys in an in-memory storage such as a dictionary. However, this of your API running behind the load balancer. As each API won't be able to

[05:43] see the keys in the other copies. You will also lose your item potency keys application. So, this is why we don't store them in memory. So, when a request comes in, you need to first check to see whether that idempotency key that's been

[05:55] sent is in your storage. And if it is, then you just send back the response that you have stored. If the idempotency key doesn't exist, then you carry on with the normal operation. And if it's successful, then you store the

[06:07] idempotency key along with the response before sending it back to the user. How long you store your idempotency keys for really depends on the application. If protect yourself from a user clicking on something multiple times, then you might

[06:21] only need to store them for say 10 minutes. But if you have some sort of need to store them for longer. Typically, we store idempotency keys for anywhere between 24 and 48 hours, but you need to check with your application

[06:35] for longer. If you are using something like Redis or DynamoDB to store your keys, then you can use the time to live or TTL so that your keys are automatically deleted for you. You may also want to consider idempotency

[06:49] validation in your API. If a request comes in with the same idempotency key endpoint, then we're just going to assume that that's the same request and send them out the same response. However, there is a chance that the user

[07:02] has just used the same idempotency key by mistake, and the request body is protect yourself against this sort of scenario, then you can take a hash of idempotency key. And then when the

[07:15] whether it matches the hash as well. And if it doesn't, then return an error to the user. If you want to implement this in a .NET Core API, then you can make use of the distributed caching library and then store your keys in Redis. And

[07:28] then you can create a filter, which you can then add to each of the endpoints where you want idempotency. To make this even easier, there is a NuGet package called Item Potent API that does all the heavy lifting for you. If you want to

[07:40] to the code showing my videos, then you can subscribe on Patreon. Patreon members also get access to exclusive content from me, as well as a private Discord community, and generous

[07:52] discounts to my courses once I release them. Thank you for watching this video, and I'll hopefully see you in the next one.