API Gateways in System Design Interviews w/ Ex-Meta Staff Engineer

[00:02] understand an API gateway, it makes sense for us to first chronicle its recent history. And so we can start with the 2000s. In the 2000s, most production monolith. This meant that the client would hit a single URL which would route

[00:17] server which handled every single feature and talk to the database when architecture and it was really easy to reason about. But then fast forward to the 2010 2012 range. These are loose

[00:32] the industry where we slice that monolith into a bunch of different microservices. Hence microser architecture. But now this client has an issue that it ran into. It has to either know the URL of every single one of the

[00:47] microservices and when to call each of them respectively or alternatively it would route all of its requests to a single microser like microser one and determine when to forward that request on to the other

[01:00] microservices. Either of these approaches was clumsy. Now the big issue ever wanted to change some simple routing logic we would need to redeploy routing logic we would need to redeploy an entire microser. This wasn't ideal.

[01:13] And so fast forwarding then just one year 2013 to 2014ish area. This is when we introduced the first generation API gateways by putting a thin layer in front of those microservices. And so now clients could stick to just one

[01:27] endpoint like in the beginning and it would be the endpoint of the API gateway whose responsibility is to determine which micros service that request should be routed to and then route it to the appropriate microser. And so at this

[01:41] point now our routing issues are solved. But there was one more problem and that's that each of these microservices here had all of this repeated shared amongst all of them. Things like

[01:55] authentication, rate limiting, logging, all of this was repeated. And so the realization, and this has kind of continued through the mid2010s all the way until now, was that we can kind of put some of this shared middleware into

[02:09] the API gateway. So all of these repeatable chores could live in this thin layer. And we taught the gateway to do things like terminate TLS, verify O tokens, throttle abusers, uh log metrics. It can even cache some common

[02:22] responses that can all happen here along with that routing. And so now every single team just focus on shipping pure business logic and the client only needs a single URL and we have our gateway in the middle which is kind of that shared

[02:36] guard rail for our entire platform. And so this is where we ended up. This is where we are today with an API gateway that sits in front of our microservices responsibility is still to handle that routing uh but it also handles a bunch

[02:50] of standard middleware operations as well. Now we can take a look at what is actually happening inside that API gateway. Let's zoom in just a little gateway. Let's zoom in just a little bit. And so here is my little nifty

[03:03] drawing of the API gateway. We have all of our clients making requests. Those requests are being forwarded on to each of our services respectively. And then see within the API gateway, it's responsible for four main things.

[03:16] middleware, routing to the correct services, and then transforming the response. And so let's quickly go by or go through each of these one by one. And so the first is that request validation. And so this just takes in the incoming

[03:29] sure that it has the proper formatting. Does it have the right headers? Does it have the right body if needed? If not, we can immediately reject it. If it does, we move on to step two. Step two is where we run any of that middleware

[03:41] times this involves us actually making a request to an external service. Like maybe if we're handling rate limiting, we have Reddus over here. and the API gateway makes a request out to Reddus or in the case of O maybe there's an

[03:55] in the case of O maybe there's an external O service over here, right? Uh those connections as we need to, but be mindful every single request is doing this. So we want it to be fast and then we route it to the correct service. So

[04:09] this is again that most important bit. So we would look up in a routing map and here so you can see it. This would just be a basic config like this which maps paths from the API route to the location of the services. And so we would look up

[04:24] in this map say I just got a uh request for slash messages route that needs to go to the messaging service. see how that works here using this config. And then the messaging service or whichever service is going to do its thing,

[04:38] then returning that response back to the API gateway at which point the API gateway may need to transform that response. And so for example, if any of these were using a different protocol like RPC, gRPC, something like this, and

[04:52] our clients are using REST, then what would happen is that we would take in that RPC response, transform it back into valid JSON in the form of that's expected uh in the REST response and then return that REST response back to

[05:07] our clients. Let's take a quick look at some of the most popular API gateways today. And so you have two classes here. You have the managed API gateways. These are part of your managed cloud solutions like in AWS, Amazon's API gateway in

[05:22] Azure, Microsoft Azure. You have Azure's API management. I'm sure that Google don't know it off the top of my head. alternatives. So things like Kong, Tyke, Express Gateway. These are some of the

[05:35] of these technologies, you've already used an API gateway. Um, okay. As we wrap up here, the main thing that I want you to have as a takeaway from this is when you're in a system design interview, put your API gateway down and

[05:51] move on. Don't spend a lot of time on it. The reality is that with a microser architecture, an API gateway is an expectation. It's largely taken for granted. And so, put it there. Explain that it handles routing, maybe some

[06:04] middleware, but keep moving. The only mistake you could make is to spend too important part of your design. So, it's good to understand, but it shouldn't be the focus. All right, with that, this was short, sweet, to the point.

[06:17] Hopefully, you guys found it valuable. Um, good luck with your upcoming Um, good luck with your upcoming interviews, and I'll see you again soon.