What Is Redis Really About? Why Is It So Popular?

[00:02] so many system design problems? And why do so many teams rely on it in production? Reddus is versatile. It's worth the time to learn it well. In this video, we focus on three ideas. How Reddus executes commands, how it stores

[00:17] and persists data, and how people use it in real systems. Let's start with what Reddus is. Reddus is a single threaded in-memory data structure server. This short description highs three important design choices.

[00:31] First, single thread execution. Reddus processes commands one at a time in a single thread. Strictly speaking, Reddus 6 and newer added IO threads for networking, but the actual command logic still runs sequentially. The order is

[00:46] predictable. The first request in is the first request processed. There are no locks to reason about and no concurrent writes to the same key. If one command blocks, every other command waits behind it. You might ask, if radus uses only

[01:01] one thread to run commands, how does it stay fast? Reddus hides latency with batching. Clients can bundle commands with pipelining while rep set of commands in the transaction. One network round trip now carries many commands.

[01:15] The single thread still executes each command in order, but the socket stays busy instead of waiting for each request response pair. Second, Reddus keeps data response pair. Second, Reddus keeps data in RAM. The upside is very low latency.

[01:29] Reddus can respondse in sub millisecond time even when we send hundreds of commands per second. The downside is durability. If the machine dies, the data in memory is gone unless we configure persistence carefully. This

[01:43] trade-off is central to how teams use Reddus. We'll come back to this. Third, Reddus is a key value store that exposes data structures directly. A value in radius can be many things. For example, it can be a string, a list, a hash, a

[01:59] set, a sort of set, or a stream. The protocol is small and simple, but Reddus provides many specialized commands for each data structure. Here's a simple example of a counter. We call set counter five. Get counter returns five.

[02:15] Increment counter bumps it atomically to six. Each command acts on a key. Atomicity matters when multiple clients touch the same key. Because Reddus runs commands one at a time, increment completes as a single atomic step.

[02:31] Multiple clients incrementing the same counter won't interfere with each other. Now let's move on to persistent and durability. Because Reddus lives in memory, we need to plan for crashes. Different teams make different choices.

[02:45] Many teams run Reddus as a pure cache with persistence turned off. The database is the source of truth. Rights go to the database. Reddus only stores cached results. If radus crashes in this setup, we lose the cache. The

[03:00] application rebuilds it on demand by quering the database. No important data is lost because we never treated radus as the source of truth. Some teams skip this persistence but add replicas. The primary note handles all rights.

[03:14] Replicas handle read traffic. If the primary dies, a replica is promoted to replace it. In this case, we may lose availability for a few second during failover, but we preserve most of the cache data in memory on the replicas.

[03:28] This trace this IO overhead for memory overhead. Each replica roughly doubles the memory footprint. Another option is to enable RDB snapshots. We configure Reddus to take a snapshot every few minutes. On restart, Reddus loads the

[03:43] snapshot into memory. This allows radius to come back with warm data instead of an empty cache. We accept losing the rights that happened after the last snapshot. For a cache workload, this is usually a reasonable tradeoff because

[03:56] the cache warms up quite quickly. When reddus holds data we cannot afford to lose, we turn on that penon file. A common configuration is a pen only. Yes. With append fsync every sack. Reddus appends each write to a log on disk. The

[04:11] operating system flushes buffer right to disk roughly once per second. In a crash, we lose at most 1 second of data. We can configure f-sync after every command for a stronger durability, but that is much slower and has a big impact

[04:26] on throughput. Most teams avoid that unless the data sets is small and latency is not critical. In practice, the real question is whether rather should hold critical state at all or whether it should just be used as a

[04:38] cache. Many teams keep critical data in a durable database and rely on Reddus a durable database and rely on Reddus mainly for caching and ephemeral state. That covers persistence. Next, let's talk about scaling Reddus. Most teams

[04:52] start with a single Reddus instance. On decent hardware, a single nook can handle a large number of operations per second for many workloads. When read become the bottleneck, the first scaling step is to add replicas. The primary

[05:05] nook accepts all rights. Replicas serve read traffic. This increases read throughput. But write throughput is still kept by the primary. When right volume grows too large for a single instance, many teams adopt client side

[05:19] sharding. We run multiple independent reddus instances. The application hashes each key and picks a reddus note based on the hash. For a static cluster, a simple modular hash works. For dynamic scaling, libraries such as katama use

[05:35] consistent hashing so that adding or removing a note does not reshuffle all keys. Each note here is independent. There is no cross, no coordination and no distributed protocol between Reddus servers. It would treat Reddus as a

[05:49] cache and no failure simply causes cache misses for the subset of keys that live on that node until the cache repopulates. Reddus cluster also exists as an option. It provides automatic sharding and failover. But this comes

[06:03] with added complexity. Operating and debugging a reddus cluster setup is also more complex than running independent nodes. Because of this, many teams prefer simple client side sharding for cache workloads and only adopt Reddus

[06:16] cluster when they needed specific guarantees. Now let's review some common reddus workloads. Classic use case is caching. A service checks reddus before hitting the database. If there's a cache hit, we

[06:29] returns the value immediately. If there is a cache miss, we query the database, store the result in Reddus, and return the response to the client. Over time, the cash fills up, we need a cleanup strategy. One approach is to set a time

[06:43] to live on each key. After the TTL expires, the key stops being returned and is cleaned up lazily. Another approach is to configure a memory limit and an eviction policy so that Reddus evicts keys, for example, the least

[06:59] recently used one when memory is full. Multiple service instances can share counters through Reddus. We use atomic increment commands on keys that represent a user, an IP, or an API token. Combined with TTLs or lure

[07:14] scripts, we can implement various ray limiting algorithms without adding a separate coordination service. There are many nuances here, so we dedicate an entire video to this topic. Check the link in the description. So far, Reddus

[07:29] looks like a fast key value store. The data structure server part becomes powerful when we look at features like sort leaderboards are a common example. A sort of set maintains items order by score. We can insert a player with a

[07:44] score, update a score when it changes, query the top end players, or look up a player's rank. This operation typically run in algorithmic time relative to the size of the set. This pattern generalizes well. We can build trending

[07:59] post lists, most active users, top sellers, and many other top end ranking problems on top of sort of set. Let's wrap up. Reddus is fast, predictable, and versatile. Single thread execution keeps behavior simple to reason about

[08:15] inmemory storage delivers very low latency. Native data structure such as hashes and sort of sets solve problems that would be clunky to implement in a relational database. Most teams start with a single instance at replicas for

[08:29] read traffic and availability. Then use client side sharding when they need more right throughput. When we understand these trade-off around execution, persistence and data structures, Reddus fits cleanly into our system designs.

[08:44] interview? Join our community where we offer comprehensive courses on system design, coding, behavioral questions, machine learning, and object-oriented machine learning, and object-oriented design. Learn more at byitebico.com.