Redis

Among the vast landscape of database technologies, Redis stands out not for solving one problem exceptionally well, but for solving many problems remarkably well. This versatility makes Redis invaluable in system design, both in production systems and technical interviews. Rather than learning dozens of specialized tools, mastering Redis provides solutions for caching, rate limiting, distributed locking, leaderboards, geospatial indexing, message queuing, and real-time communication. Understanding Redis deeply—its capabilities, limitations, and trade-offs—equips engineers to design scalable systems that handle diverse challenges with a consistent, well-understood foundation.

What Makes Redis Special: Redis describes itself as a “data structure store,” and this characterization captures its essence perfectly. Unlike traditional databases that force you to model everything as rows and columns or documents, Redis provides familiar programming data structures—strings, hashes, lists, sets, sorted sets, streams—as first-class primitives accessible over the network. This makes reasoning about Redis operations intuitive. If you understand how a hash map works in memory, you understand how Redis hashes work at scale. If you grasp priority queues, you grasp Redis sorted sets. This conceptual simplicity, combined with extreme performance, makes Redis approachable yet powerful.

Two architectural decisions define Redis: it’s in-memory and single-threaded. The in-memory nature delivers microsecond read latencies and handles hundreds of thousands of operations per second on modest hardware. The single-threaded execution model eliminates complex concurrency bugs and makes operation semantics straightforward—commands execute atomically in the order received. These choices create constraints—limited dataset size based on available RAM, no parallelism within a single instance—but the trade-offs favor speed and simplicity over flexibility. For the vast majority of caching, coordination, and real-time data scenarios, these trade-offs are exactly right.

The durability question looms large with Redis. Being in-memory and optimized for speed means Redis doesn’t offer the same persistence guarantees as traditional disk-based databases. Redis provides persistence mechanisms—RDB snapshots and Append-Only Files—but these trade performance for durability. The default configuration prioritizes speed, accepting potential data loss on crashes. For use cases where losing seconds or minutes of data is catastrophic, either configure more aggressive persistence, use managed services like AWS MemoryDB that provide disk-backed durability, or recognize Redis isn’t the right tool. For caching, session storage, real-time leaderboards, and similar workloads where some data loss is acceptable, Redis excels.

Core Data Structures and Operations: Redis organizes data as a key-value store where keys are strings and values can be any of Redis’s supported data structures. This fundamental model means every piece of data in Redis has a unique key identifier. Choosing effective keys is critical—keys determine data distribution across cluster nodes, affect memory usage, and influence query patterns. Well-designed key naming schemes like user:{userId}:sessions or product:{productId}:inventory make data organization clear and enable efficient operations.

The command interface uses a simple text-based protocol that’s both human-readable and efficient. Commands follow intuitive patterns based on data structures. For strings, SET assigns values, GET retrieves them, and INCR atomically increments numeric values. For hashes, HSET sets field values, HGET retrieves them, and HGETALL fetches entire objects. Sets support SADD for adding members, SISMEMBER for existence checks, and SINTER for computing intersections. The full command set is extensive but organized logically by data type, making it discoverable and memorable.

Sorted sets deserve special attention as they enable surprisingly sophisticated use cases. Unlike regular sets, sorted sets associate each member with a numeric score and maintain members in score order. This enables efficient range queries, rank lookups, and leaderboard operations. Operations like ZADD insert members with scores, ZRANGE retrieves ranges by rank, ZRANGEBYSCORE retrieves ranges by score, and ZRANK finds a member’s position. These operations execute in logarithmic time, making sorted sets perfect for leaderboards, priority queues, and time-series data with timestamp scores.

Beyond basic data structures, Redis supports probabilistic data structures like Bloom filters for space-efficient set membership tests, geospatial indexes for location-based queries, and time-series data structures for efficiently storing temporal data. HyperLogLog provides approximate cardinality counting using minimal memory—perfect for tracking unique visitors or events at massive scale. This breadth of specialized structures means Redis often has a purpose-built solution for common system design challenges.

Infrastructure Configurations and Scaling: Redis can run as a single node, with high-availability replicas, or as a cluster. Single-node deployments are simple but lack redundancy and scale limitations. High-availability configurations use leader-follower replication where one primary node handles writes and multiple replicas synchronously or asynchronously replicate data. If the primary fails, a replica can be promoted automatically, minimizing downtime. This configuration provides redundancy and read scaling through replica distribution but doesn’t help with write scaling or dataset size limitations.

Redis clusters partition data across multiple nodes to scale beyond single-node memory and throughput limitations. Clusters use hash slots—16,384 of them—that map keys to specific nodes. When clients connect to a cluster, they retrieve the hash slot mapping and cache it locally. Subsequent operations go directly to the node owning the relevant hash slot, eliminating central coordinators that could bottleneck requests. If a client requests data from the wrong node, the server responds with a MOVED redirection and the client updates its cached mapping.

This clustering model is deliberately simple, which creates important constraints. Most significantly, Redis generally expects all data for a single operation to reside on one node. Cross-node operations are either impossible or require special patterns like hash tags that force related keys to the same node. This means scaling Redis requires thoughtful key design—grouping related data under keys that hash to the same slot. For example, storing all data for user:123 under keys like user:123:profile, user:123:sessions, user:123:preferences ensures related data stays together if you use hash tags like user:{123}:profile.

The gossip protocol maintains cluster state awareness. Nodes periodically exchange information about which nodes are responsible for which hash slots and which nodes are healthy. This eventually consistent view means clusters can operate during network partitions, though clients might briefly connect to stale nodes before redirections update their mappings. For most applications, this trade-off between availability and consistency is acceptable, especially given Redis’s focus on speed over strict consistency guarantees.

Performance Characteristics: Redis’s performance profile is extraordinary, routinely handling 100,000+ operations per second on single instances with read latencies measured in microseconds. This performance transforms architecture patterns—operations that would be unthinkable with traditional databases become viable. Firing off 100 individual requests to Redis to build a complex view isn’t ideal, but it’s often acceptable where the same pattern with SQL databases would be catastrophic. This changes how you think about query optimization and batching.

The in-memory nature makes this performance possible. Disk I/O is the bottleneck for most databases—even SSDs require milliseconds for random access versus nanoseconds for RAM. By keeping all data in memory and using simple data structures optimized for cache locality, Redis eliminates the fundamental performance constraint other databases face. The single-threaded model contributes as well—no lock contention, no context switching overhead, just pure sequential execution of commands.

However, understanding Redis performance requires recognizing algorithmic complexity. While simple operations like GET, SET, HGET, and LPUSH execute in constant time, operations on larger data structures can be expensive. KEYS, which scans all keys matching a pattern, is O(N) where N is the total number of keys—potentially millions—and will block all other operations on that instance. Use SCAN instead for production systems. Similarly, operations like SMEMBERS that return entire sets or ZRANGE with large ranges can be costly. Always consider the size of data structures and operation complexity when designing Redis usage patterns.

Redis as a Cache: Caching is Redis’s most common use case, and it excels here. The fundamental pattern maps cache keys to values stored as strings, hashes, or other appropriate structures. For caching product details, you might use keys like product:{productId} with values stored as JSON strings or Redis hashes containing fields like name, price, and description. Retrieving cached data becomes a simple GET or HGETALL operation returning results in microseconds.

Time-to-live (TTL) capabilities are essential for cache management. Setting a TTL on keys ensures automatic expiration after a specified duration. Redis guarantees that expired keys won’t be returned by read operations and uses TTLs to decide which keys to evict when memory limits are reached. This makes cache invalidation straightforward—set appropriate TTLs based on how long data can be stale, and Redis handles cleanup automatically. For product catalogs that change infrequently, TTLs of hours make sense. For stock prices updating constantly, TTLs of seconds ensure freshness.

Eviction policies control behavior when Redis reaches memory limits. The allkeys-lru policy evicts least-recently-used keys regardless of TTL, making the cache self-managing. The volatile-lru policy only evicts keys with TTLs set, preserving keys intended to be permanent. The allkeys-random policy evicts random keys. Choosing the right policy depends on your data characteristics and whether all keys should be equally eligible for eviction or only certain temporary keys.

Cache-aside is the standard pattern where application code checks Redis first, queries the backing database on cache misses, and populates Redis for future requests. This gives applications full control over caching logic but requires handling the complexity of keeping cache and database synchronized. Write-through patterns have applications write to both cache and database simultaneously, ensuring consistency but adding latency. Write-behind patterns queue database writes for asynchronous processing, improving write latency but risking data loss on failures.

Distributed Locking with Redis: Distributed locks coordinate access to shared resources across multiple processes or servers, ensuring only one process modifies data at a time. Redis provides building blocks for implementing distributed locks, though the details matter significantly for correctness. The simplest approach uses atomic increment operations—INCR on a key returns the new value, which is 1 if you were first or greater than 1 if someone else acquired the lock. Combined with TTLs for automatic lock release if holders crash, this provides basic mutual exclusion.

More sophisticated locking requires the SET command with options: SET lock:resource unique-identifier NX PX 30000 sets the key only if it doesn’t exist (NX) with a 30-second expiration (PX 30000). The unique identifier ensures processes only release locks they hold by checking the identifier before deletion. When releasing, use Lua scripts to atomically check the identifier and delete the key, preventing race conditions where lock expiration and release interleave incorrectly.

The Redlock algorithm provides distributed consensus across multiple Redis instances for stronger guarantees. Clients attempt to acquire locks on a majority of independent Redis instances using timestamps and unique identifiers. If a majority grants the lock within a time window, the client successfully acquired it. This protects against single Redis instance failures but adds complexity and latency. For many applications, single-instance locks with TTLs and proper identifier handling suffice.

Critical considerations include fencing tokens for preventing race conditions when lock holders experience pauses. If a process acquires a lock, pauses due to garbage collection, and the lock expires and is acquired by another process, both might believe they hold the lock. Fencing tokens are monotonically increasing numbers issued with locks. Protected resources track the highest token seen and reject operations with lower tokens. This requires resource-level support but provides airtight correctness.

When discussing distributed locks in interviews, acknowledge the complexity and edge cases. Distributed locks are notoriously difficult to implement correctly, and many production systems use simpler approaches like database transactions when possible. If your primary database can provide the necessary coordination, prefer that over adding Redis-based locking complexity. Use distributed locks when you need coordination across resources that span multiple systems or when database-level locking creates performance bottlenecks.

Leaderboards with Sorted Sets: Sorted sets make implementing leaderboards trivial compared to traditional databases. Each member has a score, and Redis maintains them in sorted order. For a game leaderboard, ZADD leaderboard 1500 player123 adds player123 with score 1500. ZINCRBY leaderboard 50 player123 atomically increments their score by 50. Retrieving the top players is just ZREVRANGE leaderboard 0 9, returning the top 10 in descending score order. Finding a player’s rank uses ZREVRANK leaderboard player123.

These operations execute in logarithmic time—O(log N) where N is the number of members. This means leaderboards with millions of entries still perform blazingly fast. Compare this to SQL databases where maintaining sorted leaderboards requires indexes that degrade write performance, and complex queries with ORDER BY and LIMIT clauses that can be slow even with proper indexing. Redis sorted sets are purpose-built for this exact use case.

For scenarios like trending posts or search results ranked by relevance and engagement, sorted sets shine. Store post IDs as members with composite scores combining likes, recency, and other factors. ZADD trending_posts:keyword 500 post123 adds a post about a keyword with score 500 representing total engagement. Periodically clean up low-ranked posts with ZREMRANGEBYRANK trending_posts:keyword 0 -101 to keep only the top 100 posts, preventing unbounded growth.

Time-based leaderboards benefit from Unix timestamps as scores. For “most active users in the last hour,” use ZADD active_users timestamp userId when users perform actions. Querying becomes ZRANGEBYSCORE active_users (currentTime-3600) currentTime, retrieving all users active in the last hour. The sorted set naturally ages out old entries, and periodic cleanup using ZREMRANGEBYSCORE removes stale data.

Rate Limiting Patterns: Redis’s atomic operations and TTLs make it ideal for rate limiting. The fixed-window algorithm limits requests to N per time window W. When a request arrives, INCR the key for the user or IP address. If the returned value exceeds N, reject the request. Set EXPIRE on first increment to reset the counter after time W. This is simple and works well, though it allows bursts at window boundaries—if the limit is 100 per minute, a user could make 100 requests at 0:59 and another 100 at 1:00.

Sliding window rate limiters provide smoother limiting using sorted sets with timestamps. ZADD ratelimit:user123 timestamp requestId records each request with its timestamp as the score. Before allowing a request, ZREMRANGEBYSCORE ratelimit:user123 0 (currentTime-windowSize) removes old requests outside the window. Then ZCARD ratelimit:user123 counts remaining requests. If under the limit, allow the request and add it to the sorted set. This provides true sliding windows at the cost of more memory per tracked entity.

Token bucket algorithms allow bursts while maintaining average rates. Store available tokens as a numeric value, incrementing by R tokens per second up to a maximum bucket size B. When requests arrive, DECRBY tokens:user123 1 removes a token. If the result is non-negative, allow the request. Otherwise, reject it. Periodic background jobs or Lua scripts replenish tokens at the configured rate. This algorithm is more complex but provides better burst handling for legitimate traffic spikes.

Distributed rate limiting across multiple application servers requires coordination through Redis to maintain accurate counts. Each application server checks and increments shared Redis counters rather than maintaining local state. This works because Redis operations are atomic—concurrent increments from multiple servers produce correct total counts. The alternative of local rate limiting per server is simpler but less accurate, potentially allowing 10 servers × 100 requests/second = 1000 requests/second when the intended limit was 100 requests/second total.

Geospatial Indexing: Redis natively supports geospatial indexing through commands that use geohashes under the hood. GEOADD locations longitude latitude member adds locations to a geospatial index. For a ride-sharing app, GEOADD available_drivers -122.4 37.8 driver123 adds driver123 at San Francisco coordinates. Finding nearby drivers uses GEOSEARCH available_drivers FROMLONLAT -122.41 37.79 BYRADIUS 5 km, returning all drivers within 5 kilometers of the specified location.

The time complexity is O(N + log M) where N is the number of results and M is the number of items in the spatial index. The logarithmic component comes from Redis using geohashes to narrow candidates to grid-aligned bounding boxes. Since these boxes are square and the query might be circular, a second pass filters candidates to only include items actually within the specified radius. For most practical applications with thousands or tens of thousands of indexed items and queries returning dozens of results, this performs excellently.

Geospatial commands support both radius and bounding box queries. BYRADIUS finds items within a circular area, while BYBOX uses rectangular regions. Additional options like WITHDIST include distances in results, WITHCOORD includes coordinates, and COUNT limits results. These features make Redis suitable for implementing location-based features like “find nearby restaurants,” “drivers within 10 km,” or “users in this neighborhood.”

Updates to geospatial indexes use the same GEOADD command—if a member already exists, its coordinates update. For ride-sharing drivers constantly moving, periodic updates every few seconds keep the index current. Removing members uses ZREM because geospatial indexes are implemented as sorted sets with special encoding. This implementation detail means you can use sorted set commands for operations like removing drivers who go offline.

Event Sourcing with Streams: Redis Streams provide append-only logs similar to Kafka topics, enabling event sourcing and message queue patterns. XADD stream-name * field1 value1 field2 value2 appends entries to a stream with automatically generated IDs containing timestamps. For order processing, XADD orders * orderId 123 userId 456 total 99.99 records order events. Streams maintain these events durably, and multiple consumers can process them independently.

Consumer groups enable distributed processing with exactly-once semantics per group. XGROUP CREATE orders order-processors $ creates a consumer group starting from the latest stream entry. Workers use XREADGROUP GROUP order-processors consumer1 COUNT 10 STREAMS orders > to claim up to 10 unprocessed messages. Redis tracks which messages each consumer group has processed, preventing duplicate processing across the group while allowing different groups to process the same stream independently.

The pending entry list (PEL) tracks messages claimed by consumers but not yet acknowledged. If a worker crashes after claiming messages but before processing them, those messages remain in the PEL. Other workers can claim these pending messages using XCLAIM or XAUTOCLAIM, enabling fault tolerance. This mechanism ensures messages are processed at least once even when workers fail, though applications must implement idempotency to handle potential duplicate processing.

Compared to Kafka, Redis Streams are simpler to operate and sufficient for moderate scale—millions of messages per day. Kafka excels at extreme scale—billions of messages per day—with stronger guarantees, longer retention, and more sophisticated consumer coordination. For many applications, Redis Streams provide the right balance of capability and operational simplicity. In interviews, understanding when the simpler solution suffices demonstrates good judgment about managing complexity.

Pub/Sub for Real-Time Communication: Redis Pub/Sub implements the publish-subscribe pattern where publishers send messages to channels and subscribers receive all messages published to their subscribed channels. SPUBLISH channel message publishes to a channel (the S prefix indicates sharded pub/sub for scalability). SSUBSCRIBE channel subscribes to receive messages. This pattern works well for real-time notifications, chat systems, and broadcasting updates to multiple clients.

The key characteristic of Pub/Sub is ephemerality—messages aren’t persisted. If a subscriber is offline when a message is published, it misses that message forever. This makes Pub/Sub perfect for real-time, transient communication where missing messages is acceptable. Think live sports scores, chat messages in active conversations, or real-time dashboard updates. For scenarios requiring guaranteed delivery or the ability to replay missed messages, Redis Streams or dedicated message brokers like Kafka are more appropriate.

Sharded Pub/Sub, introduced in recent Redis versions, scales by distributing channels across cluster nodes. Previously, all Pub/Sub messages routed through a single node, creating bottlenecks for high-volume applications. Sharded Pub/Sub hashes channel names to determine which node handles them, enabling linear scaling. Clients maintain one connection per cluster node rather than one per channel, making even applications with millions of channels feasible without overwhelming connection limits.

Combining patterns addresses limitations. For durable fan-out where offline subscribers need to catch up on missed messages, use Pub/Sub for real-time delivery to online clients while also writing messages to Streams or a database. Online clients get instant notifications via Pub/Sub, while offline clients query Streams or the database when reconnecting. This hybrid approach provides both real-time delivery and durability without forcing all clients through the slower durable path.

Hot Key Problems and Mitigations: The hot key problem occurs when load distributes unevenly across keys, causing some cluster nodes to handle disproportionate traffic. If you’re caching e-commerce products and one product goes viral, receiving 100,000 requests per second while other products receive 10 requests per second, that single product might overwhelm the node responsible for its hash slot even though the cluster overall has plenty of capacity.

Several mitigation strategies exist. Application-level caching adds an in-memory cache in application servers for hot keys, reducing Redis requests for frequently accessed data. This trades memory in application servers for reduced Redis load and lower latency. Cache invalidation becomes more complex, but for read-heavy hot keys like viral content, short TTLs of seconds still dramatically reduce load.

Key replication deliberately stores the same data under multiple keys, distributing load across nodes. Instead of product:123, create product:123:1, product:123:2, through product:123:10. Clients randomly select one replica, spreading 100,000 requests across ten keys on different nodes. This increases memory usage by the replication factor but makes hot keys tractable. The trade-off is coordinated invalidation—updates must invalidate all replicas.

Read replicas provide another solution. Redis supports read-only replicas that asynchronously replicate data from primary nodes. Routing read traffic to replicas while writes go to primaries distributes load. For extremely hot keys, dynamically scaling replica count based on traffic provides elastic capacity. Managed Redis services often support this automatically, adding replicas when CPU usage spikes and removing them when load subsides.

In interviews, proactively recognizing hot key scenarios demonstrates depth. When designing celebrity social media profiles, viral posts, flash sales, or any system with power-law access distributions, explicitly call out the hot key risk and propose mitigations. This shows you understand not just how systems work in theory but the practical challenges of production systems at scale.

When Redis Isn’t the Answer: Despite Redis’s versatility, it’s not appropriate for every scenario. Strong durability requirements where losing even seconds of data is unacceptable make Redis problematic unless you use managed services with disk-backed durability or configure aggressive persistence at the cost of performance. Financial transactions, order confirmations, and critical state transitions belong in traditional databases with ACID guarantees.

Complex relational queries with joins, aggregations, and filtering across multiple dimensions are better served by SQL databases. Redis supports basic filtering through pattern matching and set operations, but complex analytical queries are painful. If your use case centers on ad-hoc queries across structured data, PostgreSQL or similar relational databases are more appropriate.

Dataset size limitations matter. Redis datasets must fit in memory, making it expensive for storing terabytes of cold data. Disk-based databases or object stores like S3 are more cost-effective for large, infrequently accessed data. Use Redis for hot data accessed frequently—caches, session stores, real-time leaderboards—while colder data lives in cheaper storage.

Strong consistency requirements across distributed operations are difficult with Redis. While Redis provides atomic operations within single instances and some coordination through distributed locks, it doesn’t provide the same consistency guarantees as traditional databases with ACID transactions. If you need serializable isolation for complex multi-step operations, traditional databases are safer choices.

Redis excels as a complement to traditional databases rather than a replacement. Use PostgreSQL or similar for durable, authoritative data with complex querying needs. Use Redis for caching frequently accessed data, coordinating distributed operations, maintaining real-time leaderboards, and handling high-throughput writes before batching to persistent storage. This separation of concerns lets each technology do what it does best.

Redis has earned its place as one of the most valuable tools in the system design toolkit through versatility, performance, and simplicity. Its data structure orientation makes reasoning about operations intuitive, its in-memory architecture delivers microsecond latencies, and its breadth of capabilities—caching, distributed locking, pub/sub, streams, geospatial indexing, sorted sets for leaderboards and rate limiting—means a single well-understood technology solves many common challenges. Success with Redis comes from understanding its sweet spot: high-performance operations on moderately sized datasets where some data loss is acceptable, using it as a complement to rather than replacement for traditional databases. Master Redis’s capabilities, recognize its limitations, and design mitigations for its challenges like hot keys and durability, and you’ll be equipped to build scalable, performant systems that leverage one of the most battle-tested technologies in modern infrastructure.