Tech - Redis

Posted at 2022-04-29Updated at 2022-05-03 interview  interview tech 

Q & A of redis^[1][2][3].

# Basics

# What is Redis?

Redis is an open-source, advance key value data store and cache. It is also referred as a data structure server which keys not only contains strings, but also hashes, sets, lists, and sorted sets.

Redis, which stands for Remote Dictionary Server, is a fast, open-source, in-memory key-value data store for use as a database, cache, message broker, and queue.
You can run atomic operations, like appending to a string; incrementing the value in a hash; pushing an element to a list; computing set intersection, union and difference; or getting the member with highest ranking in a sorted set.
In order to achieve performance, Redis works with an in-memory dataset. Depending on your use case, you can persist it either by dumping the dataset to disk every once in a while, or by appending each command to a log. Persistence can be optionally disabled, if you just need a feature-rich, networked, in-memory cache.
Redis is a popular choice for caching, session management, gaming, leaderboards, real-time analytics, geospatial, ride-hailing, chat/messaging, media streaming, and pub/sub apps.

# What is the meaning of Redis?

Redis stands for Remote Dictionary Server.

# How is Redis different from other databases?

Redis is a NoSQL, open-source, in-memory data-structure store. It follows the principle of key-value store.

It is extremely fast, persistent, portable and supports many languages.

# In which language Redis is written?

Redis is written in ANSI C and mostly used for cache solution and session management. It creates unique keys for store values.

# What is the usage of Redis?

Redis is a special key-value store database that can function as a NoSQL database or as a memory-cache store to improve performance when serving data that is stored in system memory.

# What are the main features of Redis?

• Redis is very simple to install setup and manage.
• Redis is very fast. It can execute 100000 queries per second.
• Redis is fast because data is being persistent in memory as well as stored on the disk.
• Redis is very fast because it loads the whole dataset in primary memory.
• Redis operations working on different data types are atomic so these operations can be accomplished safely i.e. to set or increase a key, add or remove elements from a set or increase a counter.
• It supports various types of data structure such as strings, hashes, sets, lists, sorted sets etc.
• Redis supports a variety of languages i.e. C, C++, C#, Ruby, Python, Twisted Python, PHP, Erlang, Tcl, Perl, Lua, Java, Scala etc.
• If your favorite language is not supported yet, you can write your own client library, as the Protocol is pretty simple.
• Redis supports simple master to slave replication.
• Redis is portable.

# Explain the Replication Features of Redis?

Replication is important in order to archive high level of availability in big data systems. The data needs to be replicated at n number of places. This follows the master-slave approach where the master copy is maintained by master-slave and replicated to n other nodes.

# What are the advantages of using Redis?

• Redis is very fast.
• It supports a server-side locking.
• It has a rich client side library.
• It is a good counter.
• It supports Atomic Operation.

# What are the disadvantages/limitations of using Redis?

• It is single threaded.
• It has got limited client support for consistent hashing.
• It has significant overhead for persistence.
• It is not deployed widely.

# What is the difference between Redis and RDBMS?

• Redis is a NoSQL database while RDBMS is an SQL database.
• Redis follows the key-value structure while RDBMS follows the table structure.
• Redis extremely fast while RDBMS is comparatively slow.
• Redis stores all the dataset in primary memory while RDBMS stores its dataset in secondary memory.
• Redis is generally used to store small and frequently used files while RDBMS is used to store big files.
• Redis provides only official support for Linux, BSD, Mac OS X, Solaris. It doesn?t provide official support for Windows currently while RDBMS provides support for both.

# Differentiate Memcached and Redis?

# Mention Operation Keys of Redis.

• TYPE key
• TTL key
• EXPIRE key seconds
• EXPIREAT key timestamp
• EXISTS key
• DEL key

# Which are the different data types used in Redis?

• Strings
• Hashes
• Lists
• Sets
• Sorted Sets

# We all know that Reds is fast, but is it also durable?

In Redis, there is always a trade-of between durability and speed. In the case of system failure, it may lose data which is not stored.

# How can you enhance the durability of Redis?

• Whenever a new command is added to the append log file, call Fsysnc() each time.
• Keep calling Fsysnc() in every second. Despite of the 1 second data lose in the case of system fails.

# What are the concerned things while using Redis?

• Consistent method selection in order to name and prefix the keys. Namespace management.
• Make key prefixes registry which can map every documents to their owner applications.
• Designing, implementing and testing the Garbage collection mechanism for each class we keep into the redis architecture.
• Maintain a sharding library before investing so much into the application.

# Does Redis persist data?

Redis supports so-called “snapshots”. This means that it will do a complete copy of whats in memory at some points in time (e.g. every full hour). When you lose power between two snapshots, you will lose the data from the time between the last snapshot and the crash (doesn’t have to be a power outage…). Redis trades data safety versus performance, like most NoSQL-DBs do.

Redis saves data in one of the following cases:

• automatically from time to time
• when you manually call BGSAVE command
• when redis is shutting down

But data in redis is not really persistent, because:

• crash of redis process means losing all changes since last save
• BGSAVE operation can only be performed if you have enough free RAM (the amount of extra RAM is equal to the size of redis DB)

# Is Redis just a cache?

Like a cache Redis offers:

• in memory key-value storage

But unlike a cash Redis:

• Supports multiple datatypes (strings, hashes, lists, sets, sorted sets, bitmaps, and hyperloglogs)
• It provides an ability to store cache data into physical storage (if needed).
• Supports pub-sub model
• Redis cache provides replication for high availability (master/slave)
• Supports ultra-fast lua-scripts. Its execution time equals to C commands execution.
• Can be shared across multiple instances of the application (instead of in-memory cache for each app instance)

# Does Redis support transactions?

MULTI, EXEC, DISCARD and WATCH are the foundation of transactions in Redis. They allow the execution of a group of commands in a single step, with two important guarantees:

• All the commands in a transaction are serialized and executed sequentially. It can never happen that a request issued by another client is served in the middle of the execution of a Redis transaction. This guarantees that the commands are executed as a single isolated operation.
• Either all of the commands or none are processed, so a Redis transaction is also atomic.

# How are Redis pipelining and transaction different?

Pipelining is primarily a network optimization. It essentially means the client buffers up a bunch of commands and ships them to the server in one go. The commands are not guaranteed to be executed in a transaction. The benefit here is saving network round trip time for every command.

Redis is single threaded so an individual command is always atomic, but two given commands from different clients can execute in sequence, alternating between them for example.

Multi/exec, however, ensures no other clients are executing commands in between the commands in the multi/exec sequence.

# How does Redis handle multiple threads (from different clients) updating the same data structure in Redis?

Redis is actually single-threaded, which is how every command is guaranteed to be atomic. While one command is executing, no other command will run.

A single-threaded program can definitely provide concurrency at the I/O level by using an I/O (de)multiplexing mechanism and an event loop (which is what Redis does). The fact that Redis operations are atomic is simply a consequence of the single-threaded event loop. The interesting point is atomicity is provided at no extra cost (it does not require synchronization between threads).

# What is the difference between Redis replication and sharding?

• Sharding, also known as partitioning, is splitting the data up by key. Sharding is useful to increase performance, reducing the hit and memory load on any one resource.
• While replication, also known as mirroring, is to copy all data. Replication is useful for getting a high availability of reads. If you read from multiple replicas, you will also reduce the hit rate on all resources, but the memory requirement for all resources remains the same.

Any key-value store (of which Redis is only one example) supports sharding, though certain cross-key functions will no longer work. Redis supports replication out of the box.

# What’s the advantage of Redis vs using memory?

Redis is a remote data structure server. It is certainly slower than just storing the data in local memory (since it involves socket roundtrips to fetch/store the data). However, it also brings some interesting properties:

• Redis can be accessed by all the processes of your applications, possibly running on several nodes (something local memory cannot achieve).
• Redis memory storage is quite efficient, and done in a separate process. If the application runs on a platform whose memory is garbage collected (node.js, java, etc …), it allows handling a much bigger memory cache/store. In practice, very large heaps do not perform well with garbage collected languages.
• Redis can persist the data on disk if needed.
• Redis is a bit more than a simple cache: it provides various data structures, various item eviction policies, blocking queues, pub/sub, atomicity, Lua scripting, etc …
• Redis can replicate its activity with a master/slave mechanism in order to implement high-availability.

Basically, if you need your application to scale on several nodes sharing the same data, then something like Redis (or any other remote key/value store) will be required.

# When to use Redis Lists data type?

Redis lists are ordered collections of strings. They are optimized for inserting, reading, or removing values from the top or bottom (aka: left or right) of the list.
Redis provides many commands for leveraging lists, including commands to push/pop items, push/pop between lists, truncate lists, perform range queries, etc.
Lists make great durable, atomic, queues. These work great for job queues, logs, buffers, and many other use cases.

# When to use Redis Sets?

Sets are unordered collections of unique values. They are optimized to let you quickly check if a value is in the set, quickly add/remove values, and to measure overlap with other sets.

These are great for things like access control lists, unique visitor trackers, and many other things. Most programming languages have something similar (usually called a Set). This is like that, only distributed.

Redis provides several commands to manage sets. Obvious ones like adding, removing, and checking the set are present. So are less obvious commands like popping/reading a random item and commands for performing unions and intersections with other sets.

# When to use Redis Sets?

Sets are unordered collections of unique values. They are optimized to let you quickly check if a value is in the set, quickly add/remove values, and to measure overlap with other sets.

These are great for things like access control lists, unique visitor trackers, and many other things. Most programming languages have something similar (usually called a Set). This is like that, only distributed.

Redis provides several commands to manage sets. Obvious ones like adding, removing, and checking the set are present. So are less obvious commands like popping/reading a random item and commands for performing unions and intersections with other sets.

# When to use Redis over MongoDB?

It depends on kind of dev team you are and your application needs but some notes when to use Redis is probably a good idea:

• Caching
  Caching using MongoDB simply doesn’t make a lot of sense. It would be too slow.

• If you have enough time to think about your DB design.
  You can’t simply throw in your documents into Redis. You have to think of the way you in which you want to store and organize your data. One example are hashes in Redis. They are quite different from “traditional”, nested objects, which means you’ll have to rethink the way you store nested documents. One solution would be to store a reference inside the hash to another hash (something like key: [id of second hash]). Another idea would be to store it as JSON, which seems counter-intuitive to most people with a SQL-background. Redis’s non-traditional approach requires more effort to learn, but greater flexibility.

• If you need really high performance.
  Beating the performance Redis provides is nearly impossible. Imagine you database being as fast as your cache. That’s what it feels like using Redis as a real database.

• If you don’t care that much about scaling.
  Scaling Redis is not as hard as it used to be. For instance, you could use a kind of proxy server in order to distribute the data among multiple Redis instances. Master-slave replication is not that complicated, but distributing you keys among multiple Redis-instances needs to be done on the application site (e.g. using a hash-function, Modulo etc.). Scaling MongoDB by comparison is much simpler.

# Advanced

# Why is redis so fast?

1. Redis is a RAM-based data store. RAM access is at least 1000 times faster than random disk access.
2. Redis leverages IO multiplexing and single-threaded execution loop for execution efficiency.
3. Redis leverages several efficient lower-level data structures.

# Explain a use case for Sorted Set in Redis

Sorted Sets are also collections of unique values. These ones, as the name implies, are ordered. They are ordered by a score, then lexicographically.

This data type is optimized for quick lookups by score. Getting the highest, lowest, or any range of values in between is extremely fast.

If you add users to a sorted set along with their high score, you have yourself a perfect leader-board. As new high scores come in, just add them to the set again with their high score and it will re-order your leader-board. Also great for keeping track of the last time users visited and who is active in your application.

Storing values with the same score causes them to be ordered lexicographically (think alphabetically). This can be useful for things like auto-complete features.

Many of the sorted set commands are similar to commands for sets, sometimes with an additional score parameter. Also included are commands for managing scores and querying by score.

# How can I exploit multiple CPU/cores for Redis?

First Redis is single threaded but it’s not very frequent that CPU becomes your bottleneck with Redis, as usually Redis is either memory or network bound.

For instance, using pipelining Redis running on an average Linux system can deliver even 1 million requests per second, so if your application mainly uses O(N) or O(log n) commands, it is hardly going to use too much CPU.

However, to maximize CPU usage you can start multiple instances of Redis in the same box and treat them as different servers. At some point a single box may not be enough anyway, so if you want to use multiple CPUs you can start thinking of some way to shard earlier.

With Redis 4.0 Redis team started to make Redis more threaded. For now this is limited to deleting objects in the background, and to blocking commands implemented via Redis modules. For future releases, the plan is to make Redis more and more threaded.

# How would you efficiently store JSON in Redis?

There are many ways to store an array of Objects in Redis:

1. Store the entire object as JSON-encoded string in a single key and keep track of all Objects using a set (or list, if more appropriate):

1
2
3

INCR id:users
SET user:{id} '{"name":"Fred","age":25}'
SADD users {id}

Use when:

• If you use most of the fields on most of your accesses.
• If there is variance on possible keys

Store each Object’s properties in a Redis hash:

1
2
3

INCR id:users
HMSET user:{id} name "Fred" age 25
SADD users {id}

Use when:

• If you use just single fields on most of your accesses.
• If you always know which fields are available
• If there are a lot of fields in the Object
• Your Objects are not nested with other Objects
• You tend to only access a small subset of fields at a time

Store each Object as a JSON string in a Redis hash:

1
2

INCR id:users
HMSET users {id} '{"name":"Fred","age":25}'

Use when:

• really care about not polluting the main key namespace

Store each property of each Object in a dedicated key:

1
2
3
4

INCR id:users
SET user:{id}:name "Fred"
SET user:{id}:age 25
SADD users {id}

Use when:

• almost never preferred unless the property of the Object needs to have specific TTL or something

Option 4 is generally not preferred. Options 1 and 2 are very similar, and they are both pretty common. Option 1 (generally speaking) may be most preferable because it allows you to store more complicated Objects (with multiple layers of nesting, etc.) Option 3 is used when you really care about not polluting the main key namespace (i.e. you don’t want there to be a lot of keys in your database and you don’t care about things like TTL, key sharding, or whatever).

Also as a rule of the thumb, go for the option which requires fewer queries on most of your use cases.

# If there’s a way to check if a key already exists in a Redis list?

Your options are as follows:

• Using LREM and replacing it if it was found.
• Maintaining a separate SET in conjunction with your LIST
• Looping through the LIST until you find the item or reach the end.

Redis lists are implemented as a, hence the limitations. I think your best option is maintaining a duplicate SET. Regardless, make sure your actions are atomic with MULTI-EXEC or Lua scripts.

# What do the terms “CPU bound” and “I/O bound” mean in context of Redis?

• A program is CPU bound if it would go faster if the CPU were faster, i.e. it spends the majority of its time simply using the CPU (doing calculations). A program that computes new digits of π will typically be CPU-bound, it’s just crunching numbers.
• A program is I/O bound if it would go faster if the I/O subsystem was faster. Which exact I/O system is meant can vary; I typically associate it with disk, but of course networking or communication in general is common too. A program that looks through a huge file for some data might become I/O bound, since the bottleneck is then the reading of the data from disk (actually, this example is perhaps kind of old-fashioned these days with hundreds of MB/s coming in from SSDs).
• Memory bound means the rate at which a process progresses is limited by the amount memory available and the speed of that memory access. A task that processes large amounts of in memory data, for example multiplying large matrices, is likely to be Memory * Bound.
• Cache bound means the rate at which a process progress is limited by the amount and speed of the cache available. A task that simply processes more data than fits in the cache will be cache bound.

It’s not very frequent that CPU becomes your bottleneck with Redis, as usually Redis is either memory or network bound.

# What is AOF persistence in Redis?

Redis AOF (Append Only Files) persistence logs every write operation received by the server, that will be played again at server startup, reconstructing the original dataset.

Redis must be explicitly configured for AOF persistence, if this is required, and this will result in a performance penalty as well as growing logs. It may suffice for relatively reliable persistence of a limited amount of data flow.

# What is Pipelining in Redis and when to use one?

If you have many redis commands you want to execute you can use pipelining to send them to redis all-at-once instead of one-at-a-time.

Normally when you execute a command, each command is a separate request/response cycle. With pipelining, Redis can buffer several commands and execute them all at once, responding with all of the responses to all of your commands in a single reply.

This can allow you to achieve even greater throughput on bulk importing or other actions that involve lots of commands.

# When to use Redis Hashes data type?

Hashes are sort of like a key value store within a key value store. They map between string fields and string values. Field->value maps using a hash are slightly more space efficient than key->value maps using regular strings.

Hashes are useful as a namespace, or when you want to logically group many keys. With a hash you can grab all the members efficiently, expire all the members together, delete all the members together, etc. Great for any use case where you have several key/value pairs that need to grouped.

One example use of a hash is for storing user profiles between applications. A redis hash stored with the user ID as the key will allow you to store as many bits of data about a user as needed while keeping them stored under a single key. The advantage of using a hash instead of serializing the profile into a string is that you can have different applications read/write different fields within the user profile without having to worry about one app overriding changes made by others (which can happen if you serialize stale data).

# Why Redis does not support roll backs?

Redis commands can fail during a transaction, but still Redis will execute the rest of the transaction instead of rolling back.

There are good opinions for this behavior:

• Redis commands can fail only if called with a wrong syntax (and the problem is not detectable during the command queueing), or against keys holding the wrong data type: this means that in practical terms a failing command is the result of a programming errors, and a kind of error that is very likely to be detected during development, and not in production.
• Redis is internally simplified and faster because it does not need the ability to roll back.

# How much faster is Redis than MongoDB?

The general answer is that Redis 10 - 30% faster when the data set fits within working memory of a single machine.

# Is Redis a durable datastore (“D” from ACID)?

Redis is not usually deployed as a “durable” datastore (in the sense of the “D” in ACID.), even with journaling. Most use cases intentionally sacrifice a little durability in return for speed.

However, the “append only file” storage mode can optionally be configured to operate in a durable manner, at the cost of performance. It will have to pay for an fsync() on every modification. To configure this, set these two options in your .conf file:

1
2

appendonly yes
appendfsync always

# RDB and AOF, which one should I use?

Redis provides two persistence schemes, RDB and AOF:

RDB: Generates a snapshot of data in Redis memory within a specified time interval, which is a binary file dumpr.rdb

AOF: Record all Redis write commands except queries, and restore data by re-executing these commands when the Redis service starts.

If you are very concerned about your data, but still can withstand data loss within minutes, then you can only use RDB persistence.
AOF adds every command that Redis executes to disk. Processing huge writes can degrade Redis’s performance. I wonder if you can accept it.
Database backup and disaster recovery:
Timely generation of RDB snapshot is very convenient for database backup, and RDB restores data sets faster than AOF restores.
Redis supports both RDB and AOF. When the system restarts, Redis will use AOF first to recover data, so that the lost data will be minimized.

# What are the underlying data structures used for Redis?

• Strings are implemented using a C dynamic string library so that we don’t pay (asymptotically speaking) for allocations in append operations. This way we have O(N) appends, for instance, instead of having quadratic behavior.
• Lists are implemented with linked lists.
• Sets and Hashes are implemented with hash tables.
• Sorted sets are implemented with skip lists^[4] (a peculiar type of balanced trees).

# How is Redis different from other key-value stores?

• Redis has a different evolution path in the key-value DBs where values can contain more complex data types, with atomic operations defined on those data types. Redis data types are closely related to fundamental data structures and are exposed to the programmer as such, without additional abstraction layers.
• Redis is an in-memory but persistent on disk database, so it represents a different trade off where very high write and read speed is achieved with the limitation of data sets that can’t be larger than memory. Another advantage of in-memory databases is that the memory representation of complex data structures is much simpler to manipulate compared to the same data structures on disk, so Redis can do a lot with little internal complexity. At the same time the two on-disk storage formats (RDB and AOF) don’t need to be suitable for random access, so they are compact and always generated in an append-only fashion (Even the AOF log rotation is an append-only operation, since the new version is generated from the copy of data in memory). However this design also involves different challenges compared to traditional on-disk stores. Being the main data representation on memory, Redis operations must be carefully handled to make sure there is always an updated version of the data set on disk.

# What’s the Redis memory footprint?

To give you a few examples (all obtained using 64-bit instances):

• An empty instance uses ~ 3MB of memory.
• 1 Million small Keys -> String Value pairs use ~ 85MB of memory.
• 1 Million Keys -> Hash value, representing an object with 5 fields, use ~ 160 MB of memory.

Testing your use case is trivial. Use the redis-benchmark utility to generate random data sets then check the space used with the INFO memory command.

64-bit systems will use considerably more memory than 32-bit systems to store the same keys, especially if the keys and values are small. This is because pointers take 8 bytes in 64-bit systems. But of course the advantage is that you can have a lot of memory in 64-bit systems, so in order to run large Redis servers a 64-bit system is more or less required. The alternative is sharding.

# Why does Redis keep its entire dataset in memory?

In the past the Redis developers experimented with Virtual Memory and other systems in order to allow larger than RAM datasets, but after all we are very happy if we can do one thing well: data served from memory, disk used for storage. So for now there are no plans to create an on disk backend for Redis. Most of what Redis is, after all, a direct result of its current design.

If your real problem is not the total RAM needed, but the fact that you need to split your data set into multiple Redis instances, please read the partitioning page in this documentation for more info.

Redis Ltd., the company sponsoring Redis development, has developed a “Redis on Flash” solution that uses a mixed RAM/flash approach for larger data sets with a biased access pattern. You may check their offering for more information, however this feature is not part of the open source Redis code base.

# Can you use Redis with a disk-based database?

Yes, a common design pattern involves taking very write-heavy small data in Redis (and data you need the Redis data structures to model your problem in an efficient way), and big blobs of data into an SQL or eventually consistent on-disk database. Similarly sometimes Redis is used in order to take in memory another copy of a subset of the same data stored in the on-disk database. This may look similar to caching, but actually is a more advanced model since normally the Redis dataset is updated together with the on-disk DB dataset, and not refreshed on cache misses.

# How can I reduce Redis’ overall memory usage?

If you can, use Redis 32 bit instances. Also make good use of small hashes, lists, sorted sets, and sets of integers, since Redis is able to represent those data types in the special case of a few elements in a much more compact way. There is more info in the Memory Optimization page^[5].

# What happens if Redis runs out of memory?

Redis has built-in protections allowing the users to set a max limit on memory usage, using the maxmemory option in the configuration file to put a limit to the memory Redis can use. If this limit is reached, Redis will start to reply with an error to write commands (but will continue to accept read-only commands).

You can also configure Redis to evict keys when the max memory limit is reached. See the eviction policy docs^[6] for more information on this.

# Are Redis on-disk snapshots atomic?

Yes, the Redis background saving process is always forked when the server is outside of the execution of a command, so every command reported to be atomic in RAM is also atomic from the point of view of the disk snapshot.

# What is the maximum number of keys a single Redis instance can hold? What is the maximum number of elements in a Hash, List, Set, and Sorted Set?

Redis can handle up to 2^32 keys, and was tested in practice to handle at least 250 million keys per instance.

Every hash, list, set, and sorted set, can hold 2^32 elements.

In other words your limit is likely the available memory in your system.

# Why does my replica have a different number of keys its master instance?

If you use keys with limited time to live (Redis expires) this is normal behavior. This is what happens:

• The primary generates an RDB file on the first synchronization with the replica.
• The RDB file will not include keys already expired in the primary but which are still in memory.
• These keys are still in the memory of the Redis primary, even if logically expired. They’ll be considered non-existent, and their memory will be reclaimed later, either incrementally or explicitly on access. While these keys are not logically part of the dataset, they are accounted for in the INFO output and in the DBSIZE command.
• When the replica reads the RDB file generated by the primary, this set of keys will not be loaded.

Because of this, it’s common for users with many expired keys to see fewer keys in the replicas. However, logically, the primary and replica will have the same content.