What is the fastest way to insert bulk data to Redis?

So, you want to insert bulk data to Redis and do not have any clue which is the fastest way to insert bulk data to Redis. Continue reading this article as the mystery is going to get unfolded here.

Redis CLI Utility to insert bulk data to Redis

So you must be wondering by now, What’s the fastest method to load data to Redis? Shall I use a Java Program, a Python Program or another? Well, the fastest way to load data to Redis is by using redis-cli utility in pipe mode.

However, before you start loading the data by using this method, you have to make sure that the data is in the format which is understood by redis-cli –pipe mode. 

Alright, So I have a file that contains a list of products and their prices and it looks like below, how do I load it using “redis-cli –pipe” mode?

product_prices.csv
chair,200
table,400
sofa,30
blah,900
blah,blah
blah,blah

Formatting data

You can’t load this data straight away in this format. You have to change the data to the below format in the file. 

SET chair 200
SET table 400
SET sofa 30
SET blah 900
SET blah 900

Using Redis CLI and Pipe

Once you’ve formatted the data, you’re good to load the data by using the below command.

cat product_prices.csv | redis-cli --pipe

If your Redis server is running on a different server then you can explicitly specify the host of the Redis server by using the “-h” argument as shown below:

cat product_prices.csv | redis-cli -h 10.128.0.2 --pipe

OK, So tell me why this is the fastest method.

What makes it the fastest approach to insert data into Redis?

Well, the reason is that all the data is going to be sent to Redis Server at once. Redis needs to reply to us only once as an acknowledgment of how much data has been written and how many failed. This saves latency and turnaround time, thus increasing throughput.

Conclusion

Well, That’s all for now. If you love learning Redis then continue reading the next posts to learn more.

Scroll to Top