What is the fastest way to insert bulk data to Redis?

Beginners, Redis / 04/04/2022

So, you want to insert bulk data to Redis and is really not having any clue that which is the fastest way to insert bulk data to Redis. Continue reading this article as the mystery is going to get unfolded here.

Get any of the courses at a very special price. The offer is available only for a limited time.

I am feeling bored, Let’s begin with a meme?
So you must be wondering by now, What’s the fastest method to load data to Redis? Shall I use a Java Program, a Python Program or other? Well, the fastest way to load data to redis is by using redis-cli utility in pipe mode.

However, before you start loading the data by using this method, you have to make sure that the data is in the format which is understood by redis-cli –pipe mode. 

Alright, So I have a file which contains list of products and its prices and it looks like below, how do I load it using “redis-cli –pipe” mode?

product_prices.csv
chair,200
table,400
sofa,30
blah,900
blah,blah
blah,blah
….,….
….,….
No, You can’t load this data straight away in this format. You have to change the data to below format in the file. 

SET chair 200
SET table 400
SET sofa 30
SET blah 900
SET blah 900
etc.

Once you’ve done that, you’re good to load the data by using below command.

cat product_prices.csv | redis-cli --pipe

If your redis-server is running on a different server than you can explicitly specify the host of the redis server by using -h argument as shown below:

cat product_prices.csv | redis-cli -h 10.128.0.2 --pipe

Ok, So tell me why this is the fastest method?

Well, the reason is that all the data is going to be sent to redis and redis needs to reply us only once as an acknowledgement with how many data has been written and how many failed. This saves latency and turnaround time, thus increases throughput.

Well, That’s all for now. If you’re loving learning Redis than continuing reading next posts to learn more.

Get any of the courses at a very special price. The offer is available only for a limited time.

 
%d bloggers like this:

Privacy Policy - Terms and Conditions