DEV Community

Pavel Evstigneev
Pavel Evstigneev

Posted on

Copy Redis Data to New Server

Sometimes we need to copy redis data to new instance

I know 3 simple ways to do it:

  • continuous migration tools
  • simple script to copy data
  • redis master-slave replication

Other options, more complex:


It monitors changes in source server/cluster and apply them in destination server. Useful when we migrate app that actively writes to redis to use new redis instance. There are other similar tools, but this one works and easy to configure.

Check README for supported types, It's written in C, easier to run it with docker

docker pull ahmadgh21/redis-migrate-tool
docker run -it ahmadgh21/redis-migrate-tool bash
# inside container
mkdir /tmp
chmod 777 /tmp
apt-get update
apt install nano
nano /etc/redis-migrate-tool/rmt.conf
Enter fullscreen mode Exit fullscreen mode

Config example

type: single
hash: fnv1a_64
hash_tag: "{}"
distribution: ketama
servers :
- <source IP>:6379 # <--- change this

type: single
- <destination IP>:6379 # <--- change this

Enter fullscreen mode Exit fullscreen mode

run migrate-tool inside container

/opt/redis-migrate-tool -c /etc/redis-migrate-tool/rmt.conf
Enter fullscreen mode Exit fullscreen mode

Make own script

This script simply copy all keys from server A to server B (it doesn't support all data types, find it in code)

require 'redis'

puts "Connecting to redis://#{ARGV[0]} and redis://#{ARGV[1]}"

redis_src = "redis://#{ARGV[0]}")
redis_dest = "redis://#{ARGV[1]}")

dest_keys = redis_dest.keys("*")

redis_src.keys("*").each do |key|
  next if dest_keys.include?(key)
  type = redis_src.type(key)

  if type == "hash"
    value = redis_src.hgetall(key)
    puts "HASH   #{key} : #{value}"
    redis_dest.mapped_hmset(key, value)
  elsif type == "list"
    values = redis_src.lrange(key, 0, -1)
    puts "LIST.  #{key} : #{values}"
    redis_dest.lpush(key, values)
  elsif type == "set"
    values = redis_src.smembers(key)
    puts "SET    #{key} : #{values}"
    redis_dest.sadd(key, values)
    data = redis_src.get(key)
    puts "STRING #{key} : #{data}"
    redis_dest.set(key, data)

puts "Complete"
Enter fullscreen mode Exit fullscreen mode

Make sure you have ruby redis library installed (gem install redis)

Run the script

ruby redis_copy.rb source_IP destination_IP
Enter fullscreen mode Exit fullscreen mode

Setup redis replication

I think this is most proper option but also require more preparation

on master we need to update config

protected-mode no
requirepass 1234abcd
Enter fullscreen mode Exit fullscreen mode

on replica server we need to put in config:

replicaof <master ip> 6379
masterauth 1234abcd
Enter fullscreen mode Exit fullscreen mode

restart master, then replica, check logs

check status in redis-cli

INFO replication
Enter fullscreen mode Exit fullscreen mode

more docs:

Pros and Cons

Simple script - It copy values once, can be used for not critical data such as cache, rate limit, and for rarely updated data such as configuration

I usually run it once before updating apps to connect to new server, and once again after to make sure all data is in sync

redis-migrate-tool - Constantly replicates all data from source to destination. For apps that constantly write updates to redis, e.g. user sessions, temporary important values.

But it doesn't replicate writes from destination server back to source. To avoid situation when one process write/update to new server and other process can't find it in old server, we need to restart all our apps at one time with updated config. (I usually scale app down to 1 replica during migration)

redis replication - replica is not writable, migration may require short downtime or we need to modify code to handle it smoothly. In our code we can try to save data to new server, and if got "not writable" error then save to old server

Have a smooth redis migration to everyone who's doing it now :)

Top comments (0)