How to Migrate Redis from One Server to Another
Starting from Redis version 5.0.0, Redis supports the REPLICAOF command, which assists in migrating data by making the destination Redis server a replica of the source Redis server. You can perform this migration with just a few simple steps:
Step 1: Make the destination Redis server a replica of the source Redis server by running the following command:
REPLICAOF host port
Step 2: Transform the destination Redis server into a master server by executing the following command:
REPLICAOF NO ONE
That's it!
AWS ElastiCache does not support REPLICAOF
When migrating Redis, we encountered a struggle as AWS Elasticache restricts access to certain commands, including "MIGRATE" and "REPLICAOF".
Solution
I discovered a repository written in Go that helped us address this issue by running a one-time application.
It appeared promising, but it hadn't been updated for several years. I identified some problems:
- It used LPOP, which resulted in the removal of list elements from the "From Redis Instance."
- It ran in a single thread, causing significant delays in migrating large records.
- It did not support certain key types, including Set and SortedSet.
I forked the repository and made the necessary fixes & improvements.
Usage
Step 1: Create config file
config.yaml
old_redis:
host: localhost # IP redis server
port: 6379 # Port redis server
password: "" # Password of redis server, leave empty if there is no password
new_redis:
host: localhost
port: 6380
password: ""
migration_databases: [0,1,2,3,4,5,6,7,8] # Databases list which needs to be migrated
concurrent_workers: 5
clear_before_migration: true
Step 2: Run using Docker
docker run \
--rm \
-v /path/to/config.yaml:/data/config.yaml \
huanttok/redis-migrator migrate \
--config.file=/data/config.yaml \
--log.level=debug
(To call localhost while using Docker, use host.docker.internal instead.
Source code
huantt
/
redis-migrator
This tool is designed to migrate Redis database keys from one server to another, addressing the pain point of missing 'REPLICAOF' on AWS ElastiCache
About
The Redis Migrator is a Golang-based tool designed to migrate database keys from one Redis cluster to another.
Since Redis 5.0.0, you can use REPLICAOF command to replicate data from one redis to another.
This application is developed for the older version of Redis or AWS ElastiCache instances that do not support REPLICAOF command (AWS docs).
(This repo is forked from opstree/redis-migration. The original repo has been inactive for over 3 years, so I decided to continue its development without creating pull requests.)
Supported key types:
- String
- Hash
- List
- Set
- Sorted Set
Usage
Installation
Using Go
go install github.com/huantt/redis-migrator@latest
Using Docker
docker run \
--rm \
-v /path/to/migrate.yaml:/data/migrate.yaml \
huanttok/redis-migrator migrate \
--config.file=/data/migrate.yaml \
--log.level=debug
To call localhost while using Docker, use host.docker.internal instead.
Configuration
For using redis-migrator, we have to create a configuration file and provide some needful information to it. An example…
Top comments (0)