Redis 4.0.9 版本集群扩容

一、实验环境

1、系统版本： Centos 7.6   x86_64 2、Redis版本 4.0.9 3、集群IP信息172.16.100.201  172.16.100.202  172.16.100.2034、准备加入集群的IP信息172.16.100.204  172.16.100.205  172.16.100.2065、目的将Redis集群的3主3从，扩容为6主6从，204、205、206 三个新节点各新增1主1从！

二、安装配置Redis集群

1、在3台服务器上安装redis，每台2个实例，如下：172.16.100.201  8000 8001172.16.100.202  8002 8003172.16.100.203  8004 8005开始安装#tar xf redis-4.0.9.tar.gz#cd redis-4.0.9#make#mv redis-4.0.9  /opt/app/redis80002、提供配置文件#cat /opt/app/redis8000/redis.confdaemonize yesport 8000bind 172.16.100.201appendonly nosave 900 1save 300 10save 60 10000000dir /opt/app/redis8000/db/pidfile /opt/app/redis8000/redis_8000.pidlogfile /opt/app/redis8000/logs/redis8000.logcluster-enabled yescluster-config-file nodes8000.confcluster-node-timeout 20000cluster-require-full-coverage no3、修改系统参数#vim  /etc/security/limits.conf *   soft      nofile   65535*   hard      nofile   65535*   soft      nproc    65535*   hard      nproc    65535#vim /etc/sysctl.conf#redisvm.overcommit_memory=1net.core.somaxconn = 511关闭大页面#echo never > /sys/kernel/mm/transparent_hugepage/enabled使配置生效#sysctl  -p启动：/opt/app/redis8000/src/redis-server /opt/app/redis8000/redis.conf 以上是201服务器8000节点配置的例子，202和203服务器2个节点配置，参考上面即可，注意监听ip地址和端口、路径名称需要作出相应的修改。

三、创建Redis 4.0.9 集群（在201服务器操作即可）

1、安装软件包#yum install ruby rubygems -y#gem install redis -v 3.3.52、创建集群#/opt/app/redis8000/src/redis-trib.rb create --replicas 1 172.16.100.201:8000 172.16.100.201:8001 172.16.100.202:8002 172.16.100.202:8003 172.16.100.203:8004 172.16.100.203:80053、查看集群IP信息172.16.100.203:8005 slave172.16.100.201:8001 slave 172.16.100.203:8004 master    10923-16383172.16.100.202:8003 slave 172.16.100.201:8000 master    0-5460 172.16.100.202:8002 master    5461-10922将线上对应3主3从Redis集群slot的dump.rdb拷贝到对应节点，并启动，目的是让实验环境Redis集群有数据，或者自己造一些数据，此步骤省略！为了演示效果，我这里手动set几个key，如下：172.16.100.201:8000> set name1 jodan…………172.16.100.201:8000> set name10 kawayi…………

四、将新节点加入集群并迁移slot(在172.16.100.201服务器操作)

注:在204、205、206各启动2个redis实例，端口如下分配：204  master 8006  slave  8007205  master 8008  slave  8009206  master 8010  slave  80111、配置启动新服务器Redis(一共3台服务器6个redis实例)，配置方法参考步骤二，这里不再赘述。2、将3个master节点加入集群原集群详细信息：  3主3从# /opt/app/redis8000/src/redis-cli  -h 172.16.100.201 -p 8000  cluster nodes11ae0bcc3aeae0cb7e40c28f407e3d2a30504989 172.16.100.202:8002@18002 master - 0 1575702279000 3 connected 5461-10922b87a9f751b1eb4496c2eff495d216c88c22065ac 172.16.100.203:8004@18004 master - 0 1575702277798 5 connected 10923-1638395ccd1953fa78a1ffa599d65b72d547f8277c3b5 172.16.100.201:8001@18001 slave b87a9f751b1eb4496c2eff495d216c88c22065ac 0 1575702279804 5 connected2c67f0832908ae6375fec57b2399a0a8f4c9e9cc 172.16.100.203:8005@18005 slave 11ae0bcc3aeae0cb7e40c28f407e3d2a30504989 0 1575702279000 6 connected9a02a2120d4a0f9bd8b3ae7e11ffe28d0e03d6fc 172.16.100.201:8000@18000 myself,master - 0 1575702278000 1 connected 0-5460187eba647c002ff47674c71225c428fb8db59444 172.16.100.202:8003@18003 slave 9a02a2120d4a0f9bd8b3ae7e11ffe28d0e03d6fc 0 1575702276795 4 connected将172.16.100.204:8006 加入集群，/opt/app/redis8000/src/redis-trib.rb add-node 172.16.100.204:8006 172.16.100.201:8000将172.16.100.205:8008 加入集群/opt/app/redis8000/src/redis-trib.rb add-node 172.16.100.205:8008 172.16.100.201:8000将172.16.100.206:8010 加入集群/opt/app/redis8000/src/redis-trib.rb add-node 172.16.100.206:8010 172.16.100.201:8000查看加入3个master节点之后的集群信息：#/opt/app/redis8000/src/redis-cli  -h 172.16.100.201 -p 8000  cluster nodes           11ae0bcc3aeae0cb7e40c28f407e3d2a30504989 172.16.100.202:8002@18002 master - 0 1575704459051 3 connected 5461-1092295ccd1953fa78a1ffa599d65b72d547f8277c3b5 172.16.100.201:8001@18001 slave b87a9f751b1eb4496c2eff495d216c88c22065ac 0 1575704457045 5 connected187eba647c002ff47674c71225c428fb8db59444 172.16.100.202:8003@18003 slave 9a02a2120d4a0f9bd8b3ae7e11ffe28d0e03d6fc 0 1575704457000 4 connected9a02a2120d4a0f9bd8b3ae7e11ffe28d0e03d6fc 172.16.100.201:8000@18000 myself,master - 0 1575704458000 1 connected 0-5460c4772e933405eda022b0f7b837a67f05690c76da 172.16.100.204:8006@18006 master - 0 1575704458048 0 connected   564d7877350b0a4789f18162c393a04584f9d311 172.16.100.205:8008@18008 master - 0 1575704455000 7 connectedb87a9f751b1eb4496c2eff495d216c88c22065ac 172.16.100.203:8004@18004 master - 0 1575704455040 5 connected 10923-163833dc917e0d7a1387caf646add2d06814cdb3560ab 172.16.100.206:8010@18010 master - 0 1575704458548 0 connected2c67f0832908ae6375fec57b2399a0a8f4c9e9cc 172.16.100.203:8005@18005 slave 11ae0bcc3aeae0cb7e40c28f407e3d2a30504989 0 1575704457000 6 connected可以看到新加入集群的节点是没有slot的，所以下面就需要将其他3个master节点的部分slot迁移过去。3、开始slot自动迁移#ruby /opt/app/redis8000/src/redis-trib.rb rebalance --auto-weights --use-empty-masters 172.16.100.201:8000>>> Performing Cluster Check (using node 172.16.100.201:8000)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.>>> Rebalancing across 6 nodes. Total weight = 6Moving 2732 slots from 172.16.100.202:8002 to 172.16.100.204:8006#################   花费时间长短根据迁移数据量大小Moving 2731 slots from 172.16.100.203:8004 to 172.16.100.206:8010#################Moving 2731 slots from 172.16.100.201:8000 to 172.16.100.205:8008#################结束之后，再次查看集群信息：# /opt/app/redis8000/src/redis-cli  -h 172.16.100.201 -p 8000  cluster nodes                                11ae0bcc3aeae0cb7e40c28f407e3d2a30504989 172.16.100.202:8002@18002 master - 0 1575706557000 3 connected 8193-1092295ccd1953fa78a1ffa599d65b72d547f8277c3b5 172.16.100.201:8001@18001 slave b87a9f751b1eb4496c2eff495d216c88c22065ac 0 1575706557000 5 connected187eba647c002ff47674c71225c428fb8db59444 172.16.100.202:8003@18003 slave 9a02a2120d4a0f9bd8b3ae7e11ffe28d0e03d6fc 0 1575706554000 4 connected9a02a2120d4a0f9bd8b3ae7e11ffe28d0e03d6fc 172.16.100.201:8000@18000 myself,master - 0 1575706555000 1 connected 2731-5460c4772e933405eda022b0f7b837a67f05690c76da 172.16.100.204:8006@18006 master - 0 1575706558000 9 connected 5461-8192564d7877350b0a4789f18162c393a04584f9d311 172.16.100.205:8008@18008 master - 0 1575706557237 11 connected 0-2730b87a9f751b1eb4496c2eff495d216c88c22065ac 172.16.100.203:8004@18004 master - 0 1575706556000 5 connected 13654-163833dc917e0d7a1387caf646add2d06814cdb3560ab 172.16.100.206:8010@18010 master - 0 1575706559243 10 connected 10923-136532c67f0832908ae6375fec57b2399a0a8f4c9e9cc 172.16.100.203:8005@18005 slave 11ae0bcc3aeae0cb7e40c28f407e3d2a30504989 0 1575706558240 6 connected可以看出新加入集群的3个master节点已经有slot了。查看前面手动加的key是否成功迁移到了新master节点，如下：172.16.100.201:8000> get name1-> Redirected to slot [12933] located at 172.16.100.206:8010    #新节点"jodan"172.16.100.206:8010> get name2-> Redirected to slot [742] located at 172.16.100.205:8008      #新节点"kobe"172.16.100.205:8008> get name3-> Redirected to slot [4807] located at 172.16.100.201:8000"onier"172.16.100.201:8000> get name4-> Redirected to slot [8736] located at 172.16.100.202:8002"curry"172.16.100.202:8002> get name5-> Redirected to slot [12801] located at 172.16.100.206:8010    #新节点"james"172.16.100.206:8010> get name6-> Redirected to slot [610] located at 172.16.100.205:8008      #新节点"durant"172.16.100.205:8008> get name7-> Redirected to slot [4675] located at 172.16.100.201:8000"timu"172.16.100.201:8000> get name8-> Redirected to slot [9132] located at 172.16.100.202:8002"park"172.16.100.202:8002> get name9-> Redirected to slot [13197] located at 172.16.100.206:8010     #新节点"qiaozhi"  可以看到有几个key已经成功迁移到了新节点205和206！4、为新master节点添加slave节点(在172.16.100.201节点操作)为了避免单点故障，设计主从如下：204              205             2068006 M1         8008 M2          8010 M38007 S3         8009 S1          8011 S2将205@8009这个节点加为 204@8006 的从节点/opt/app/redis8000/src/redis-trib.rb add-node --slave --master-id c4772e933405eda022b0f7b837a67f05690c76da 172.16.100.205:8009 172.16.100.204:8006将206@8011这个节点加为 205@8008 的从节点/opt/app/redis8000/src/redis-trib.rb add-node --slave --master-id 564d7877350b0a4789f18162c393a04584f9d311 172.16.100.206:8011 172.16.100.205:8008将204@8007这个节点加为 206@8010 的从节点/opt/app/redis8000/src/redis-trib.rb add-node --slave --master-id 3dc917e0d7a1387caf646add2d06814cdb3560ab 172.16.100.204:8007 172.16.100.206:8010再次查看集群信息如下：#/opt/app/redis8000/src/redis-cli  -h 172.16.100.201 -p 8000  cluster nodes11ae0bcc3aeae0cb7e40c28f407e3d2a30504989 172.16.100.202:8002@18002 master - 0 1575708808057 3 connected 8193-109221d7c87447dd419ac7a2260dce726af03e7965c1a 172.16.100.204:8007@18007 slave 3dc917e0d7a1387caf646add2d06814cdb3560ab 0 1575708799027 10 connected95ccd1953fa78a1ffa599d65b72d547f8277c3b5 172.16.100.201:8001@18001 slave b87a9f751b1eb4496c2eff495d216c88c22065ac 0 1575708807000 5 connected187eba647c002ff47674c71225c428fb8db59444 172.16.100.202:8003@18003 slave 9a02a2120d4a0f9bd8b3ae7e11ffe28d0e03d6fc 0 1575708806049 4 connected9a02a2120d4a0f9bd8b3ae7e11ffe28d0e03d6fc 172.16.100.201:8000@18000 myself,master - 0 1575708806000 1 connected 2731-5460c4772e933405eda022b0f7b837a67f05690c76da 172.16.100.204:8006@18006 master - 0 1575708803000 9 connected 5461-8192564d7877350b0a4789f18162c393a04584f9d311 172.16.100.205:8008@18008 master - 0 1575708801000 11 connected 0-2730735012e7c6f181f7e8a197fc1667ff7bc075361d 172.16.100.206:8011@18011 slave 564d7877350b0a4789f18162c393a04584f9d311 0 1575708807054 11 connectedb87a9f751b1eb4496c2eff495d216c88c22065ac 172.16.100.203:8004@18004 master - 0 1575708805047 5 connected 13654-163835ba06662b9b260ca2cadcf2f7692e5e0e1a91474 172.16.100.205:8009@18009 slave c4772e933405eda022b0f7b837a67f05690c76da 0 1575708804045 9 connected3dc917e0d7a1387caf646add2d06814cdb3560ab 172.16.100.206:8010@18010 master - 0 1575708805000 10 connected 10923-136532c67f0832908ae6375fec57b2399a0a8f4c9e9cc 172.16.100.203:8005@18005 slave 11ae0bcc3aeae0cb7e40c28f407e3d2a30504989 0 1575708806000 6 connected迁移结束！！！

不足之处，请多多交流指出。