Redis主从集群搭建及容灾部署(哨兵sentinel)
Redis容灾部署（哨兵Sentinel）
1、sentinel介绍
Redis Sentinel
Sentinel(哨兵)是用于监控redis集群中Master状态的工具，其已经被集成在redis2.4+的版本中
2、Sentinel作用：
1)：Master状态检测
2)：如果Master异常，则会进行Master-Slave切换，将其中一个Slave作为Master，将之前的Master作为Slave
3)：Master-Slave切换后，master_redis.conf、slave_redis.conf和sentinel.conf的内容都会发生改变，即master_redis.conf中会多一行slaveof的配置，sentinel.conf的监控目标会随之调换
3、Sentinel工作方式：
1)：每个Sentinel以每秒钟一次的频率向它所知的Master，Slave以及其他 Sentinel 实例发送一个 PING 命令
2)：如果一个实例（instance）距离最后一次有效回复 PING 命令的时间超过 down-after-milliseconds 选项所指定的值， 则这个实例会被 Sentinel 标记为主观下线。
3)：如果一个Master被标记为主观下线，则正在监视这个Master的所有 Sentinel 要以每秒一次的频率确认Master的确进入了主观下线状态。
4)：当有足够数量的 Sentinel（大于等于配置文件指定的值）在指定的时间范围内确认Master的确进入了主观下线状态， 则Master会被标记为客观下线
5)：在一般情况下， 每个 Sentinel 会以每 10 秒一次的频率向它已知的所有Master，Slave发送 INFO 命令
6)：当Master被 Sentinel 标记为客观下线时，Sentinel 向下线的 Master 的所有 Slave 发送 INFO 命令的频率会从 10 秒一次改为每秒一次
7)：若没有足够数量的 Sentinel 同意 Master 已经下线， Master 的客观下线状态就会被移除。
若 Master 重新向 Sentinel 的 PING 命令返回有效回复， Master 的主观下线状态就会被移除。
4、主观下线和客观下线
主观下线：Subjectively Down，简称 SDOWN，指的是当前 Sentinel 实例对某个redis服务器做出的下线判断。
客观下线：Objectively Down， 简称 ODOWN，指的是多个 Sentinel 实例在对Master Server做出 SDOWN 判断，并且通过 SENTINEL is-master-down-by-addr 命令互相交流之后，得出的Master Server下线判断，然后开启failover.
通俗来讲就是：
redis的sentinel系统用来管理多个redis服务器，可以实现一个功能上实现HA的集群。该系统主要执行三个任务：
1、监控（ Monitoring ）： Redis Sentinel实时监控主服务器和从服务器运行状态。
2、提醒（notification）： 当被监控的某个 Redis 服务器出现问题时， Redis Sentinel 可以向系统管理员发送通知， 也可以通过 API 向其他程序发送通知
3. 故障迁移（failover发）：自动主从切换
4. 统一的配置管理：连接者询问sentinel取得主从的地址
5、Raft分布式算法
1. 主要用途：用于分布式系统，系统容错，以及选出领头羊
2. 作者：Diego Ongaro，毕业于哈佛
3. 目前用到这个算法的项目有：
a. CoreOS : 见下面
b. ectd : a distributed, consistent shared configuration
c. LogCabin : 分布式存储系统
d. redis sentinel : redis 的监控系统
Sentinel使用的Raft算法核心: 原则
1. 所有sentinel都有选举的领头羊的权利
2. 每个sentinel都会要求其他sentinel选举自己为领头羊(主要由发现redis客观下线的sentinel先发起选举)
3. 每个sentinel只有一次选举的机会
4. 采用先到先得的原则
5. 一旦加入到系统了，则不会自动清除(这一点很重要, why?)
6. 每个sentinel都有唯一的uid，不会因为重启而变更
7. 达到领头羊的条件是 N/2 + 1个sentinel选择了自己
8. 采用配置纪元，如果一次选举出现脑裂，则配置纪元会递增，进入下一次选举，所有sentinel都会处于统一配置纪元，以最新的为标准。
Raft算法核心: 可视图
Raft Visualization (算法演示)
Raft分布式算法的应用
coreos:云计算新星 Docker 正在以火箭般的速度发展，与它相关的生态圈也渐入佳境，CoreOS 就是其中之一。CoreOS 是一个全新的、面向数据中心设计的 Linux 操作系统，在2014年7月发布了首个稳定版本，目前已经完成了800万美元的A轮融资。
一个简单的主从结构加sentinel集群的架构图如下：
192.168.110.134（master redis，sentinel）----------->192.168.110.135（slave redis，sentinel）
| \ / |
| \ / |
| \ / |
| \ / |
| \ / |
| \ |
| \ |
| / \ |
| / \ |
| / \ |
| / \ |
192.168.110.132(sentinel)----------->192.168.110.133(slave redis，sentinel)
上图是一主一从节点，加上两个部署了sentinel的集群，sentinel集群之间会互相通信，沟通交流redis节点的状态，做出相应的判断并进行处理，这里的主观下线状态和客观下线状态是比较重要的状态，它们决定了是否进行故障转移。
可以 通过订阅指定的频道信息，当服务器出现故障得时候通知管理员 客户端可以将 Sentinel 看作是一个只提供了订阅功能的 Redis 服务
器，你不可以使用 PUBLISH 命令向这个服务器发送信息，但你可以用 SUBSCRIBE 命令或者 PSUBSCRIBE 命令， 通过订阅给定的频道来获取
相应的事件提醒。
6、配置哨兵
1）、配置一个哨兵Sentinel的模式：
把源码中的sentinel.conf配置文件拷贝到指定目录
[root@web1 ~]# mkdir /usr/local/sentinel
[root@web1 ~]# cd /usr/local/src/redis-3.2.6/
[root@web1 redis-3.2.6]# cp sentinel.conf /usr/local/sentinel/
[root@web1 redis-3.2.6]# cd /usr/local/sentinel/
[root@web1 sentinel]# vim sentinel.conf
[root@web1 sentinel]# grep -v "^#" sentinel.conf |grep -v "^$"
port 26379
dir /tmp
sentinel monitor mymaster 192.168.110.134 6379 1
sentinel down-after-milliseconds mymaster 30000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000
例子：
#sentinel端口
port 26380
#工作路径
dir "/usr/local/redis-6380"
# 守护进程模式
daemonize yes
# 指明日志文件名
logfile "./sentinel.log"
#哨兵监控的master，主从配置一样，在进行主从切换时6379会变成当前的master端口，
sentinel monitor mymaster 192.168.137.40 6379 1
# master或slave多长时间（默认30秒）不能使用后标记为s_down状态。
sentinel down-after-milliseconds mymaster 5000
#若sentinel在该配置值内未能完成failover操作（即故障时master/slave自动切换），则认为本次failover失败。
sentinel failover-timeout mymaster 18000
#设置master和slaves验证密码
sentinel auth-pass mymaster 123456
#哨兵程序自动添加的部分
# Generated by CONFIG REWRITE
sentinel config-epoch mymaster 0
sentinel leader-epoch mymaster 1
###指明了当前群集的从库的ip和端口，在主从切换时该值会改变
sentinel known-slave mymaster 192.168.137.40 6380
###除了当前的哨兵还有哪些监控的哨兵
sentinel known-sentinel mymaster 192.168.137.40 26379 7a88891a6147e202a53601ca16a3d438e9d55c9d
sentinel current-epoch 1
主要是这一项：
sentinel monitor mymaster 192.168.110.134 6379 2
my mymaster 后跟的是master的ip和端口
最后一个'2'代表我要启动只要有2个sentinel认为master下线，就认为该master客观下线，启动failover并选举产生新的master。
通常最后一个参数不能多于启动的sentinel实例数。
[root@web2 ~]# mkdir /usr/local/sentinel
[root@web2 ~]# cp /usr/local/src/redis-3.2.6/sentinel.conf /usr/local/sentinel/
[root@web2 ~]# cd /usr/local/sentinel/
[root@web2 sentinel]# vim sentinel.conf
[root@web2 sentinel]# grep -v "^#" sentinel.conf |grep -v "^$"
port 26379
dir /tmp
sentinel monitor mymaster 192.168.110.134 6379 2
sentinel down-after-milliseconds mymaster 30000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000
以后台启动模式启动两个sentinel（哨兵）:
[root@web1 sentinel]# redis-sentinel /usr/local/sentinel/sentinel.conf &
[1] 18378
[root@web1 sentinel]# 18378:X 16 Dec 15:46:16.732 * Increased maximum number of open files to 10032 (it was originally set to 1024).
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 3.2.6 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in sentinel mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 26379
| `-._ `._ / _.-' | PID: 18378
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
18378:X 16 Dec 15:46:16.734 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
18378:X 16 Dec 15:46:16.734 # Sentinel ID is 28c5f96738e4fe3335088d037a85ef47dfe9ab8f
18378:X 16 Dec 15:46:16.734 # +monitor master mymaster 192.168.110.134 6379 quorum 1
由上图可以看到：
1、 哨兵已经启动，它的id为236f14b361fc5a0dc0621cf88823ed6e6252b2f3
2、 为master数据库添加了一个监控
3、 发现了2个slave（由此可以看出，哨兵无需配置slave，只需要指定master，哨兵会自动发现slave）
[root@web1 sentinel]# ps aux|grep redis
root 18357 0.1 1.0 135572 2440 ? Ssl 15:40 0:00 redis-server *:6379
root 18378 0.1 1.0 133528 2572 pts/1 Sl 15:46 0:00 redis-sentinel *:26379 [sentinel]
root 18400 0.0 0.3 103244 864 pts/1 S+ 15:51 0:00 grep redis
6、从Redis宕机测试
把其中一个从redis 的进程 kill掉：
[[root@web1 sentinel]# kill 18357
[root@web1 sentinel]# ps aux|grep redis
root 18378 0.1 1.0 133528 2572 pts/1 Sl 15:46 0:01 redis-sentinel *:26379 [sentinel]
root 18444 0.0 0.3 103244 864 pts/1 S+ 16:00 0:00 grep redis
30秒后哨兵的控制台输出：
[root@web1 sentinel]# 18378:X 16 Dec 16:01:16.452 # +sdown master mymaster 192.168.110.134 6379
18378:X 16 Dec 16:01:16.452 # +odown master mymaster 192.168.110.134 6379 #quorum 1/1
18378:X 16 Dec 16:01:16.452 # +new-epoch 9
18378:X 16 Dec 16:01:16.452 # +try-failover master mymaster 192.168.110.134 6379
18378:X 16 Dec 16:01:16.470 # +vote-for-leader 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 9
18378:X 16 Dec 16:01:16.470 # +elected-leader master mymaster 192.168.110.134 6379
18378:X 16 Dec 16:01:16.470 # +failover-state-select-slave master mymaster 192.168.110.134 6379
18378:X 16 Dec 16:01:16.546 # +selected-slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379
18378:X 16 Dec 16:01:16.546 * +failover-state-send-slaveof-noone slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379
18378:X 16 Dec 16:01:16.605 * +failover-state-wait-promotion slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379
18378:X 16 Dec 16:01:16.699 # +promoted-slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379
18378:X 16 Dec 16:01:16.699 # +failover-state-reconf-slaves master mymaster 192.168.110.134 6379
18378:X 16 Dec 16:01:16.759 # +failover-end master mymaster 192.168.110.134 6379
18378:X 16 Dec 16:01:16.759 # +switch-master mymaster 192.168.110.134 6379 192.168.110.135 6379
18378:X 16 Dec 16:01:16.760 * +slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
18378:X 16 Dec 16:01:46.858 # +sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
30秒后哨兵的控制台输出：
18378:X 16 Dec 16:01:16.759 # +switch-master mymaster 192.168.110.134 6379 192.168.110.135 6379
18378:X 16 Dec 16:01:16.760 * +slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
18378:X 16 Dec 16:01:46.858 # +sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
说明已经监控到刚才我人kill掉的master宕机了
可以看出，slave重新已经转换成立主库 -sdown：说明是恢复服务。
[root@web2 sentinel]# redis-cli
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
127.0.0.1:6379>
把192.168.110.134恢复：
[root@web1 sentinel]# redis-server /usr/local/redis/etc/redis.conf
[root@web1 sentinel]# 18378:X 16 Dec 16:06:59.825 # -sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
18378:X 16 Dec 16:07:09.806 * +convert-to-slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
[root@web1 sentinel]#
[root@web1 sentinel]#
[root@web1 sentinel]# ps aux|grep redis
root 18378 0.2 1.0 133528 2636 pts/1 Sl 15:46 0:02 redis-sentinel *:26379 [sentinel]
root 18487 0.1 1.0 135572 2564 ? Ssl 16:06 0:00 redis-server *:6379
root 18491 0.0 0.3 103244 864 pts/1 S+ 16:08 0:00 grep redis
192.168.110.134是作为从库加入到复制中的（主库不会切换过来）：
测试下：
[root@web2 sentinel]# redis-cli
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.110.134,port=6379,state=online,offset=2835,lag=1
master_repl_offset:2835
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:2834
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.110.134,port=6379,state=online,offset=9687,lag=0
master_repl_offset:9832
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:9831
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.110.134,port=6379,state=online,offset=36450,lag=0
master_repl_offset:36595
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:36594
127.0.0.1:6379> set name2 lisi
OK
127.0.0.1:6379> set age2 22
OK
127.0.0.1:6379> get name2
"lisi"
127.0.0.1:6379> get age2
"22"
127.0.0.1:6379>
[root@web1 sentinel]# redis-cli
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:192.168.110.135
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:73863
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
127.0.0.1:6379> get name2
"lisi"
127.0.0.1:6379> get set2
(nil)
127.0.0.1:6379> get age2
"22"
127.0.0.1:6379>
效果不错：
把主库切换回192.168.110.134
[root@web2 sentinel]# ps aux|grep redis
root 7063 0.1 1.2 135572 3116 ? Ssl 14:35 0:13 redis-server *:6379
root 7649 0.0 0.3 103244 864 pts/1 S+ 16:48 0:00 grep redis
[root@web2 sentinel]# kill 7063
[root@web1 sentinel]# 18522:X 16 Dec 16:49:05.725 # +sdown master mymaster 192.168.110.135 6379
18522:X 16 Dec 16:49:05.725 # +odown master mymaster 192.168.110.135 6379 #quorum 1/1
18522:X 16 Dec 16:49:05.725 # +new-epoch 10
18522:X 16 Dec 16:49:05.725 # +try-failover master mymaster 192.168.110.135 6379
18522:X 16 Dec 16:49:05.736 # +vote-for-leader 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 10
18522:X 16 Dec 16:49:05.736 # +elected-leader master mymaster 192.168.110.135 6379
18522:X 16 Dec 16:49:05.736 # +failover-state-select-slave master mymaster 192.168.110.135 6379
18522:X 16 Dec 16:49:05.791 # +selected-slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
18522:X 16 Dec 16:49:05.791 * +failover-state-send-slaveof-noone slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
18522:X 16 Dec 16:49:05.875 * +failover-state-wait-promotion slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
18522:X 16 Dec 16:49:06.233 # +promoted-slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
18522:X 16 Dec 16:49:06.233 # +failover-state-reconf-slaves master mymaster 192.168.110.135 6379
18522:X 16 Dec 16:49:06.331 # +failover-end master mymaster 192.168.110.135 6379
18522:X 16 Dec 16:49:06.331 # +switch-master mymaster 192.168.110.135 6379 192.168.110.134 6379
18522:X 16 Dec 16:49:06.331 * +slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379
18522:X 16 Dec 16:49:36.394 # +sdown slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379
切换成功
[root@web1 sentinel]# redis-cli
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
127.0.0.1:6379>
恢复192.168.110.135的redis：
[root@web2 sentinel]# redis-server /etc/redis.conf
127.0.0.1:6379> 18522:X 16 Dec 16:52:32.351 # -sdown slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379
18522:X 16 Dec 16:52:42.305 * +convert-to-slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379
127.0.0.1:6379>
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.110.135,port=6379,state=online,offset=13653,lag=1
master_repl_offset:13667
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:13666
127.0.0.1:6379>
1
[root@web2 sentinel]# redis-cli
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:192.168.110.134
master_port:6379
master_link_status:up
master_last_io_seconds_ago:2
master_sync_in_progress:0
slave_repl_offset:16502
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
127.0.0.1:6379>
2）、配置两个哨兵Sentinel的模式：
[root@web1 sentinel]# grep -v "^$" /usr/local/sentinel/sentinel.conf |grep -v "^#"
port 26379
dir "/tmp"
sentinel myid 28c5f96738e4fe3335088d037a85ef47dfe9ab8f
sentinel monitor mymaster 192.168.110.134 6379 2
sentinel config-epoch mymaster 10
sentinel leader-epoch mymaster 10
sentinel known-slave mymaster 192.168.110.135 6379
sentinel known-sentinel mymaster 192.168.110.135 26379 16286f52de2675345d1b11bc8946adc5d3f0c360
sentinel current-epoch 10
[root@web2 sentinel]# grep -v "^$" /usr/local/sentinel/sentinel.conf |grep -v "^#"
port 26379
dir "/tmp"
sentinel myid 16286f52de2675345d1b11bc8946adc5d3f0c360
sentinel monitor mymaster 192.168.110.134 6379 2
sentinel config-epoch mymaster 0
sentinel leader-epoch mymaster 6
sentinel known-slave mymaster 192.168.110.135 6379
sentinel known-sentinel mymaster 192.168.110.134 26379 28c5f96738e4fe3335088d037a85ef47dfe9ab8f
sentinel current-epoch 10
主要是这一项：
sentinel monitor mymaster 192.168.110.134 6379 2
最后一个'2'代表我要启动只要有2个sentinel认为master下线，就认为该master客观下线，启动failover并选举产生新的master。
通常最后一个参数不能多于启动的sentinel实例数。
[root@web1 sentinel]# redis-sentinel /usr/local/sentinel/sentinel.conf &
[1] 18747
[root@web1 sentinel]# 18747:X 16 Dec 17:04:30.597 * Increased maximum number of open files to 10032 (it was originally set to 1024).
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 3.2.6 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in sentinel mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 26379
| `-._ `._ / _.-' | PID: 18747
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
18747:X 16 Dec 17:04:30.599 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
18747:X 16 Dec 17:04:30.599 # Sentinel ID is 28c5f96738e4fe3335088d037a85ef47dfe9ab8f
18747:X 16 Dec 17:04:30.599 # +monitor master mymaster 192.168.110.134 6379 quorum 2
18747:X 16 Dec 17:05:00.659 # +sdown sentinel 16286f52de2675345d1b11bc8946adc5d3f0c360 192.168.110.135 26379 @ mymaster 192.168.110.134 6379
[root@web2 sentinel]# redis-sentinel /usr/local/sentinel/sentinel.conf &
[1] 7726
You have new mail in /var/spool/mail/root
[root@web2 sentinel]# 7726:X 16 Dec 17:04:33.835 * Increased maximum number of open files to 10032 (it was originally set to 1024).
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 3.2.6 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in sentinel mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 26379
| `-._ `._ / _.-' | PID: 7726
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
7726:X 16 Dec 17:04:33.836 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
7726:X 16 Dec 17:04:33.836 # Sentinel ID is 16286f52de2675345d1b11bc8946adc5d3f0c360
7726:X 16 Dec 17:04:33.836 # +monitor master mymaster 192.168.110.134 6379 quorum 2
7726:X 16 Dec 17:04:34.668 # +new-epoch 10
7726:X 16 Dec 17:05:03.836 # +sdown sentinel 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 192.168.110.134 26379 @ mymaster 192.168.110.134 6379
上面是有问题的：原因是bind 默认是仅仅监控本机和 192.168.1.1这两个IP地址
# bind 127.0.0.1 192.168.1.1
添加： bind 0.0.0.0
就没问题了
配置文件：
[root@web1 ~]# grep -v "^$" /usr/local/sentinel/sentinel.conf |grep -v "^#"
bind 0.0.0.0
port 26379
dir "/tmp"
sentinel myid 28c5f96738e4fe3335088d037a85ef47dfe9ab8f
sentinel monitor mymaster 192.168.110.135 6379 2
sentinel config-epoch mymaster 14
sentinel leader-epoch mymaster 14
sentinel known-slave mymaster 192.168.110.134 6379
sentinel known-sentinel mymaster 192.168.110.135 26379 16286f52de2675345d1b11bc8946adc5d3f0c360
sentinel current-epoch 14
主要是下面两项：
bind 0.0.0.0
sentinel monitor mymaster 192.168.110.135 6379 2
如果设置了密码则添加下面一项：
# sentinel auth-pass
sentinel auth-pass def_master 012_345^678-90
[root@web1 ~]# redis-sentinel /usr/local/sentinel/sentinel.conf &
[1] 1963
[root@web1 ~]# 1963:X 19 Dec 11:55:29.670 * Increased maximum number of open files to 10032 (it was originally set to 1024).
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 3.2.6 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in sentinel mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 26379
| `-._ `._ / _.-' | PID: 1963
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
1963:X 19 Dec 11:55:29.671 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1963:X 19 Dec 11:55:29.671 # Sentinel ID is 28c5f96738e4fe3335088d037a85ef47dfe9ab8f
1963:X 19 Dec 11:55:29.672 # +monitor master mymaster 192.168.110.134 6379 quorum 2
上面表示启动没问题的：
kill掉一个从redis，观察sentinel控制台输出信息：
[root@web1 ~]# kill 1912
[root@web1 ~]# ps aux|grep redis
root 1963 0.2 1.0 133528 2588 pts/0 Sl 11:55 0:00 redis-sentinel 0.0.0.0:26379 [sentinel]
root 1992 0.0 0.3 103244 864 pts/0 S+ 11:58 0:00 grep redis
[root@web1 ~]# 1963:X 19 Dec 11:59:06.906 # +sdown master mymaster 192.168.110.134 6379
1963:X 19 Dec 11:59:06.963 # +odown master mymaster 192.168.110.134 6379 #quorum 2/2
1963:X 19 Dec 11:59:06.963 # +new-epoch 14
1963:X 19 Dec 11:59:06.963 # +try-failover master mymaster 192.168.110.134 6379
1963:X 19 Dec 11:59:06.972 # +vote-for-leader 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 14
1963:X 19 Dec 11:59:06.992 # 16286f52de2675345d1b11bc8946adc5d3f0c360 voted for 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 14
1963:X 19 Dec 11:59:07.035 # +elected-leader master mymaster 192.168.110.134 6379
1963:X 19 Dec 11:59:07.035 # +failover-state-select-slave master mymaster 192.168.110.134 6379
1963:X 19 Dec 11:59:07.136 # +selected-slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379
1963:X 19 Dec 11:59:07.136 * +failover-state-send-slaveof-noone slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379
1963:X 19 Dec 11:59:07.199 * +failover-state-wait-promotion slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379
1963:X 19 Dec 11:59:08.171 # +promoted-slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379
1963:X 19 Dec 11:59:08.171 # +failover-state-reconf-slaves master mymaster 192.168.110.134 6379
1963:X 19 Dec 11:59:08.260 # +failover-end master mymaster 192.168.110.134 6379
1963:X 19 Dec 11:59:08.260 # +switch-master mymaster 192.168.110.134 6379 192.168.110.135 6379
1963:X 19 Dec 11:59:08.260 * +slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
1963:X 19 Dec 11:59:38.290 # +sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
另一台：
[root@web2 ~]# redis-sentinel /usr/local/sentinel/sentinel.conf &
[1] 1809
[root@web2 ~]# 1809:X 19 Dec 11:55:35.698 * Increased maximum number of open files to 10032 (it was originally set to 1024).
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 3.2.6 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in sentinel mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 26379
| `-._ `._ / _.-' | PID: 1809
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
1809:X 19 Dec 11:55:35.700 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1809:X 19 Dec 11:55:35.700 # Sentinel ID is 16286f52de2675345d1b11bc8946adc5d3f0c360
1809:X 19 Dec 11:55:35.700 # +monitor master mymaster 192.168.110.134 6379 quorum 2
1809:X 19 Dec 11:59:06.895 # +sdown master mymaster 192.168.110.134 6379
1809:X 19 Dec 11:59:06.999 # +new-epoch 14
1809:X 19 Dec 11:59:07.001 # +vote-for-leader 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 14
1809:X 19 Dec 11:59:08.020 # +odown master mymaster 192.168.110.134 6379 #quorum 2/2
1809:X 19 Dec 11:59:08.020 # Next failover delay: I will not start a failover before Mon Dec 19 12:05:07 2016
1809:X 19 Dec 11:59:08.284 # +config-update-from sentinel 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 192.168.110.134 26379 @ mymaster 192.168.110.134 6379
1809:X 19 Dec 11:59:08.284 # +switch-master mymaster 192.168.110.134 6379 192.168.110.135 6379
1809:X 19 Dec 11:59:08.284 * +slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
1809:X 19 Dec 11:59:38.344 # +sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
[root@web2 ~]# redis-cli
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
127.0.0.1:6379>
恢复192.168.110.134：
[root@web1 ~]# redis-server /usr/local/redis/etc/redis.conf
[root@web1 ~]# 1963:X 19 Dec 12:14:24.393 # -sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
[root@web1 ~]# redis-cli
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:192.168.110.135
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:38769
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
127.0.0.1:6379>
135上的显示：
repl_backlog_histlen:0
127.0.0.1:6379> 1809:X 19 Dec 12:14:24.105 # -sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
127.0.0.1:6379>
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.110.134,port=6379,state=online,offset=2404,lag=1
master_repl_offset:2404
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:2403
也是仅仅作为从。
2）、配置三个哨兵Sentinel的模式：
[root@web2 ~]# scp redis-3.2.6.tar.gz 192.168.110.133:/root
The authenticity of host '192.168.110.133 (192.168.110.133)' can't be established.
RSA key fingerprint is 60:6c:0d:f4:3b:f8:42:b1:c9:7e:ab:c4:bc:83:d1:09.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.110.133' (RSA) to the list of known hosts.
root@192.168.110.133's password:
redis-3.2.6.tar.gz 100% 1509KB 1.5MB/s 00:00
[root@node2 ~]# iptables -I INPUT 6 -m state --state NEW -p tcp --dport 6379 -j ACCEPT
[root@node2 ~]# iptables -I INPUT 7 -m state --state NEW -p tcp --dport 26379 -j ACCEPT
[root@node2 ~]# tar -zxvf redis-3.2.6.tar.gz -C /usr/local/src/
[root@node2 ~]# cd /usr/local/src/redis-3.2.6/
[root@node2 redis-3.2.6]# make
[root@node2 redis-3.2.6]# make install
root@node2 redis-3.2.6]# mkdir /etc/redis
[root@node2 redis-3.2.6]# mkdir -p /redis/data
[root@node2 redis-3.2.6]# mkdir -p /redis/log
2、配置主从：
[root@node2 redis-3.2.6]# grep -v "^$" /etc/redis/redis.conf |grep -v "^#"
bind *
protected-mode yes
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised no
pidfile /var/run/redis_6379.pid
loglevel notice
logfile "/redis/log/redis.log"
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir /redis/data
slaveof 192.168.110.134 6379
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes
[root@node2 redis-3.2.6]# redis-server /etc/redis/redis.conf
[root@node2 redis-3.2.6]# ps aux|grep redis
root 4220 0.1 0.9 133524 2312 ? Ssl 16:36 0:00 redis-server *:6379
root 4224 0.0 0.3 103248 864 pts/0 S+ 16:36 0:00 grep redis
[root@web1 ~]# redis-cli
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=192.168.110.135,port=6379,state=online,offset=325246,lag=0
slave1:ip=192.168.110.133,port=6379,state=online,offset=325246,lag=0
master_repl_offset:325246
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:325245
127.0.0.1:6379>
2、设置哨兵：
[root@node2 redis-3.2.6]# mkdir /etc/sentinel
[root@node2 redis-3.2.6]# cp sentinel.conf /etc/sentinel/
[root@node2 redis-3.2.6]# grep -v "^$" /etc/sentinel/sentinel.conf |grep -v "^#"
bind 0.0.0.0
port 26379
dir "/tmp"
sentinel myid fd3e2ac303fd06dfa7934718d68429dc88e3b5b8
sentinel monitor mymaster 192.168.110.134 6379 2
sentinel config-epoch mymaster 15
sentinel leader-epoch mymaster 0
sentinel known-slave mymaster 192.168.110.133 6379
sentinel known-slave mymaster 192.168.110.135 6379
sentinel known-sentinel mymaster 192.168.110.135 26379 16286f52de2675345d1b11bc8946adc5d3f0c360
sentinel known-sentinel mymaster 192.168.110.134 26379 28c5f96738e4fe3335088d037a85ef47dfe9ab8f
sentinel current-epoch 15
[root@node2 redis-3.2.6]# redis-sentinel /etc/sentinel/sentinel.conf &
[1] 4240
[root@node2 redis-3.2.6]# 4240:X 18 Dec 16:48:07.492 * Increased maximum number of open files to 10032 (it was originally set to 1024).
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 3.2.6 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in sentinel mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 26379
| `-._ `._ / _.-' | PID: 4240
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
4240:X 18 Dec 16:48:07.494 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
4240:X 18 Dec 16:48:07.494 # Sentinel ID is fd3e2ac303fd06dfa7934718d68429dc88e3b5b8
4240:X 18 Dec 16:48:07.494 # +monitor master mymaster 192.168.110.134 6379 quorum 2
4240:X 18 Dec 16:48:07.499 * +slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379
4240:X 18 Dec 16:48:07.515 * +slave slave 192.168.110.133:6379 192.168.110.133 6379 @ mymaster 192.168.110.134 6379
4240:X 18 Dec 16:48:08.295 * +sentinel sentinel 16286f52de2675345d1b11bc8946adc5d3f0c360 192.168.110.135 26379 @ mymaster 192.168.110.134 6379
4240:X 18 Dec 16:48:08.328 # +new-epoch 15
4240:X 18 Dec 16:48:09.420 * +sentinel sentinel 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 192.168.110.134 26379 @ mymaster 192.168.110.134 6379
[root@node2 redis-3.2.6]# ps aux|grep redis
root 4220 0.2 1.0 133524 2408 ? Ssl 16:36 0:03 redis-server *:6379
root 4240 0.3 1.1 133528 2696 pts/0 Sl 16:48 0:02 redis-sentinel 0.0.0.0:26379 [sentinel]
root 4264 0.0 0.3 103248 860 pts/0 S+ 17:01 0:00 grep redis
[root@web2 ~]# redis-cli -p 26379
127.0.0.1:26379> info
.... ..........
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=192.168.110.134:6379,slaves=2,sentinels=2
127.0.0.1:26379> 1809:X 19 Dec 14:59:24.984 * +sentinel sentinel fd3e2ac303fd06dfa7934718d68429dc88e3b5b8 192.168.110.133 26379 @ mymaster 192.168.110.134 6379
127.0.0.1:26379>
127.0.0.1:26379> info
# Server
redis_version:3.2.6
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:2f0dd1a83d7e7010
redis_mode:sentinel
os:Linux 2.6.32-431.el6.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
gcc_version:4.4.7
process_id:1809
run_id:519043c8b9e7314a71c425400d01769b3384f77c
tcp_port:26379
uptime_in_seconds:11045
uptime_in_days:0
hz:11
lru_clock:5735772
executable:/root/redis-sentinel
config_file:/usr/local/sentinel/sentinel.conf
# Clients
connected_clients:3
client_longest_output_list:0
client_biggest_input_buf:0
blocked_clients:0
# CPU
used_cpu_sys:26.71
used_cpu_user:2.31
used_cpu_sys_children:0.00
used_cpu_user_children:0.00
# Stats
total_connections_received:3
total_commands_processed:16011
instantaneous_ops_per_sec:4
total_net_input_bytes:937405
total_net_output_bytes:97572
instantaneous_input_kbps:0.26
instantaneous_output_kbps:0.02
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
evicted_keys:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:0
migrate_cached_sockets:0
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=192.168.110.134:6379,slaves=2,sentinels=3
127.0.0.1:26379>
[root@node2 redis-3.2.6]# redis-cli -p 26379
127.0.0.1:26379> info
# Server
redis_version:3.2.6
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:98c29634d36aff37
redis_mode:sentinel
os:Linux 2.6.32-431.el6.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
gcc_version:4.4.7
process_id:4240
run_id:d0cf3dbc5259fa960ccf02ed101bc69bd878e26d
tcp_port:26379
uptime_in_seconds:630
uptime_in_days:0
hz:10
lru_clock:5656509
executable:/usr/local/src/redis-3.2.6/redis-sentinel
config_file:/etc/sentinel/sentinel.conf
# Clients
connected_clients:3
client_longest_output_list:0
client_biggest_input_buf:0
blocked_clients:0
# CPU
used_cpu_sys:1.88
used_cpu_user:0.12
used_cpu_sys_children:0.00
used_cpu_user_children:0.00
# Stats
total_connections_received:3
total_commands_processed:1819
instantaneous_ops_per_sec:2
total_net_input_bytes:106447
total_net_output_bytes:10937
instantaneous_input_kbps:0.25
instantaneous_output_kbps:0.01
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
evicted_keys:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:0
migrate_cached_sockets:0
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=192.168.110.134:6379,slaves=2,sentinels=3
[root@node2 redis-3.2.6]# redis-cli
127.0.0.1:6379> get name
"zhangsan"
127.0.0.1:6379> get age
"20"
127.0.0.1:6379> get age1
(nil)
127.0.0.1:6379> get age2
"22"
127.0.0.1:6379>
宕掉192.168.110.134的主库：
[root@web1 ~]# ps aux|grep redis
root 1963 0.2 1.1 133528 2716 pts/0 Sl 11:55 0:31 redis-sentinel 0.0.0.0:26379 [sentinel]
root 2068 0.1 1.4 135572 3468 ? Ssl 12:14 0:20 redis-server *:6379
root 2817 0.0 0.3 103244 864 pts/0 S+ 15:11 0:00 grep redis
[root@web1 ~]# kill 2068
[root@web1 ~]# ps aux|grep redis
root 1963 0.2 1.1 133528 2716 pts/0 Sl 11:55 0:31 redis-sentinel 0.0.0.0:26379 [sentinel]
root 2826 0.0 0.3 103244 864 pts/0 S+ 15:13 0:00 grep redis
[root@web1 ~]# 1963:X 19 Dec 15:13:44.721 # +sdown master mymaster 192.168.110.134 6379
1963:X 19 Dec 15:13:44.776 # +odown master mymaster 192.168.110.134 6379 #quorum 3/2
1963:X 19 Dec 15:13:44.776 # +new-epoch 16
1963:X 19 Dec 15:13:44.776 # +try-failover master mymaster 192.168.110.134 6379
1963:X 19 Dec 15:13:44.778 # +vote-for-leader 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 16
1963:X 19 Dec 15:13:44.797 # fd3e2ac303fd06dfa7934718d68429dc88e3b5b8 voted for 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 16
1963:X 19 Dec 15:13:44.804 # 16286f52de2675345d1b11bc8946adc5d3f0c360 voted for 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 16
1963:X 19 Dec 15:13:44.855 # +elected-leader master mymaster 192.168.110.134 6379
1963:X 19 Dec 15:13:44.855 # +failover-state-select-slave master mymaster 192.168.110.134 6379
1963:X 19 Dec 15:13:44.956 # +selected-slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379
1963:X 19 Dec 15:13:44.956 * +failover-state-send-slaveof-noone slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379
1963:X 19 Dec 15:13:45.023 * +failover-state-wait-promotion slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379
1963:X 19 Dec 15:13:45.898 # +promoted-slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.134 6379
1963:X 19 Dec 15:13:45.898 # +failover-state-reconf-slaves master mymaster 192.168.110.134 6379
1963:X 19 Dec 15:13:45.963 * +slave-reconf-sent slave 192.168.110.133:6379 192.168.110.133 6379 @ mymaster 192.168.110.134 6379
1963:X 19 Dec 15:13:46.107 * +slave-reconf-inprog slave 192.168.110.133:6379 192.168.110.133 6379 @ mymaster 192.168.110.134 6379
1963:X 19 Dec 15:13:46.911 # -odown master mymaster 192.168.110.134 6379
1963:X 19 Dec 15:13:47.154 * +slave-reconf-done slave 192.168.110.133:6379 192.168.110.133 6379 @ mymaster 192.168.110.134 6379
1963:X 19 Dec 15:13:47.212 # +failover-end master mymaster 192.168.110.134 6379
1963:X 19 Dec 15:13:47.212 # +switch-master mymaster 192.168.110.134 6379 192.168.110.135 6379
1963:X 19 Dec 15:13:47.213 * +slave slave 192.168.110.133:6379 192.168.110.133 6379 @ mymaster 192.168.110.135 6379
1963:X 19 Dec 15:13:47.213 * +slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
1963:X 19 Dec 15:14:17.223 # +sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.135 6379
135成了主库：
[root@web2 ~]# redis-cli
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.110.133,port=6379,state=online,offset=27695,lag=0
master_repl_offset:27695
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:27694
127.0.0.1:6379> quit
[root@node2 redis-3.2.6]# redis-cli
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:192.168.110.135
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:21626
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
127.0.0.1:6379>
再宕掉135：
[root@web2 ~]# ps aux|grep redis
root 1809 0.2 1.1 133528 2764 pts/0 Sl 11:55 0:32 redis-sentinel 0.0.0.0:26379 [sentinel]
root 2390 0.2 1.1 135572 2872 ? Ssl 14:11 0:08 redis-server *:6379
root 2675 0.0 0.3 103244 860 pts/0 S+ 15:16 0:00 grep redis
[root@web2 ~]# kill 2390
[root@web2 ~]# ps aux|grep redis
root 1809 0.2 1.1 133528 2768 pts/0 Sl 11:55 0:32 redis-sentinel 0.0.0.0:26379 [sentinel]
root 2677 0.0 0.2 103216 640 pts/0 R+ 15:17 0:00 grep redis
[root@web2 ~]# 1809:X 19 Dec 15:17:31.230 # +sdown master mymaster 192.168.110.135 6379
1809:X 19 Dec 15:17:31.271 # +new-epoch 17
1809:X 19 Dec 15:17:31.286 # +vote-for-leader 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 17
1809:X 19 Dec 15:17:31.302 # +odown master mymaster 192.168.110.135 6379 #quorum 3/2
1809:X 19 Dec 15:17:31.302 # Next failover delay: I will not start a failover before Mon Dec 19 15:23:31 2016
1809:X 19 Dec 15:17:32.002 # +config-update-from sentinel 28c5f96738e4fe3335088d037a85ef47dfe9ab8f 192.168.110.134 26379 @ mymaster 192.168.110.135 6379
1809:X 19 Dec 15:17:32.003 # +switch-master mymaster 192.168.110.135 6379 192.168.110.133 6379
1809:X 19 Dec 15:17:32.003 * +slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.133 6379
1809:X 19 Dec 15:17:32.003 * +slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.133 6379
1809:X 19 Dec 15:18:02.004 # +sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.133 6379
1809:X 19 Dec 15:18:02.004 # +sdown slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.133 6379
133成立主库：
127.0.0.1:6379>
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
127.0.0.1:6379>
[root@web1 ~]# redis-server /usr/local/redis/etc/redis.conf
[root@web1 ~]# 1963:X 19 Dec 15:21:47.745 # -sdown slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.133 6379
1963:X 19 Dec 15:21:57.749 * +convert-to-slave slave 192.168.110.134:6379 192.168.110.134 6379 @ mymaster 192.168.110.133 6379
[root@web2 ~]# redis-server /etc/redis.conf
[root@web2 ~]# 1809:X 19 Dec 15:22:34.441 # -sdown slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.133 6379
1809:X 19 Dec 15:22:44.421 * +convert-to-slave slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.133 6379
133上查看的：
127.0.0.1:6379> 4240:X 18 Dec 17:11:18.953 # -sdown slave 192.168.110.135:6379 192.168.110.135 6379 @ mymaster 192.168.110.133 6379
127.0.0.1:6379>
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=192.168.110.134,port=6379,state=online,offset=24323,lag=1
slave1:ip=192.168.110.135,port=6379,state=online,offset=24470,lag=0
master_repl_offset:24470
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:24469
127.0.0.1:6379>
[root@web1 ~]# redis-cli
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:192.168.110.133
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:113832
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
127.0.0.1:6379>
[root@web2 ~]# redis-cli
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:192.168.110.133
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:122708
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
127.0.0.1:6379>
sentinel一些命令介绍
要使用sentinel的命令，我们需要用redis-cli命令进入到sentinel：
1、INFO
sentinel的基本状态信息
2、SENTINEL masters
列出所有被监视的主服务器，以及这些主服务器的当前状态
3、SENTINEL slaves
列出给定主服务器的所有从服务器，以及这些从服务器的当前状态
4、SENTINEL get-master-addr-by-name
返回给定名字的主服务器的 IP 地址和端口号
5、SENTINEL reset
重置所有名字和给定模式 pattern 相匹配的主服务器。重置操作清除主服务器目前的所有状态， 包括正在执行中的故障转移， 并移除目前已经发现和关联的， 主服务器的所有从服务器和 Sentinel 。
6、SENTINEL failover
当主服务器失效时， 在不询问其他 Sentinel 意见的情况下， 强制开始一次自动故障迁移，但是它会给其他sentinel发送一个最新的配置，其他sentinel会根据这个配置进行更新
