LVM故障导致RHCS启动故障
1、故障提示
抓取资源管理日志发现提示如下错误
tail -f /var/log/cluster/rgmanager.log
May 6 18:21:24 yktdb1 rgmanager[17425]: State change: Local UP
May 6 18:21:24 yktdb1 rgmanager[17425]: Starting stopped service service:yktoracle
May 6 18:21:24 yktdb1 rgmanager[18533]: [lvm] HA LVM: Improper setup detected
May 6 18:21:24 yktdb1 rgmanager[18555]: [lvm] * "volume_list" not specified in lvm.conf.
May 6 18:21:24 yktdb1 rgmanager[17425]: start on lvm "yktoracledb" returned 1 (generic error)
May 6 18:21:24 yktdb1 rgmanager[17425]: #68: Failed to start service:yktoracle; return value: 1
May 6 18:21:24 yktdb1 rgmanager[17425]: Stopping service service:yktoracle
May 6 18:21:25 yktdb1 rgmanager[18586]: [script] Executing /etc/init.d/dbora stop
May 6 18:21:25 yktdb1 rgmanager[18682]: [fs] stop: Could not match /dev/yktoracledb/oracledblv with a real device
May 6 18:21:25 yktdb1 rgmanager[18720]: [lvm] HA LVM: Improper setup detected
May 6 18:21:25 yktdb1 rgmanager[18742]: [lvm] * "volume_list" not specified in lvm.conf.
May 6 18:21:25 yktdb1 rgmanager[18778]: [lvm] Deactivating yktoracledb/oracledblv
May 6 18:21:25 yktdb1 rgmanager[18800]: [lvm] Making resilient : lvchange -an yktoracledb/oracledblv
May 6 18:21:25 yktdb1 rgmanager[18825]: [lvm] Resilient command: lvchange -an yktoracledb/oracledblv --config devices{filter=["a|/dev/mapper/LUN-1800G|","a|/dev/mappe
May 6 18:21:26 yktdb1 rgmanager[17425]: Service service:yktoracle is recovering
May 6 18:21:26 yktdb1 rgmanager[17425]: #71: Relocating failed service service:yktoracle
May 6 18:21:26 yktdb1 rgmanager[17425]: Service service:yktoracle is stopped
May 6 18:21:35 yktdb1 rgmanager[17425]: State change: 192.168.10.2 UP
May 6 18:21:35 yktdb1 rgmanager[17425]: Starting stopped service service:yktoracle
May 6 18:21:36 yktdb1 rgmanager[18886]: [lvm] HA LVM: Improper setup detected
May 6 18:21:36 yktdb1 rgmanager[18908]: [lvm] * "volume_list" not specified in lvm.conf.
May 6 18:21:36 yktdb1 rgmanager[17425]: start on lvm "yktoracledb" returned 1 (generic error)
May 6 18:21:36 yktdb1 rgmanager[17425]: #68: Failed to start service:yktoracle; return value: 1
May 6 18:21:36 yktdb1 rgmanager[17425]: Stopping service service:yktoracle
May 6 18:21:36 yktdb1 rgmanager[18939]: [script] Executing /etc/init.d/dbora stop
May 6 18:21:36 yktdb1 rgmanager[19035]: [fs] stop: Could not match /dev/yktoracledb/oracledblv with a real device
May 6 18:21:36 yktdb1 rgmanager[19073]: [lvm] HA LVM: Improper setup detected
May 6 18:21:37 yktdb1 rgmanager[19095]: [lvm] * "volume_list" not specified in lvm.conf.
May 6 18:21:37 yktdb1 rgmanager[19131]: [lvm] Deactivating yktoracledb/oracledblv
May 6 18:21:37 yktdb1 rgmanager[19153]: [lvm] Making resilient : lvchange -an yktoracledb/oracledblv
May 6 18:21:37 yktdb1 rgmanager[19178]: [lvm] Resilient command: lvchange -an yktoracledb/oracledblv --config devices{filter=["a|/dev/mapper/LUN-1800G|","a|/dev/mappe
May 6 18:21:37 yktdb1 rgmanager[17425]: Service service:yktoracle is recovering
May 6 18:21:37 yktdb1 rgmanager[17425]: #71: Relocating failed service service:yktoracle
May 6 18:21:39 yktdb1 rgmanager[17425]: Service service:yktoracle is stopped
查看lvdiskplay 对应的oracledblv 状态提示 Not available
在/dev/yktoraclevg/下面竟然没有这个oracledblv
除非把clvmd停止后才这个在/dev/yktoarclevg/里就可以看了
查了好多资料都不知道怎么回事
查到一个service clvmd status 后发现 集群 vg和lv都是显示none
这一下让我找到了问题所在
直接用命令vgchange -cy yktoracledb
在查看service clvmd status
[root@yktdb1 ~]# service clvmd status
clvmd (pid 7550) 正在运行...
Clustered Volume Groups: yktoracledb
Active clustered Logical Volumes: oracledblv ysbaklv test
[root@yktdb1 ~]#
已经可以看见集群共享的vg和lv了
在查看集群状态正常了服务也启动了，然后对这个两个节点测试是否可以正常切换。
[root@yktdb1 ~]# clustat
Cluster Status for ytkcluter @ Sun May 7 11:53:49 2017
Member Status: Quorate
Member Name ID Status
------ ---- ---- ------
192.168.10.1 1 Online, Local, rgmanager
192.168.10.2 2 Online, rgmanager
Service Name Owner (Last) State
------- ---- ----- ------ -----
service:yktoracle 192.168.10.1 started
