MHA配置搭建
发表于:2025-01-21 作者:千家信息网编辑
千家信息网最后更新 2025年01月21日,1、一、MHA部署环境:MHA_Manager: 192.168.1.100Mysql_Master:10.20.20.12Mysql_Slave-1:10.20.20.13Mysql_Slave-2
千家信息网最后更新 2025年01月21日MHA配置搭建1、一、MHA部署环境:
MHA_Manager: 192.168.1.100
Mysql_Master:10.20.20.12
Mysql_Slave-1:10.20.20.13
Mysql_Slave-2:10.20.20.14
软件包下载地址:
http://yunpan.cn/c3S5IAKdhqXJe 访问密码 5ea4
注意:关闭防火墙、selinux
二、准备阶段:
step1:建立信任关系:
在Node:10.20.20.12上:
[root@mysqlmaster /]#ssh-keygen -t rsa
[root@mysqlmaster /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.12
[root@mysqlmaster /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.13
[root@mysqlmaster /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.14
在Node:10.20.20.13上:
[root@mysqlslave1 /]#ssh-keygen -t rsa
[root@mysqlslave1 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.12
[root@mysqlslave1 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.13
[root@mysqlslave1 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.14
在Node:10.20.20.14上:
[root@mysqlslave2 /]#ssh-keygen -t rsa
[root@mysqlslave2 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.12
[root@mysqlslave2 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.13
[root@mysqlslave2 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.14
step2:部署mysql服务(源码搭建)
1、安装编译代码需要的包
yum -y install make gcc-c++ cmake bison-devel ncurses-devel
2、下载MySQL 5.6.14
wget http://cdn.mysql.com/Downloads/MySQL-5.6/mysql-5.6.27.tar.gz
tar xvf mysql-5.6.17.tar.gz
cd mysql-5.6.27
3、编译安装
cmake \
-DCMAKE_INSTALL_PREFIX=/usr/local/mysql \
-DMYSQL_DATADIR=/data/mysql \
-DSYSCONFDIR=/etc \
-DWITH_MYISAM_STORAGE_ENGINE=1 \
-DWITH_INNOBASE_STORAGE_ENGINE=1 \
-DWITH_MEMORY_STORAGE_ENGINE=1 \
-DWITH_READLINE=1 \
-DMYSQL_UNIX_ADDR=/usr/local/mysql/mysql.sock \
-DMYSQL_TCP_PORT=3306 \
-DENABLED_LOCAL_INFILE=1 \
-DWITH_PARTITION_STORAGE_ENGINE=1 \
-DEXTRA_CHARSETS=all \
-DDEFAULT_CHARSET=utf8 \
-DDEFAULT_COLLATION=utf8_general_ci
make && make install
配置MySQL
创建用户组
groupadd mysql
useradd -g mysql mysql
修改/usr/local/mysql权限
chown -R mysql:mysql /usr/local/mysql
chown -R mysql:mysql /data
配置my.cnf(详见conf文件)
以下是简单的my.cnf配置
[root@mysqlmaster /]# cat /etc/my.cnf
[mysqld]
log-bin=mysql-bin
server-id=12
skip-name-resolve
[root@mysqlslave1 ~]# cat /etc/my.cnf
[mysqld]
log-bin=mysql-bin
server-id=13
read-only=1
relay_log_purge=0
skip-name-resolve
[root@mysqlslave2 .ssh]# cat /etc/my.cnf
[mysqld]
log-bin=mysql-bin
server-id=14
read-only=1
relay_log_purge=0
skip-name-resolve
初始化配置
cd /usr/local/mysql
进入安装路径,执行初始化配置脚本,创建系统自带的数据库和表
scripts/mysql_install_db --basedir=/usr/local/mysql --datadir=/data/mysql --user=mysql
启动MySQL
cp support-files/mysql.server /etc/init.d/mysql
chkconfig mysql on
service mysql start
配置环境变量
vi /etc/profile
export PATH=/usr/local/mysql/bin:$PATH
生效环境变量
source /etc/profile
执行下面的命令修改root密码
mysql -uroot
mysql> SET PASSWORD = PASSWORD('123456');
若要设置root用户可以远程访问,执行
mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'172.16.%' IDENTIFIED BY 'password' WITH GRANT OPTION;
三、创建MySQL主从复制
step1:创建数据库管理账号和复制账号(主库、备库 都执行)
grant all privileges on *.* to 'root'@'10.20.20.%' identified by '123456';
grant Replication slave,super on *.* to 'repler'@'10.20.20.%' identified by '123.com';
grant all privileges on *.* to 'root'@'192.168.1.%' identified by '123456';
flush privileges;
step2:建立mysql之间的主从复制
登录主服务器的mysql,查询master的状态
mysql>show master status;
+------------------+----------+--------------+------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+------------------+----------+--------------+------------------+
| mysql-bin.000001 | 308 | | |
+------------------+----------+--------------+------------------+
配置从服务器Slave
mysql>change master to master_host='10.20.20.12',master_user='repler',master_password='123.com',master_log_file='mysql-bin.000001',master_log_pos=308;
四、创建软链接和设置从库为read only
step1:ln -s /usr/local/mysql/bin/* /usr/local/bin/
注意:需要做这个软连接,否则再执行masterha_check_repl校验的时候,会报Can't exec "mysqlbinlog": 没有那个文件或目录 的错误。
step2:所有从服务器上my.cnf里面添加:read-only=1和relay_log_purge=0
五、安装MHA_Manager
1. 安装epel源
rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm
yum --disablerepo=epel -y update ca-certificates
2. 安装依赖包
yum -y install perl-DBD-MySQL perl-Config-Tiny perl-Log-Dispatch perl-Params-Validate perl-TimeDate perl-ExtUtils-CBuilder perl-ExtUtils-MakeMaker perl-CPAN perl-Parallel-ForkManager perl-Log-Message
yum -y install perl-NTLM.noarch --如果要发邮件send_report,需要安装这个包,否则就不发邮件
3、 安装MHA_Node(MHA_Manager需要)
tar -zxvf mha4mysql-manager-0.57.tar.gz
cd mha4mysql-manager-0.57
perl Makefile.PL
make
make install
tar -zxf Test-Simple-1.001014.tar.gz
cd Test-Simple-1.001014
perl Makefile.PL
make
make install
tar -zxf CPAN-Perl-Releases-2.48.tar.gz
cd CPAN-Perl-Releases-2.48
perl Makefile.PL
make
make install
tar -zxf Log-Dispatch-Perl-0.04.tar.gz
cd Log-Dispatch-Perl-0.04
perl Makefile.PL
make
make install
mkdir /etc/masterha/
mkdir /etc/masterha/scripts
mkdir /masterha/app1 -p
六、安装MHA_Node(在所有的Mysql服务器上)
1. 安装epel源
rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm
yum --disablerepo=epel -y update ca-certificates
2. 安装依赖包
yum -y install perl-DBD-MySQL perl-Config-Tiny perl-Log-Dispatch perl-Params-Validate perl-TimeDate perl-ExtUtils-CBuilder perl-ExtUtils-MakeMaker perl-CPAN perl-Parallel-ForkManager
3、 安装MHA_Node
tar -zxf mha4mysql-node-0.57.tar.gz
cd mha4mysql-node-0.57
perl Makefile.PL
make
make install
mkdir /masterha/app1 -p
七、配置MHA_Manager(192.168.1.100):
vim /etc/masterha/app1.cnf
[server default]
#workdir on the management server
manager_workdir=/masterha/app1/
manager_log=/masterha/app1/manager.log
#workdir on the node for mysql server
remote_workdir=/masterha/app1
master_binlog_dir=/data/mysql
check_repl_delay=0
#mysql user and password
user=root
password=123456
#replication_user
repl_user=repler
repl_password=123.com
#checking master every second
ping_interval=1
#ssh user
ssh_user=root
#检测master的可用性
secondary_check_script=masterha_secondary_check -s 10.20.20.13 -s 10.20.20.14
#自动故障VIP切换调用脚本
master_ip_failover_script=/etc/masterha/scripts/master_ip_failover
#手动故障切换调用脚本
master_ip_online_change_script=/etc/masterha/scripts/master_ip_online_change
#发送邮件脚本
report_script=/mha/scripts/send_report
[server1]
hostname=10.20.20.12
candidate_master=1
[server2]
hostname=10.20.20.13
candidate_master=1
[server3]
hostname=10.20.20.14
no_master=1
2、自动故障VIP切换调用脚本
vi /etc/masterha/scripts/master_ip_failover (在MHA_manager 节点上)
#!/usr/bin/env perl
use strict;
use warnings FATAL => 'all';
use Getopt::Long;
my (
$command, $ssh_user, $orig_master_host, $orig_master_ip,
$orig_master_port, $new_master_host, $new_master_ip, $new_master_port
);
my $vip = '192.168.1.100'; #Virtual IP
my $gateway = '192.168.1.1'; #Gateway IP
my $interface = 'eth0'; #bind to interface
my $key = "1";
my $ssh_stop_vip = "/sbin/ifconfig $interface:$key down";
GetOptions(
'command=s' => \$command,
'ssh_user=s' => \$ssh_user,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
);
exit &main();
sub main {
print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";
if ( $command eq "stop" || $command eq "stopssh" ) {
# $orig_master_host, $orig_master_ip, $orig_master_port are passed.
# If you manage master ip address at global catalog database,
# invalidate orig_master_ip here.
my $exit_code = 1;
eval {
print "Disabling the VIP on old master: $orig_master_host \n";
&stop_vip();
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {
# all arguments are passed.
# If you manage master ip address at global catalog database,
# activate new_master_ip here.
# You can also grant write access (create user, set read_only=0, etc) here.
my $exit_code = 10;
eval {
print "Enabling the VIP - $vip on the new master - $new_master_host \n";
&start_vip();
$exit_code = 0;
};
if ($@) {
warn $@;
exit $exit_code;
}
elsif ( $command eq "status" ) {
print "Checking the Status of the script.. OK \n";
`ssh $ssh_user\@$orig_master_host \" $ssh_start_vip \"`;
exit 0;
}
else {
&usage();
exit 1;
}
}
# A simple system call that enable the VIP on the new master
sub start_vip() {
`ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# A simple system call that disable the VIP on the old_master
sub stop_vip() {
`ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}
sub usage {
print
"Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
}
3、手动故障切换调用脚本 (在MHA_manager)
vi /etc/masterha/scripts/master_ip_online_change
#!/usr/bin/env perl
# Copyright (C) 2011 DeNA Co.,Ltd.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
## Note: This is a sample script and is not complete. Modify the script based on your environment.
use strict;
use warnings FATAL => 'all';
use Getopt::Long;
use MHA::DBHelper;
use MHA::NodeUtil;
use Time::HiRes qw( sleep gettimeofday tv_interval );
use Data::Dumper;
my $_tstart;
my $_running_interval = 0.1;
my (
$command, $orig_master_host, $orig_master_ip,
$orig_master_port, $orig_master_user, $orig_master_ssh_user,
$new_master_host, $new_master_ip, $new_master_port,
$new_master_user, $new_master_ssh_user, $orig_master_is_new_slave,
$orig_master_password, $new_master_password,
);
my $vip = '192.168.1.100'; ###Virtual IP
my $gateway = '192.168.1.1'; ###Gateway IP
my $interface = 'eth0'; ###interface
my $key = "1";
my $ssh_start_vip = "/sbin/ifconfig $interface:$key $vip/24";
my $flush_arp = "/sbin/arping -I $interface -c 2 -s $vip $gateway";
my $ssh_stop_vip = "/sbin/ifconfig $interface:$key down";
my $ssh_user = "root";
GetOptions(
'command=s' => \$command,
'ssh_user=s' => \$ssh_user,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'orig_master_user=s' => \$orig_master_user,
'orig_master_password=s' => \$orig_master_password,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
'new_master_user=s' => \$new_master_user,
'new_master_password=s' => \$new_master_password,
'orig_master_ssh_user=s' => \$orig_master_ssh_user,
'new_master_ssh_user=s' => \$new_master_ssh_user,
'orig_master_is_new_slave' => \$orig_master_is_new_slave,
);
exit &main();
sub current_time_us {
my ( $sec, $microsec ) = gettimeofday();
my $curdate = localtime($sec);
return $curdate . " " . sprintf( "d", $microsec );
}
sub sleep_until {
my $elapsed = tv_interval($_tstart);
if ( $_running_interval > $elapsed ) {
sleep( $_running_interval - $elapsed );
}
}
sub get_threads_util {
my $dbh = shift;
my $my_connection_id = shift;
my $running_time_threshold = shift;
my $type = shift;
$running_time_threshold = 0 unless ($running_time_threshold);
$type = 0 unless ($type);
my @threads;
my $sth = $dbh->prepare("SHOW PROCESSLIST");
$sth->execute();
while ( my $ref = $sth->fetchrow_hashref() ) {
my $id = $ref->{Id};
my $user = $ref->{User};
my $host = $ref->{Host};
my $command = $ref->{Command};
my $state = $ref->{State};
my $query_time = $ref->{Time};
my $info = $ref->{Info};
$info =~ s/^\s*(.*?)\s*$/$1/ if defined($info);
next if ( $my_connection_id == $id );
next if ( defined($query_time) && $query_time < $running_time_threshold );
next if ( defined($command) && $command eq "Binlog Dump" );
next if ( defined($user) && $user eq "system user" );
next
if ( defined($command)
&& $command eq "Sleep"
&& defined($query_time)
&& $query_time >= 1 );
if ( $type >= 1 ) {
next if ( defined($command) && $command eq "Sleep" );
next if ( defined($command) && $command eq "Connect" );
}
if ( $type >= 2 ) {
next if ( defined($info) && $info =~ m/^select/i );
next if ( defined($info) && $info =~ m/^show/i );
}
push @threads, $ref;
}
return @threads;
}
sub main {
if ( $command eq "stop" ) {
## Gracefully killing connections on the current master
# 1. Set read_only= 1 on the new master
# 2. DROP USER so that no app user can establish new connections
# 3. Set read_only= 1 on the current master
# 4. Kill current queries
# * Any database access failure will result in script die.
my $exit_code = 1;
eval {
## Setting read_only=1 on the new master (to avoid accident)
my $new_master_handler = new MHA::DBHelper();
# args: hostname, port, user, password, raise_error(die_on_error)_or_not
$new_master_handler->connect( $new_master_ip, $new_master_port,
$new_master_user, $new_master_password, 1 );
print current_time_us() . " Set read_only on the new master.. ";
$new_master_handler->enable_read_only();
if ( $new_master_handler->is_read_only() ) {
print "ok.\n";
}
else {
die "Failed!\n";
}
$new_master_handler->disconnect();
# Connecting to the orig master, die if any database error happens
my $orig_master_handler = new MHA::DBHelper();
$orig_master_handler->connect( $orig_master_ip, $orig_master_port,
$orig_master_user, $orig_master_password, 1 );
## Drop application user so that nobody can connect. Disabling per-session binlog beforehand
#$orig_master_handler->disable_log_bin_local();
#print current_time_us() . " Drpping app user on the orig master..\n";
#FIXME_xxx_drop_app_user($orig_master_handler);
## Waiting for N * 100 milliseconds so that current connections can exit
my $time_until_read_only = 15;
$_tstart = [gettimeofday];
my @threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
while ( $time_until_read_only > 0 && $#threads >= 0 ) {
if ( $time_until_read_only % 5 == 0 ) {
printf
"%s Waiting all running %d threads are disconnected.. (max %d milliseconds)\n",
current_time_us(), $#threads + 1, $time_until_read_only * 100;
if ( $#threads < 5 ) {
print Data::Dumper->new( [$_] )->Indent(0)->Terse(1)->Dump . "\n"
foreach (@threads);
}
}
sleep_until();
$_tstart = [gettimeofday];
$time_until_read_only--;
@threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
}
## Setting read_only=1 on the current master so that nobody(except SUPER) can write
print current_time_us() . " Set read_only=1 on the orig master.. ";
$orig_master_handler->enable_read_only();
if ( $orig_master_handler->is_read_only() ) {
print "ok.\n";
}
else {
die "Failed!\n";
}
## Waiting for M * 100 milliseconds so that current update queries can complete
my $time_until_kill_threads = 5;
@threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
while ( $time_until_kill_threads > 0 && $#threads >= 0 ) {
if ( $time_until_kill_threads % 5 == 0 ) {
printf
"%s Waiting all running %d queries are disconnected.. (max %d milliseconds)\n",
current_time_us(), $#threads + 1, $time_until_kill_threads * 100;
if ( $#threads < 5 ) {
print Data::Dumper->new( [$_] )->Indent(0)->Terse(1)->Dump . "\n"
foreach (@threads);
}
}
sleep_until();
$_tstart = [gettimeofday];
$time_until_kill_threads--;
@threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
&stop_vip();
## Terminating all threads
print current_time_us() . " Killing all application threads..\n";
$orig_master_handler->kill_threads(@threads) if ( $#threads >= 0 );
print current_time_us() . " done.\n";
#$orig_master_handler->enable_log_bin_local();
$orig_master_handler->disconnect();
## After finishing the script, MHA executes FLUSH TABLES WITH READ LOCK
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {
## Activating master ip on the new master
# 1. Create app user with write privileges
# 2. Moving backup script if needed
# 3. Register new master's ip to the catalog database
# If exit code is 0 or 10, MHA does not abort
my $exit_code = 10;
eval {
my $new_master_handler = new MHA::DBHelper();
# args: hostname, port, user, password, raise_error_or_not
$new_master_handler->connect( $new_master_ip, $new_master_port,
$new_master_user, $new_master_password, 1 );
## Set read_only=0 on the new master
#$new_master_handler->disable_log_bin_local();
print current_time_us() . " Set read_only=0 on the new master.\n";
$new_master_handler->disable_read_only();
## Creating an app user on the new master
#print current_time_us() . " Creating app user on the new master..\n";
#FIXME_xxx_create_app_user($new_master_handler);
#$new_master_handler->enable_log_bin_local();
$new_master_handler->disconnect();
## Update master ip on the catalog database, etc
print "Enabling the VIP - $vip on the new master - $new_master_host \n";
&start_vip();
&flush_arp();
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
exit 0;
}
else {
&usage();
exit 1;
}
}
# A simple system call that enable the VIP on the new master
sub start_vip() {
`ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# A simple system call that disable the VIP on the old_master
sub stop_vip() {
`ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}
#flush arp
sub flush_arp() {
`ssh $ssh_user\@$new_master_host \" $flush_arp \"`;
}
sub usage {
print
"Usage: master_ip_online_change --command=start|stop|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
die;
}
4、自动发邮件脚本(自动切换后会触发邮件, 在MHA_manager)
vi /etc/masterha/scripts/send_report
#!/usr/bin/perl
# Copyright (C) 2011 DeNA Co.,Ltd.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
## Note: This is a sample script and is not complete. Modify the script based on your environment.
use strict;
use warnings FATAL => 'all';
use Mail::Sender;
use Getopt::Long;
#new_master_host and new_slave_hosts are set only when recovering master succeeded
my ( $dead_master_host, $new_master_host, $new_slave_hosts, $subject, $body );
my $smtp='smtp.chinadaas.com';
my $mail_from='support@chinadaas.com';
my $mail_user='support@chinadaas.com';
my $mail_pass='ZS_123456';
#my $mail_to=['chenzhibin@chinadaas.com'];
my $mail_to='chenzhibin@chinadaas.com';
GetOptions(
'orig_master_host=s' => \$dead_master_host,
'new_master_host=s' => \$new_master_host,
'new_slave_hosts=s' => \$new_slave_hosts,
'subject=s' => \$subject,
'body=s' => \$body,
);
# Do whatever you want here
mailToContacts($smtp,$mail_from,$mail_user,$mail_pass,$mail_to,$subject,$body);
sub mailToContacts {
my ($smtp, $mail_from, $mail_user, $mail_pass, $mail_to, $subject, $msg ) = @_;
open my $DEBUG, ">/var/log/masterha/app1/mail.log"
or die "Can't open the debug file:$!\n";
my $sender = new Mail::Sender {
ctype => 'text/plain;charset=utf-8',
encoding => 'utf-8',
smtp => $smtp,
from => $mail_from,
auth => 'LOGIN',
TLS_allowed => '0',
authid => $mail_user,
authpwd => $mail_pass,
to => $mail_to,
subject => $subject,
debug => $DEBUG
};
$sender->MailMsg(
{
msg => $msg,
debug => $DEBUG
}
) or print $Mail::Sender::Error;
return 1;
}
exit 0;
5、赋予执行权限
chmod +x /etc/masterha/scripts/master_ip_failover
chmod +x /etc/masterha/scripts/master_ip_online_change
chmod +x /etc/masterha/scripts/send_report
八、检查状态
检查ssh认证状态(在mha_manager上)
masterha_check_ssh --conf=/etc/masterha/app1.cnf
检查复制状态(在mha_manager上)
masterha_check_repl --conf=/etc/masterha/app1.cnf
九、启动和关闭管理节点
可以通过如下命令启动mha:
nohup masterha_manager --conf=/etc/masterha/app1.cnf >> /dev/null 2>&1 &
可以通过如下命令停止mha
masterha_stop --conf=/etc/app1.cnf
十、查看管理节点启动后的服务:
ps -ef |grep master_ip_
ps -ef |grep masterha
或
masterha_check_status -conf=/etc/masterha/app1.cnf
或
查看管理节点日志信息
tail -f /masterha/app1/manager.log
十一、自动切换测试
在主上建立VIP(只有在故障时,才会触发脚本,才会生成VIP):
第一次需要在主库Mysql_Master:10.101.110.90上手工绑定vip
ifconfig eth0:1 10.101.110.10/24
宕掉主库(service mysql stop),可以通过日志(tail -f /masterha/app1/manager.log),观察到自动提升10.20.20.13为主,并将其余的从库的主库切为了10.20.20.13
同时,VIP在新主10.20.20.13上自动创建
注意一下事项:
发生主备切换后,manager服务会自动停掉,且在/masterha/app1下面生成
app1.failover.complete,若再次发生切换需要删除app1.failover.complete文件
rm -fr /masterha/app1/app1.failover.complete
*****如果自动切换后,需要手工执行mha manager启动脚本,进行监控master 节点,并且删除 rm -fr /masterha/app1/app1.failover.complete
常情况下自动切换以后,原master可能已经废弃掉,待原master主机修复后,如果数据完整的情况下,
想把原来master重新作为新主库的slave,这时我们可以借助当时自动切换时刻的MHA日志来完成对原master的修复
下面是提取相关日志的命令:(在mha_manager上)
原主库修复,变为备库需要以下步骤
mha manager节点
[root@mha manager]# grep -i "All other slaves should start" /masterha/app1/manager.log
Mon Apr 21 22:28:33 2014 - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='10.20.20.13', (新主库的ip)MASTER_PORT=3306, MASTER_LOG_FILE='mysql-bin.000022', MASTER_LOG_POS=506716, MASTER_USER='repl', MASTER_PASSWORD='xxx';
获取上述信息以后,就可以直接在修复后的原master上执行change master to相关操作,重新作为从库了
十二、手工切换
有时候需要手动切换MySQL主服务器,可以使用masterha_master_switch命令,不过它调用的不是master_ip_failover_script脚本,而是master_ip_online_change_script脚本,但调用参数类似,脚本可以互用。
在mha_manager机器上:
masterha_master_switch --master_state=alive --conf=/etc/masterha/app1.cnf --new_master_host=10.20.20.12 --new_master_port=3306 --orig_master_is_new_slave --running_updates_limit=10000 --interactive=0
十三清除relay log (备库恢复的binlog日志)
对于MHA场景下,对于某些滞后从库的恢复依赖于其他从库的relay log,因此采取禁用自动删除功能以及定期清理的办法。对于清理过多过大的relay log需要注意引起的复制延迟资源开销等。MHA可通过purge_relay_logs脚本及配合cronjob来完成此项任务
在几个MHA node节点上(MHA manager不需要):
添加如下脚本和任务执行计划:
[root@JBWDB03 shell]# cat /usr/local/shell/purge_relay_logs.sh
#!/bin/bash
user=root
passwd='-s4)SVqJ'
port=3306
log_dir='/etc/masterha/log'
work_dir='/etc/masterha/data'
purge='/usr/local/bin/purge_relay_logs'
if [ ! -d $log_dir ]
then
mkdir $log_dir -p
fi
if [ ! -d $work_dir ]
then
mkdir $work_dir -p
fi
$purge --user=$user --password=$passwd --disable_relay_log_purge --port=$port --workdir=$work_dir >> $log_dir/purge_relay_logs.log 2>&1
[root@JBWDB03 shell]# crontab -l
##每天凌晨1点清空relay log
00 01 */1 * * /bin/bash /usr/local/shell/purge_relay_logs.sh
MHA_Manager: 192.168.1.100
Mysql_Master:10.20.20.12
Mysql_Slave-1:10.20.20.13
Mysql_Slave-2:10.20.20.14
软件包下载地址:
http://yunpan.cn/c3S5IAKdhqXJe 访问密码 5ea4
注意:关闭防火墙、selinux
二、准备阶段:
step1:建立信任关系:
在Node:10.20.20.12上:
[root@mysqlmaster /]#ssh-keygen -t rsa
[root@mysqlmaster /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.12
[root@mysqlmaster /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.13
[root@mysqlmaster /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.14
在Node:10.20.20.13上:
[root@mysqlslave1 /]#ssh-keygen -t rsa
[root@mysqlslave1 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.12
[root@mysqlslave1 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.13
[root@mysqlslave1 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.14
在Node:10.20.20.14上:
[root@mysqlslave2 /]#ssh-keygen -t rsa
[root@mysqlslave2 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.12
[root@mysqlslave2 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.13
[root@mysqlslave2 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.14
step2:部署mysql服务(源码搭建)
1、安装编译代码需要的包
yum -y install make gcc-c++ cmake bison-devel ncurses-devel
2、下载MySQL 5.6.14
wget http://cdn.mysql.com/Downloads/MySQL-5.6/mysql-5.6.27.tar.gz
tar xvf mysql-5.6.17.tar.gz
cd mysql-5.6.27
3、编译安装
cmake \
-DCMAKE_INSTALL_PREFIX=/usr/local/mysql \
-DMYSQL_DATADIR=/data/mysql \
-DSYSCONFDIR=/etc \
-DWITH_MYISAM_STORAGE_ENGINE=1 \
-DWITH_INNOBASE_STORAGE_ENGINE=1 \
-DWITH_MEMORY_STORAGE_ENGINE=1 \
-DWITH_READLINE=1 \
-DMYSQL_UNIX_ADDR=/usr/local/mysql/mysql.sock \
-DMYSQL_TCP_PORT=3306 \
-DENABLED_LOCAL_INFILE=1 \
-DWITH_PARTITION_STORAGE_ENGINE=1 \
-DEXTRA_CHARSETS=all \
-DDEFAULT_CHARSET=utf8 \
-DDEFAULT_COLLATION=utf8_general_ci
make && make install
配置MySQL
创建用户组
groupadd mysql
useradd -g mysql mysql
修改/usr/local/mysql权限
chown -R mysql:mysql /usr/local/mysql
chown -R mysql:mysql /data
配置my.cnf(详见conf文件)
以下是简单的my.cnf配置
[root@mysqlmaster /]# cat /etc/my.cnf
[mysqld]
log-bin=mysql-bin
server-id=12
skip-name-resolve
[root@mysqlslave1 ~]# cat /etc/my.cnf
[mysqld]
log-bin=mysql-bin
server-id=13
read-only=1
relay_log_purge=0
skip-name-resolve
[root@mysqlslave2 .ssh]# cat /etc/my.cnf
[mysqld]
log-bin=mysql-bin
server-id=14
read-only=1
relay_log_purge=0
skip-name-resolve
初始化配置
cd /usr/local/mysql
进入安装路径,执行初始化配置脚本,创建系统自带的数据库和表
scripts/mysql_install_db --basedir=/usr/local/mysql --datadir=/data/mysql --user=mysql
启动MySQL
cp support-files/mysql.server /etc/init.d/mysql
chkconfig mysql on
service mysql start
配置环境变量
vi /etc/profile
export PATH=/usr/local/mysql/bin:$PATH
生效环境变量
source /etc/profile
执行下面的命令修改root密码
mysql -uroot
mysql> SET PASSWORD = PASSWORD('123456');
若要设置root用户可以远程访问,执行
mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'172.16.%' IDENTIFIED BY 'password' WITH GRANT OPTION;
三、创建MySQL主从复制
step1:创建数据库管理账号和复制账号(主库、备库 都执行)
grant all privileges on *.* to 'root'@'10.20.20.%' identified by '123456';
grant Replication slave,super on *.* to 'repler'@'10.20.20.%' identified by '123.com';
grant all privileges on *.* to 'root'@'192.168.1.%' identified by '123456';
flush privileges;
step2:建立mysql之间的主从复制
登录主服务器的mysql,查询master的状态
mysql>show master status;
+------------------+----------+--------------+------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+------------------+----------+--------------+------------------+
| mysql-bin.000001 | 308 | | |
+------------------+----------+--------------+------------------+
配置从服务器Slave
mysql>change master to master_host='10.20.20.12',master_user='repler',master_password='123.com',master_log_file='mysql-bin.000001',master_log_pos=308;
四、创建软链接和设置从库为read only
step1:ln -s /usr/local/mysql/bin/* /usr/local/bin/
注意:需要做这个软连接,否则再执行masterha_check_repl校验的时候,会报Can't exec "mysqlbinlog": 没有那个文件或目录 的错误。
step2:所有从服务器上my.cnf里面添加:read-only=1和relay_log_purge=0
五、安装MHA_Manager
1. 安装epel源
rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm
yum --disablerepo=epel -y update ca-certificates
2. 安装依赖包
yum -y install perl-DBD-MySQL perl-Config-Tiny perl-Log-Dispatch perl-Params-Validate perl-TimeDate perl-ExtUtils-CBuilder perl-ExtUtils-MakeMaker perl-CPAN perl-Parallel-ForkManager perl-Log-Message
yum -y install perl-NTLM.noarch --如果要发邮件send_report,需要安装这个包,否则就不发邮件
3、 安装MHA_Node(MHA_Manager需要)
tar -zxvf mha4mysql-manager-0.57.tar.gz
cd mha4mysql-manager-0.57
perl Makefile.PL
make
make install
tar -zxf Test-Simple-1.001014.tar.gz
cd Test-Simple-1.001014
perl Makefile.PL
make
make install
tar -zxf CPAN-Perl-Releases-2.48.tar.gz
cd CPAN-Perl-Releases-2.48
perl Makefile.PL
make
make install
tar -zxf Log-Dispatch-Perl-0.04.tar.gz
cd Log-Dispatch-Perl-0.04
perl Makefile.PL
make
make install
mkdir /etc/masterha/
mkdir /etc/masterha/scripts
mkdir /masterha/app1 -p
六、安装MHA_Node(在所有的Mysql服务器上)
1. 安装epel源
rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm
yum --disablerepo=epel -y update ca-certificates
2. 安装依赖包
yum -y install perl-DBD-MySQL perl-Config-Tiny perl-Log-Dispatch perl-Params-Validate perl-TimeDate perl-ExtUtils-CBuilder perl-ExtUtils-MakeMaker perl-CPAN perl-Parallel-ForkManager
3、 安装MHA_Node
tar -zxf mha4mysql-node-0.57.tar.gz
cd mha4mysql-node-0.57
perl Makefile.PL
make
make install
mkdir /masterha/app1 -p
七、配置MHA_Manager(192.168.1.100):
vim /etc/masterha/app1.cnf
[server default]
#workdir on the management server
manager_workdir=/masterha/app1/
manager_log=/masterha/app1/manager.log
#workdir on the node for mysql server
remote_workdir=/masterha/app1
master_binlog_dir=/data/mysql
check_repl_delay=0
#mysql user and password
user=root
password=123456
#replication_user
repl_user=repler
repl_password=123.com
#checking master every second
ping_interval=1
#ssh user
ssh_user=root
#检测master的可用性
secondary_check_script=masterha_secondary_check -s 10.20.20.13 -s 10.20.20.14
#自动故障VIP切换调用脚本
master_ip_failover_script=/etc/masterha/scripts/master_ip_failover
#手动故障切换调用脚本
master_ip_online_change_script=/etc/masterha/scripts/master_ip_online_change
#发送邮件脚本
report_script=/mha/scripts/send_report
[server1]
hostname=10.20.20.12
candidate_master=1
[server2]
hostname=10.20.20.13
candidate_master=1
[server3]
hostname=10.20.20.14
no_master=1
2、自动故障VIP切换调用脚本
vi /etc/masterha/scripts/master_ip_failover (在MHA_manager 节点上)
#!/usr/bin/env perl
use strict;
use warnings FATAL => 'all';
use Getopt::Long;
my (
$command, $ssh_user, $orig_master_host, $orig_master_ip,
$orig_master_port, $new_master_host, $new_master_ip, $new_master_port
);
my $vip = '192.168.1.100'; #Virtual IP
my $gateway = '192.168.1.1'; #Gateway IP
my $interface = 'eth0'; #bind to interface
my $key = "1";
my $ssh_stop_vip = "/sbin/ifconfig $interface:$key down";
GetOptions(
'command=s' => \$command,
'ssh_user=s' => \$ssh_user,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
);
exit &main();
sub main {
print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";
if ( $command eq "stop" || $command eq "stopssh" ) {
# $orig_master_host, $orig_master_ip, $orig_master_port are passed.
# If you manage master ip address at global catalog database,
# invalidate orig_master_ip here.
my $exit_code = 1;
eval {
print "Disabling the VIP on old master: $orig_master_host \n";
&stop_vip();
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {
# all arguments are passed.
# If you manage master ip address at global catalog database,
# activate new_master_ip here.
# You can also grant write access (create user, set read_only=0, etc) here.
my $exit_code = 10;
eval {
print "Enabling the VIP - $vip on the new master - $new_master_host \n";
&start_vip();
$exit_code = 0;
};
if ($@) {
warn $@;
exit $exit_code;
}
elsif ( $command eq "status" ) {
print "Checking the Status of the script.. OK \n";
`ssh $ssh_user\@$orig_master_host \" $ssh_start_vip \"`;
exit 0;
}
else {
&usage();
exit 1;
}
}
# A simple system call that enable the VIP on the new master
sub start_vip() {
`ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# A simple system call that disable the VIP on the old_master
sub stop_vip() {
`ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}
sub usage {
"Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
}
3、手动故障切换调用脚本 (在MHA_manager)
vi /etc/masterha/scripts/master_ip_online_change
#!/usr/bin/env perl
# Copyright (C) 2011 DeNA Co.,Ltd.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
## Note: This is a sample script and is not complete. Modify the script based on your environment.
use strict;
use warnings FATAL => 'all';
use Getopt::Long;
use MHA::DBHelper;
use MHA::NodeUtil;
use Time::HiRes qw( sleep gettimeofday tv_interval );
use Data::Dumper;
my $_tstart;
my $_running_interval = 0.1;
my (
$command, $orig_master_host, $orig_master_ip,
$orig_master_port, $orig_master_user, $orig_master_ssh_user,
$new_master_host, $new_master_ip, $new_master_port,
$new_master_user, $new_master_ssh_user, $orig_master_is_new_slave,
$orig_master_password, $new_master_password,
);
my $vip = '192.168.1.100'; ###Virtual IP
my $gateway = '192.168.1.1'; ###Gateway IP
my $interface = 'eth0'; ###interface
my $key = "1";
my $ssh_start_vip = "/sbin/ifconfig $interface:$key $vip/24";
my $flush_arp = "/sbin/arping -I $interface -c 2 -s $vip $gateway";
my $ssh_stop_vip = "/sbin/ifconfig $interface:$key down";
my $ssh_user = "root";
GetOptions(
'command=s' => \$command,
'ssh_user=s' => \$ssh_user,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'orig_master_user=s' => \$orig_master_user,
'orig_master_password=s' => \$orig_master_password,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
'new_master_user=s' => \$new_master_user,
'new_master_password=s' => \$new_master_password,
'orig_master_ssh_user=s' => \$orig_master_ssh_user,
'new_master_ssh_user=s' => \$new_master_ssh_user,
'orig_master_is_new_slave' => \$orig_master_is_new_slave,
);
exit &main();
sub current_time_us {
my ( $sec, $microsec ) = gettimeofday();
my $curdate = localtime($sec);
return $curdate . " " . sprintf( "d", $microsec );
}
sub sleep_until {
my $elapsed = tv_interval($_tstart);
if ( $_running_interval > $elapsed ) {
sleep( $_running_interval - $elapsed );
}
}
sub get_threads_util {
my $dbh = shift;
my $my_connection_id = shift;
my $running_time_threshold = shift;
my $type = shift;
$running_time_threshold = 0 unless ($running_time_threshold);
$type = 0 unless ($type);
my @threads;
my $sth = $dbh->prepare("SHOW PROCESSLIST");
$sth->execute();
while ( my $ref = $sth->fetchrow_hashref() ) {
my $id = $ref->{Id};
my $user = $ref->{User};
my $host = $ref->{Host};
my $command = $ref->{Command};
my $state = $ref->{State};
my $query_time = $ref->{Time};
my $info = $ref->{Info};
$info =~ s/^\s*(.*?)\s*$/$1/ if defined($info);
next if ( $my_connection_id == $id );
next if ( defined($query_time) && $query_time < $running_time_threshold );
next if ( defined($command) && $command eq "Binlog Dump" );
next if ( defined($user) && $user eq "system user" );
next
if ( defined($command)
&& $command eq "Sleep"
&& defined($query_time)
&& $query_time >= 1 );
if ( $type >= 1 ) {
next if ( defined($command) && $command eq "Sleep" );
next if ( defined($command) && $command eq "Connect" );
}
if ( $type >= 2 ) {
next if ( defined($info) && $info =~ m/^select/i );
next if ( defined($info) && $info =~ m/^show/i );
}
push @threads, $ref;
}
return @threads;
}
sub main {
if ( $command eq "stop" ) {
## Gracefully killing connections on the current master
# 1. Set read_only= 1 on the new master
# 2. DROP USER so that no app user can establish new connections
# 3. Set read_only= 1 on the current master
# 4. Kill current queries
# * Any database access failure will result in script die.
my $exit_code = 1;
eval {
## Setting read_only=1 on the new master (to avoid accident)
my $new_master_handler = new MHA::DBHelper();
# args: hostname, port, user, password, raise_error(die_on_error)_or_not
$new_master_handler->connect( $new_master_ip, $new_master_port,
$new_master_user, $new_master_password, 1 );
print current_time_us() . " Set read_only on the new master.. ";
$new_master_handler->enable_read_only();
if ( $new_master_handler->is_read_only() ) {
print "ok.\n";
}
else {
die "Failed!\n";
}
$new_master_handler->disconnect();
# Connecting to the orig master, die if any database error happens
my $orig_master_handler = new MHA::DBHelper();
$orig_master_handler->connect( $orig_master_ip, $orig_master_port,
$orig_master_user, $orig_master_password, 1 );
## Drop application user so that nobody can connect. Disabling per-session binlog beforehand
#$orig_master_handler->disable_log_bin_local();
#print current_time_us() . " Drpping app user on the orig master..\n";
#FIXME_xxx_drop_app_user($orig_master_handler);
## Waiting for N * 100 milliseconds so that current connections can exit
my $time_until_read_only = 15;
$_tstart = [gettimeofday];
my @threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
while ( $time_until_read_only > 0 && $#threads >= 0 ) {
if ( $time_until_read_only % 5 == 0 ) {
printf
"%s Waiting all running %d threads are disconnected.. (max %d milliseconds)\n",
current_time_us(), $#threads + 1, $time_until_read_only * 100;
if ( $#threads < 5 ) {
print Data::Dumper->new( [$_] )->Indent(0)->Terse(1)->Dump . "\n"
foreach (@threads);
}
}
sleep_until();
$_tstart = [gettimeofday];
$time_until_read_only--;
@threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
}
## Setting read_only=1 on the current master so that nobody(except SUPER) can write
print current_time_us() . " Set read_only=1 on the orig master.. ";
$orig_master_handler->enable_read_only();
if ( $orig_master_handler->is_read_only() ) {
print "ok.\n";
}
else {
die "Failed!\n";
}
## Waiting for M * 100 milliseconds so that current update queries can complete
my $time_until_kill_threads = 5;
@threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
while ( $time_until_kill_threads > 0 && $#threads >= 0 ) {
if ( $time_until_kill_threads % 5 == 0 ) {
printf
"%s Waiting all running %d queries are disconnected.. (max %d milliseconds)\n",
current_time_us(), $#threads + 1, $time_until_kill_threads * 100;
if ( $#threads < 5 ) {
print Data::Dumper->new( [$_] )->Indent(0)->Terse(1)->Dump . "\n"
foreach (@threads);
}
}
sleep_until();
$_tstart = [gettimeofday];
$time_until_kill_threads--;
@threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
&stop_vip();
## Terminating all threads
print current_time_us() . " Killing all application threads..\n";
$orig_master_handler->kill_threads(@threads) if ( $#threads >= 0 );
print current_time_us() . " done.\n";
#$orig_master_handler->enable_log_bin_local();
$orig_master_handler->disconnect();
## After finishing the script, MHA executes FLUSH TABLES WITH READ LOCK
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {
## Activating master ip on the new master
# 1. Create app user with write privileges
# 2. Moving backup script if needed
# 3. Register new master's ip to the catalog database
# If exit code is 0 or 10, MHA does not abort
my $exit_code = 10;
eval {
my $new_master_handler = new MHA::DBHelper();
# args: hostname, port, user, password, raise_error_or_not
$new_master_handler->connect( $new_master_ip, $new_master_port,
$new_master_user, $new_master_password, 1 );
## Set read_only=0 on the new master
#$new_master_handler->disable_log_bin_local();
print current_time_us() . " Set read_only=0 on the new master.\n";
$new_master_handler->disable_read_only();
## Creating an app user on the new master
#print current_time_us() . " Creating app user on the new master..\n";
#FIXME_xxx_create_app_user($new_master_handler);
#$new_master_handler->enable_log_bin_local();
$new_master_handler->disconnect();
## Update master ip on the catalog database, etc
print "Enabling the VIP - $vip on the new master - $new_master_host \n";
&start_vip();
&flush_arp();
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
exit 0;
}
else {
&usage();
exit 1;
}
}
# A simple system call that enable the VIP on the new master
sub start_vip() {
`ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# A simple system call that disable the VIP on the old_master
sub stop_vip() {
`ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}
#flush arp
sub flush_arp() {
`ssh $ssh_user\@$new_master_host \" $flush_arp \"`;
}
sub usage {
"Usage: master_ip_online_change --command=start|stop|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
die;
}
4、自动发邮件脚本(自动切换后会触发邮件, 在MHA_manager)
vi /etc/masterha/scripts/send_report
#!/usr/bin/perl
# Copyright (C) 2011 DeNA Co.,Ltd.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
## Note: This is a sample script and is not complete. Modify the script based on your environment.
use strict;
use warnings FATAL => 'all';
use Mail::Sender;
use Getopt::Long;
#new_master_host and new_slave_hosts are set only when recovering master succeeded
my ( $dead_master_host, $new_master_host, $new_slave_hosts, $subject, $body );
my $smtp='smtp.chinadaas.com';
my $mail_from='support@chinadaas.com';
my $mail_user='support@chinadaas.com';
my $mail_pass='ZS_123456';
#my $mail_to=['chenzhibin@chinadaas.com'];
my $mail_to='chenzhibin@chinadaas.com';
GetOptions(
'orig_master_host=s' => \$dead_master_host,
'new_master_host=s' => \$new_master_host,
'new_slave_hosts=s' => \$new_slave_hosts,
'subject=s' => \$subject,
'body=s' => \$body,
);
# Do whatever you want here
mailToContacts($smtp,$mail_from,$mail_user,$mail_pass,$mail_to,$subject,$body);
sub mailToContacts {
my ($smtp, $mail_from, $mail_user, $mail_pass, $mail_to, $subject, $msg ) = @_;
open my $DEBUG, ">/var/log/masterha/app1/mail.log"
or die "Can't open the debug file:$!\n";
my $sender = new Mail::Sender {
ctype => 'text/plain;charset=utf-8',
encoding => 'utf-8',
smtp => $smtp,
from => $mail_from,
auth => 'LOGIN',
TLS_allowed => '0',
authid => $mail_user,
authpwd => $mail_pass,
to => $mail_to,
subject => $subject,
debug => $DEBUG
};
$sender->MailMsg(
{
msg => $msg,
debug => $DEBUG
}
) or print $Mail::Sender::Error;
return 1;
}
exit 0;
5、赋予执行权限
chmod +x /etc/masterha/scripts/master_ip_failover
chmod +x /etc/masterha/scripts/master_ip_online_change
chmod +x /etc/masterha/scripts/send_report
八、检查状态
检查ssh认证状态(在mha_manager上)
masterha_check_ssh --conf=/etc/masterha/app1.cnf
检查复制状态(在mha_manager上)
masterha_check_repl --conf=/etc/masterha/app1.cnf
九、启动和关闭管理节点
可以通过如下命令启动mha:
nohup masterha_manager --conf=/etc/masterha/app1.cnf >> /dev/null 2>&1 &
可以通过如下命令停止mha
masterha_stop --conf=/etc/app1.cnf
十、查看管理节点启动后的服务:
ps -ef |grep master_ip_
ps -ef |grep masterha
或
masterha_check_status -conf=/etc/masterha/app1.cnf
或
查看管理节点日志信息
tail -f /masterha/app1/manager.log
十一、自动切换测试
在主上建立VIP(只有在故障时,才会触发脚本,才会生成VIP):
第一次需要在主库Mysql_Master:10.101.110.90上手工绑定vip
ifconfig eth0:1 10.101.110.10/24
宕掉主库(service mysql stop),可以通过日志(tail -f /masterha/app1/manager.log),观察到自动提升10.20.20.13为主,并将其余的从库的主库切为了10.20.20.13
同时,VIP在新主10.20.20.13上自动创建
注意一下事项:
发生主备切换后,manager服务会自动停掉,且在/masterha/app1下面生成
app1.failover.complete,若再次发生切换需要删除app1.failover.complete文件
rm -fr /masterha/app1/app1.failover.complete
*****如果自动切换后,需要手工执行mha manager启动脚本,进行监控master 节点,并且删除 rm -fr /masterha/app1/app1.failover.complete
常情况下自动切换以后,原master可能已经废弃掉,待原master主机修复后,如果数据完整的情况下,
想把原来master重新作为新主库的slave,这时我们可以借助当时自动切换时刻的MHA日志来完成对原master的修复
下面是提取相关日志的命令:(在mha_manager上)
原主库修复,变为备库需要以下步骤
mha manager节点
[root@mha manager]# grep -i "All other slaves should start" /masterha/app1/manager.log
Mon Apr 21 22:28:33 2014 - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='10.20.20.13', (新主库的ip)MASTER_PORT=3306, MASTER_LOG_FILE='mysql-bin.000022', MASTER_LOG_POS=506716, MASTER_USER='repl', MASTER_PASSWORD='xxx';
获取上述信息以后,就可以直接在修复后的原master上执行change master to相关操作,重新作为从库了
十二、手工切换
有时候需要手动切换MySQL主服务器,可以使用masterha_master_switch命令,不过它调用的不是master_ip_failover_script脚本,而是master_ip_online_change_script脚本,但调用参数类似,脚本可以互用。
在mha_manager机器上:
masterha_master_switch --master_state=alive --conf=/etc/masterha/app1.cnf --new_master_host=10.20.20.12 --new_master_port=3306 --orig_master_is_new_slave --running_updates_limit=10000 --interactive=0
十三清除relay log (备库恢复的binlog日志)
对于MHA场景下,对于某些滞后从库的恢复依赖于其他从库的relay log,因此采取禁用自动删除功能以及定期清理的办法。对于清理过多过大的relay log需要注意引起的复制延迟资源开销等。MHA可通过purge_relay_logs脚本及配合cronjob来完成此项任务
在几个MHA node节点上(MHA manager不需要):
添加如下脚本和任务执行计划:
[root@JBWDB03 shell]# cat /usr/local/shell/purge_relay_logs.sh
#!/bin/bash
user=root
passwd='-s4)SVqJ'
port=3306
log_dir='/etc/masterha/log'
work_dir='/etc/masterha/data'
purge='/usr/local/bin/purge_relay_logs'
if [ ! -d $log_dir ]
then
mkdir $log_dir -p
fi
if [ ! -d $work_dir ]
then
mkdir $work_dir -p
fi
$purge --user=$user --password=$passwd --disable_relay_log_purge --port=$port --workdir=$work_dir >> $log_dir/purge_relay_logs.log 2>&1
[root@JBWDB03 shell]# crontab -l
##每天凌晨1点清空relay log
00 01 */1 * * /bin/bash /usr/local/shell/purge_relay_logs.sh
脚本
切换
配置
服务
节点
命令
故障
日志
服务器
邮件
状态
管理
可以通过
手动
数据
文件
环境
检查
主从
任务
数据库的安全要保护哪些东西
数据库安全各自的含义是什么
生产安全数据库录入
数据库的安全性及管理
数据库安全策略包含哪些
海淀数据库安全审计系统
建立农村房屋安全信息数据库
易用的数据库客户端支持安全管理
连接数据库失败ssl安全错误
数据库的锁怎样保障安全
软件开发实践日志
网络安全长城行动新闻
abap数据库显示字段
app开发网络安全怎么保证
游戏软件开发贴吧
网络安全面临的困难
成都腾讯天美软件开发
熟悉网络管理或网络安全
控制系统网络安全防范
网络安全法 csl
服务器w10快还是Linux
迅雷搜索软件开发
以太坊 数据库
场馆运营管理服务器
成都网络安全大会企业
网络安全意识形态工作方案
怀旧服有职业的服务器
深圳网络安全员培训课程
网络技术市值
服务器怎么搭建网站
知乎网络安全教育平台
ip地址数据库怎么建
第一层服务器是什么
曹妃甸区专业性软件开发常见问题
越南网络安全措施
宿迁有没有软件开发公司
java应用服务器需要什么环境
农信社举办网络安全法
软件开发职业目标生涯规划
底层软件开发 方向