11gR2 dataguard 备库文件损坏处理

1. 环境模拟：

   主库：

SQL> select DATABASE_ROLE,open_mode from v$database;DATABASE_ROLE OPEN_MODE---------------- --------------------PRIMARY  READ WRITE

备库：

SQL> select DATABASE_ROLE,open_mode from v$database;DATABASE_ROLE OPEN_MODE---------------- --------------------PHYSICAL STANDBY READ ONLY WITH APPLY

2.主库中断日志传输，备库停止日志应用，然后再主库更新数据。

SQL>  ALTER SYSTEM SET log_archive_dest_state_2='reset' SCOPE=BOTH;System altered.

alert日志可以看到，不再向备库传输日志：

ALTER SYSTEM SET log_archive_dest_state_2='reset' SCOPE=BOTH;Wed Oct 21 10:41:05 2015Thread 1 advanced to log sequence 355 (LGWR switch)  Current log# 1 seq# 355 mem# 0: +DATA/phub/onlinelog/group_1.262.890480943  Current log# 1 seq# 355 mem# 1: +DATA/phub/onlinelog/group_1.263.890480945Wed Oct 21 10:41:05 2015Archived Log entry 707 added for thread 1 sequence 354 ID 0x1fffdaed dest 1:Thread 1 cannot allocate new log, sequence 356Checkpoint not complete  Current log# 1 seq# 355 mem# 0: +DATA/phub/onlinelog/group_1.262.890480943  Current log# 1 seq# 355 mem# 1: +DATA/phub/onlinelog/group_1.263.890480945Thread 1 advanced to log sequence 356 (LGWR switch)  Current log# 2 seq# 356 mem# 0: +DATA/phub/onlinelog/group_2.264.890480945  Current log# 2 seq# 356 mem# 1: +DATA/phub/onlinelog/group_2.265.890480945Wed Oct 21 10:41:06 2015Archived Log entry 708 added for thread 1 sequence 355 ID 0x1fffdaed dest 1:

备库停止日志应用：

SQL> ALTER DATABASE recover managed standby DATABASE cancel;Database altered.SQL> select open_mode from v$database;OPEN_MODE--------------------READ ONLY

测试数据：

主库，

SQL> conn scott/scott;Connected.SQL> select count(*) from test;  COUNT(*)----------     87065

备库：

SQL> select count(*) from test;

COUNT(*)

----------

87065

主库更新test表数据：

SQL> delete from test where rownum<1000;999 rows deleted.SQL> update test set owner='SCOTT' where object_id<10000;8812 rows updated.SQL> insert into test select * from test;86066 rows created.SQL> commit;Commit complete.

SQL> alter system switch logfile;System altered.SQL> /System altered.SQL> /System altered.

删除最新归档日志：

ASMCMD [+data/PHUB/ARCHIVELOG/2015_10_21] > rm -rf thread_1_seq_359.544.893674539ASMCMD [+data/PHUB/ARCHIVELOG/2015_10_21] > rm -rf thread_1_seq_358.543.893674457ASMCMD [+data/PHUB/ARCHIVELOG/2015_10_21] > rm -rf thread_1_seq_357.542.893674453ASMCMD [+data/PHUB/ARCHIVELOG/2015_10_21] >

启用主库日志传输：

SQL> ALTER system SET log_archive_dest_state_2 = 'enable';

System altered.

启用备库日志应用：

SQL> conn / as sysdbaConnected.SQL> ALTER DATABASE recover managed standby DATABASE disconnect FROM SESSION;Database altered.

查看备库alert日志：

[oracle@dg trace]$ tail -f alert_MECBS.log

Serial Media Recovery started

Managed Standby Recovery not using Real Time Apply

Waiting for all non-current ORLs to be archived...

All non-current ORLs have been archived.

Completed: ALTER DATABASE recover managed standby DATABASE disconnect FROM SESSION

Media Recovery Log +DATA/mecbs/archivelog/2015_10_21/thread_1_seq_354.606.893674715

Media Recovery Log +DATA/mecbs/archivelog/2015_10_21/thread_1_seq_355.607.893674715

Media Recovery Log +DATA/mecbs/archivelog/2015_10_21/thread_1_seq_356.605.893674715

Media Recovery Waiting for thread 1 sequence 357

Fetching gap sequence in thread 1, gap sequence 357-359

Wed Oct 21 11:01:34 2015

FAL[client]: Failed to request gap sequence

GAP - thread 1 sequence 357-359

DBID 536511065 branch 890484819

FAL[client]: All defined FAL servers have been attempted.

------------------------------------------------------------

Check that the CONTROL_FILE_RECORD_KEEP_TIME initialization

parameter is defined to a value that's sufficiently large

enough to maintain adequate log switch information to resolve

archivelog gaps.

------------------------------------------------------------

解决办法：

查询主库scn：

1. SQL> SELECT SEQUENCE#,FIRST_CHANGE#,NEXT_CHANGE# FROM v$archived_log WHERE SEQUENCE# > 356 ORDER BY 1; SEQUENCE# FIRST_CHANGE# NEXT_CHANGE#---------- ------------- ------------       357 5501419      5501424       358 5501424      5501430       359 5501430      5501524       360 5501524      5501782       360 5501524      5501782

2)根据scn，进行rman增量备份

RMAN> backup device type disk incremental from scn 5501419 database format '/home/oracle/data_%U.bak';Starting backup at 21-OCT-15using target database control file instead of recovery catalogallocated channel: ORA_DISK_1channel ORA_DISK_1: SID=400 device type=DISKallocated channel: ORA_DISK_2channel ORA_DISK_2: SID=613 device type=DISKchannel ORA_DISK_1: starting full datafile backup setchannel ORA_DISK_1: specifying datafile(s) in backup setinput datafile file number=00006 name=+DATA/phub/datafile/llc01.dbfinput datafile file number=00003 name=+DATA/phub/datafile/undotbs1.260.891340857input datafile file number=00004 name=+DATA/phub/datafile/users.269.891340843channel ORA_DISK_1: starting piece 1 at 21-OCT-15channel ORA_DISK_2: starting full datafile backup setchannel ORA_DISK_2: specifying datafile(s) in backup setinput datafile file number=00002 name=+DATA/phub/datafile/sysaux.272.891340857input datafile file number=00007 name=+DATA/phub/datafile/idx01.dbfinput datafile file number=00001 name=+DATA/phub/datafile/system.271.891340857input datafile file number=00005 name=+DATA/phub/datafile/example.287.891340843channel ORA_DISK_2: starting piece 1 at 21-OCT-15channel ORA_DISK_1: finished piece 1 at 21-OCT-15piece handle=/home/oracle/data_5lqk8pt4_1_1.bak tag=TAG20151021T111028 comment=NONEchannel ORA_DISK_1: backup set complete, elapsed time: 00:01:06channel ORA_DISK_1: starting full datafile backup setchannel ORA_DISK_1: specifying datafile(s) in backup setchannel ORA_DISK_2: finished piece 1 at 21-OCT-15piece handle=/home/oracle/data_5mqk8pt4_1_1.bak tag=TAG20151021T111028 comment=NONEchannel ORA_DISK_2: backup set complete, elapsed time: 00:01:05including current control file in backup setchannel ORA_DISK_1: starting piece 1 at 21-OCT-15channel ORA_DISK_1: finished piece 1 at 21-OCT-15piece handle=/home/oracle/data_5nqk8pv6_1_1.bak tag=TAG20151021T111028 comment=NONEchannel ORA_DISK_1: backup set complete, elapsed time: 00:00:01Finished backup at 21-OCT-15

上传备份文件到备库：

[oracle@cwogg ~]$ scp data_5* 172.16.30.228:/home/oracle/oracle@172.16.30.228's password: data_5lqk8pt4_1_1.bak                                                           100% 1248KB   1.2MB/s   00:00    data_5mqk8pt4_1_1.bak                                                           100% 7128KB   7.0MB/s   00:00    data_5nqk8pv6_1_1.bak                                                           100%   10MB  10.2MB/s   00:00

备库上进行recover：

SQL> ALTER DATABASE recover managed standby DATABASE cancel;Database altered

[oracle@dg ~]$ rman target /

Recovery Manager: Release 11.2.0.4.0 - Production on Wed Oct 21 11:19:03 2015

Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.

connected to target database: PHUB (DBID=536511065)

RMAN> catalog start with '/home/oracle/backup/';

using target database control file instead of recovery catalog

searching for all files that match the pattern /home/oracle/backup/

List of Files Unknown to the Database

=====================================

File Name: /home/oracle/backup/data_5lqk8pt4_1_1.bak

File Name: /home/oracle/backup/data_5mqk8pt4_1_1.bak

File Name: /home/oracle/backup/data_5nqk8pv6_1_1.bak

Do you really want to catalog the above files (enter YES or NO)? yes

cataloging files...

cataloging done

List of Cataloged Files

=======================

File Name: /home/oracle/backup/data_5lqk8pt4_1_1.bak

File Name: /home/oracle/backup/data_5mqk8pt4_1_1.bak

File Name: /home/oracle/backup/data_5nqk8pv6_1_1.bak

恢复备库：

[oracle@dg ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Wed Oct 21 11:22:44 2015

Copyright (c) 1982, 2013, Oracle. All rights reserved.

Connected to:

Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production

With the Partitioning, Automatic Storage Management, OLAP, Data Mining

and Real Application Testing options

SQL> shutdown immediate;

Database closed.

Database dismounted.

ORACLE instance shut down.

SQL> startup mount;

ORACLE instance started.

Total System Global Area 835104768 bytes

Fixed Size 2257840 bytes

Variable Size 541068368 bytes

Database Buffers 289406976 bytes

Redo Buffers 2371584 bytes

Database mounted.

SQL> exit

Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production

With the Partitioning, Automatic Storage Management, OLAP, Data Mining

and Real Application Testing options

[oracle@dg ~]$ rman target /

Recovery Manager: Release 11.2.0.4.0 - Production on Wed Oct 21 11:23:39 2015

Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.

connected to target database: PHUB (DBID=536511065, not open)

RMAN> recover database noredo;

Starting recover at 21-OCT-15

using target database control file instead of recovery catalog

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=36 device type=DISK

allocated channel: ORA_DISK_2

channel ORA_DISK_2: SID=37 device type=DISK

channel ORA_DISK_1: starting incremental datafile backup set restore

channel ORA_DISK_1: specifying datafile(s) to restore from backup set

destination for restore of datafile 00001: +DATA/mecbs/datafile/system.259.891103927

destination for restore of datafile 00002: +DATA/mecbs/datafile/sysaux.260.891104071

destination for restore of datafile 00005: +DATA/mecbs/datafile/example.261.891104187

destination for restore of datafile 00007: +DATA/mecbs/datafile/idx.410.891688925

channel ORA_DISK_1: reading from backup piece /home/oracle/backup/data_5mqk8pt4_1_1.bak

channel ORA_DISK_2: starting incremental datafile backup set restore

channel ORA_DISK_2: specifying datafile(s) to restore from backup set

destination for restore of datafile 00003: +DATA/mecbs/datafile/undotbs1.262.891104243

destination for restore of datafile 00004: +DATA/mecbs/datafile/users.263.891104267

destination for restore of datafile 00006: +DATA/mecbs/datafile/llc.258.891103925

channel ORA_DISK_2: reading from backup piece /home/oracle/backup/data_5lqk8pt4_1_1.bak

channel ORA_DISK_1: piece handle=/home/oracle/backup/data_5mqk8pt4_1_1.bak tag=TAG20151021T111028

channel ORA_DISK_1: restored backup piece 1

channel ORA_DISK_1: restore complete, elapsed time: 00:00:03

channel ORA_DISK_2: piece handle=/home/oracle/backup/data_5lqk8pt4_1_1.bak tag=TAG20151021T111028

channel ORA_DISK_2: restored backup piece 1

channel ORA_DISK_2: restore complete, elapsed time: 00:00:03

Finished recover at 21-OCT-15

查看备库alert日志：

Wed Oct 21 11:23:45 2015

Incremental restore complete of datafile 3 +DATA/mecbs/datafile/undotbs1.262.891104243

checkpoint is 5502788

last deallocation scn is 1354205

Wed Oct 21 11:23:45 2015

Incremental restore complete of datafile 5 +DATA/mecbs/datafile/example.261.891104187

checkpoint is 5502790

last deallocation scn is 1114995

Incremental restore complete of datafile 4 +DATA/mecbs/datafile/users.263.891104267

checkpoint is 5502788

last deallocation scn is 3

Incremental restore complete of datafile 1 +DATA/mecbs/datafile/system.259.891103927

checkpoint is 5502790

last deallocation scn is 1095967

Incremental restore complete of datafile 6 +DATA/mecbs/datafile/llc.258.891103925

checkpoint is 5502788

last deallocation scn is 1099825

Incremental restore complete of datafile 7 +DATA/mecbs/datafile/idx.410.891688925

checkpoint is 5502790

Incremental restore complete of datafile 2 +DATA/mecbs/datafile/sysaux.260.891104071

checkpoint is 5502790

last deallocation scn is 994406

开启日志应用：

SQL> ALTER DATABASE recover managed standby DATABASE disconnect FROM SESSION;

Database altered.

主库切换日志：

SQL> ALTER system switch logfile;System altered.SQL> /System altered.SQL> SELECT MAX(al.SEQUENCE#) "Last Seq Recieved", MAX(lh.SEQUENCE#) "Last Seq Applied" FROM v$archived_log al, v$log_history lh;  Last Seq Recieved Last Seq Applied----------------- ----------------      363       363

备库：

SQL> SELECT MAX(al.SEQUENCE#) "Last Seq Recieved", MAX(lh.SEQUENCE#) "Last Seq Applied" FROM v$archived_log al, v$log_history lh;

Last Seq Recieved Last Seq Applied

----------------- ----------------

363 356

此时备库仍然显示有gap：

Completed: ALTER DATABASE recover managed standby DATABASE disconnect FROM SESSIONMedia Recovery Waiting for thread 1 sequence 357Fetching gap sequence in thread 1, gap sequence 357-359Wed Oct 21 11:27:29 2015RFS[1]: Selected log 4 for thread 1 sequence 363 dbid 536511065 branch 890484819Wed Oct 21 11:27:29 2015Archived Log entry 333 added for thread 1 sequence 362 ID 0x1fffdaed dest 1:Wed Oct 21 11:27:49 2015RFS[1]: Selected log 5 for thread 1 sequence 364 dbid 536511065 branch 890484819Wed Oct 21 11:27:49 2015Archived Log entry 334 added for thread 1 sequence 363 ID 0x1fffdaed dest 1:Wed Oct 21 11:27:52 2015FAL[client]: Failed to request gap sequence GAP - thread 1 sequence 357-359 DBID 536511065 branch 890484819FAL[client]: All defined FAL servers have been attempted.------------------------------------------------------------Check that the CONTROL_FILE_RECORD_KEEP_TIME initializationparameter is defined to a value that's sufficiently largeenough to maintain adequate log switch information to resolvearchivelog gaps.------------------------------------------------------------

停止备库日志应用，在重启，

两边数据以及同步了：

SQL> archive log list;

Database log mode Archive Mode

Automatic archival Enabled

Archive destination +DATA

Oldest online log sequence 365

Next log sequence to archive 0

Current log sequence 367

SQL> archive log list;

Database log mode Archive Mode

Automatic archival Enabled

Archive destination +DATA

Oldest online log sequence 365

Next log sequence to archive 367

Current log sequence 367

但是备库alertlog仍然报错：找不到那3个归档文件：

FAL[client]: Failed to request gap sequence

GAP - thread 1 sequence 357-359

DBID 536511065 branch 890484819

FAL[client]: All defined FAL servers have been attempted.

重建备库的控制文件：

SQL> ALTER DATABASE CREATE standby controlfile AS '/tmp/standby.ctl';   Database altered.[oracle@cwogg tmp]$ scp standby.ctl 172.16.30.228:/home/oracle/oracle@172.16.30.228's password: standby.ctl                                                                     100%   10MB  10.1MB/s   00:00

[oracle@dg ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Wed Oct 21 11:40:43 2015

Copyright (c) 1982, 2013, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup nomount;

ORACLE instance started.

Total System Global Area 835104768 bytes

Fixed Size 2257840 bytes

Variable Size 541068368 bytes

Database Buffers 289406976 bytes

Redo Buffers 2371584 bytes

RMAN> restore controlfile from '/home/oracle/standby.ctl';

Starting restore at 21-OCT-15

using target database control file instead of recovery catalog

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=24 device type=DISK

channel ORA_DISK_1: copied control file copy

output file name=+DATA/mecbs/controlfile/control01.ctl

output file name=+DATA/mecbs/controlfile/control02.ctl

Finished restore at 21-OCT-15

RMAN> startup mount;

database is already started

database mounted

released channel: ORA_DISK_1

待续，，，