Oracle 11G RAC 增加节点

集群中增加节点大致分为 4 个步骤：前期准备阶段、新节点加入集群（安装 GI 软件）、新节点安装 DB 软件、给新节点分配实例。

一 . 前期准备阶段

IP 配置、系统内核配置、安装 RPM 包、 SHELL 限制、分配用户和组、配置用户环境变量、目录创建及授权、配置互信等等一些前期环境准。

[root@vastdata2 ~]#mkdir -p /u01/app/gridmkdir -p /u01/app/11.2.0/gridmkdir -p /u01/app/oraclechown -R oracle:oinstall /u01chown -R grid:oinstall /u01/app/gridchown -R grid:oinstall /u01/app/11.2.0chmod -R 775 /u01

二. 新节点加入集群（安装 GI 软件）

(1) 对新节点做安装前的检查

[grid@vastdata4 ~]$ cluvfy stage -post nodedel -n vastdata3 -verbose Performing post-checks for node removal  Checking CRS integrity... Clusterware version consistency passedThe Oracle Clusterware is healthy on node "vastdata4" CRS integrity check passedResult: Node removal check passed Post-check for node removal was successful. [grid@vastdata4 ~]$ [grid@vastdata4 ~]$ [grid@vastdata4 ~]$ [grid@vastdata4 ~]$ [grid@vastdata4 ~]$ [grid@vastdata4 ~]$ [grid@vastdata4 ~]$ [grid@vastdata4 ~]$ cluvfy stage -pre nodeadd -n vastdata3 -verbose Performing pre-checks for node addition  Checking node reachability... Check: Node reachability from node "vastdata4"  Destination Node                      Reachable?                ------------------------------------  ------------------------  vastdata3                             yes                     Result: Node reachability check passed from node "vastdata4" Checking user equivalence... Check: User equivalence for user "grid"  Node Name                             Status                    ------------------------------------  ------------------------  vastdata3                             passed                  Result: User equivalence check passed for user "grid" Checking CRS integrity... Clusterware version consistency passedThe Oracle Clusterware is healthy on node "vastdata4" CRS integrity check passed Checking shared resources... Checking CRS home location..."/u01/app/11.2.0/grid" is sharedResult: Shared resources check for node addition passed Checking node connectivity... Checking hosts config file...  Node Name                             Status                    ------------------------------------  ------------------------  vastdata3                             passed                    vastdata4                             passed                   Verification of the hosts config file successful Interface information for node "vastdata3" Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU    ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0   192.168.0.11    192.168.0.0     0.0.0.0         UNKNOWN         08:00:27:D3:55:CB 1500   eth2   10.0.0.2        10.0.0.0        0.0.0.0         UNKNOWN         08:00:27:B4:C3:C7 1500   eth3   10.0.0.3        10.0.0.0        0.0.0.0         UNKNOWN         08:00:27:02:BD:9A 1500   Interface information for node "vastdata4" Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU    ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0   192.168.0.12    192.168.0.0     0.0.0.0         UNKNOWN         08:00:27:EF:F6:15 1500   eth0   192.168.0.33    192.168.0.0     0.0.0.0         UNKNOWN         08:00:27:EF:F6:15 1500   eth0   192.168.0.35    192.168.0.0     0.0.0.0         UNKNOWN         08:00:27:EF:F6:15 1500   eth0   192.168.0.23    192.168.0.0     0.0.0.0         UNKNOWN         08:00:27:EF:F6:15 1500   eth0   192.168.0.34    192.168.0.0     0.0.0.0         UNKNOWN         08:00:27:EF:F6:15 1500   eth2   10.0.0.4        10.0.0.0        0.0.0.0         UNKNOWN         08:00:27:82:3A:57 1500   eth2   169.254.17.92   169.254.0.0     0.0.0.0         UNKNOWN         08:00:27:82:3A:57 1500   eth3   10.0.0.5        10.0.0.0        0.0.0.0         UNKNOWN         08:00:27:4C:5D:39 1500   eth3   169.254.236.167 169.254.128.0   0.0.0.0         UNKNOWN         08:00:27:4C:5D:39 1500   Check: Node connectivity for interface "eth0"  Source                          Destination                     Connected?        ------------------------------  ------------------------------  ----------------  vastdata3[192.168.0.11]         vastdata4[192.168.0.12]         yes               vastdata3[192.168.0.11]         vastdata4[192.168.0.33]         yes               vastdata3[192.168.0.11]         vastdata4[192.168.0.35]         yes               vastdata3[192.168.0.11]         vastdata4[192.168.0.23]         yes               vastdata3[192.168.0.11]         vastdata4[192.168.0.34]         yes               vastdata4[192.168.0.12]         vastdata4[192.168.0.33]         yes               vastdata4[192.168.0.12]         vastdata4[192.168.0.35]         yes               vastdata4[192.168.0.12]         vastdata4[192.168.0.23]         yes               vastdata4[192.168.0.12]         vastdata4[192.168.0.34]         yes               vastdata4[192.168.0.33]         vastdata4[192.168.0.35]         yes               vastdata4[192.168.0.33]         vastdata4[192.168.0.23]         yes               vastdata4[192.168.0.33]         vastdata4[192.168.0.34]         yes               vastdata4[192.168.0.35]         vastdata4[192.168.0.23]         yes               vastdata4[192.168.0.35]         vastdata4[192.168.0.34]         yes               vastdata4[192.168.0.23]         vastdata4[192.168.0.34]         yes             Result: Node connectivity passed for interface "eth0" Check: TCP connectivity of subnet "192.168.0.0"  Source                          Destination                     Connected?        ------------------------------  ------------------------------  ----------------  vastdata4:192.168.0.12          vastdata3:192.168.0.11          passed            vastdata4:192.168.0.12          vastdata4:192.168.0.33          passed            vastdata4:192.168.0.12          vastdata4:192.168.0.35          passed            vastdata4:192.168.0.12          vastdata4:192.168.0.23          passed            vastdata4:192.168.0.12          vastdata4:192.168.0.34          passed          Result: TCP connectivity check passed for subnet "192.168.0.0" Check: Node connectivity for interface "eth2"  Source                          Destination                     Connected?        ------------------------------  ------------------------------  ----------------  vastdata3[10.0.0.2]             vastdata3[10.0.0.3]             yes               vastdata3[10.0.0.2]             vastdata4[10.0.0.4]             yes               vastdata3[10.0.0.2]             vastdata4[10.0.0.5]             yes               vastdata3[10.0.0.3]             vastdata4[10.0.0.4]             yes               vastdata3[10.0.0.3]             vastdata4[10.0.0.5]             yes               vastdata4[10.0.0.4]             vastdata4[10.0.0.5]             yes             Result: Node connectivity passed for interface "eth2" Check: TCP connectivity of subnet "10.0.0.0"  Source                          Destination                     Connected?        ------------------------------  ------------------------------  ----------------  vastdata4:10.0.0.4              vastdata3:10.0.0.2              passed            vastdata4:10.0.0.4              vastdata3:10.0.0.3              passed            vastdata4:10.0.0.4              vastdata4:10.0.0.5              passed          Result: TCP connectivity check passed for subnet "10.0.0.0" Check: Node connectivity for interface "eth3"Checking subnet mask consistency...Subnet mask consistency check passed for subnet "192.168.0.0".Subnet mask consistency check passed for subnet "10.0.0.0".Subnet mask consistency check passed. Result: Node connectivity check passed Checking multicast communication... Checking subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0"...Check of subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0" passed. Checking subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0"...Check of subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0" passed. Check of multicast communication passed. Check: Total memory   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     2.8198GB (2956724.0KB)    1.5GB (1572864.0KB)       passed      vastdata3     2.8198GB (2956724.0KB)    1.5GB (1572864.0KB)       passed    Result: Total memory check passed Check: Available memory   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     1.3675GB (1433912.0KB)    50MB (51200.0KB)          passed      vastdata3     2.6447GB (2773172.0KB)    50MB (51200.0KB)          passed    Result: Available memory check passed Check: Swap space   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     6GB (6291448.0KB)         2.8198GB (2956724.0KB)    passed      vastdata3     6GB (6291448.0KB)         2.8198GB (2956724.0KB)    passed    Result: Swap space check passed Check: Free disk space for "vastdata4:/u01/app/11.2.0/grid,vastdata4:/tmp"   Path              Node Name     Mount point   Available     Required      Status        ----------------  ------------  ------------  ------------  ------------  ------------  /u01/app/11.2.0/grid  vastdata4     /             5.2812GB      7.5GB         failed        /tmp              vastdata4     /             5.2812GB      7.5GB         failed      Result: Free disk space check failed for "vastdata4:/u01/app/11.2.0/grid,vastdata4:/tmp" Check: Free disk space for "vastdata3:/u01/app/11.2.0/grid,vastdata3:/tmp"   Path              Node Name     Mount point   Available     Required      Status        ----------------  ------------  ------------  ------------  ------------  ------------  /u01/app/11.2.0/grid  vastdata3     /             15.7051GB     7.5GB         passed        /tmp              vastdata3     /             15.7051GB     7.5GB         passed      Result: Free disk space check passed for "vastdata3:/u01/app/11.2.0/grid,vastdata3:/tmp" Check: User existence for "grid"   Node Name     Status                    Comment                   ------------  ------------------------  ------------------------  vastdata4     passed                    exists(1100)              vastdata3     passed                    exists(1100)             Checking for multiple users with UID value 1100Result: Check for multiple users with UID value 1100 passed Result: User existence check passed for "grid" Check: Run level   Node Name     run level                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     5                         3,5                       passed      vastdata3     5                         3,5                       passed    Result: Run level check passed Check: Hard limits for "maximum open file descriptors"   Node Name         Type          Available     Required      Status            ----------------  ------------  ------------  ------------  ----------------  vastdata4         hard          65536         65536         passed            vastdata3         hard          65536         65536         passed          Result: Hard limits check passed for "maximum open file descriptors" Check: Soft limits for "maximum open file descriptors"   Node Name         Type          Available     Required      Status            ----------------  ------------  ------------  ------------  ----------------  vastdata4         soft          1024          1024          passed            vastdata3         soft          1024          1024          passed          Result: Soft limits check passed for "maximum open file descriptors" Check: Hard limits for "maximum user processes"   Node Name         Type          Available     Required      Status            ----------------  ------------  ------------  ------------  ----------------  vastdata4         hard          16384         16384         passed            vastdata3         hard          16384         16384         passed          Result: Hard limits check passed for "maximum user processes" Check: Soft limits for "maximum user processes"   Node Name         Type          Available     Required      Status            ----------------  ------------  ------------  ------------  ----------------  vastdata4         soft          2047          2047          passed            vastdata3         soft          2047          2047          passed          Result: Soft limits check passed for "maximum user processes" Check: System architecture   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     x86_64                    x86_64                    passed      vastdata3     x86_64                    x86_64                    passed    Result: System architecture check passed Check: Kernel version   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     2.6.32-431.el6.x86_64     2.6.32                    passed      vastdata3     2.6.32-431.el6.x86_64     2.6.32                    passed    Result: Kernel version check passed Check: Kernel parameter for "semmsl"   Node Name         Current       Configured    Required      Status        Comment       ----------------  ------------  ------------  ------------  ------------  ------------  vastdata4         250           250           250           passed            vastdata3         250           250           250           passed          Result: Kernel parameter check passed for "semmsl" Check: Kernel parameter for "semmns"   Node Name         Current       Configured    Required      Status        Comment       ----------------  ------------  ------------  ------------  ------------  ------------  vastdata4         32000         32000         32000         passed            vastdata3         32000         32000         32000         passed          Result: Kernel parameter check passed for "semmns" Check: Kernel parameter for "semopm"   Node Name         Current       Configured    Required      Status        Comment       ----------------  ------------  ------------  ------------  ------------  ------------  vastdata4         100           100           100           passed            vastdata3         100           100           100           passed          Result: Kernel parameter check passed for "semopm" Check: Kernel parameter for "semmni"   Node Name         Current       Configured    Required      Status        Comment       ----------------  ------------  ------------  ------------  ------------  ------------  vastdata4         128           128           128           passed            vastdata3         128           128           128           passed          Result: Kernel parameter check passed for "semmni" Check: Kernel parameter for "shmmax"   Node Name         Current       Configured    Required      Status        Comment       ----------------  ------------  ------------  ------------  ------------  ------------  vastdata4         68719476736   68719476736   1513842688    passed            vastdata3         68719476736   68719476736   1513842688    passed          Result: Kernel parameter check passed for "shmmax" Check: Kernel parameter for "shmmni"   Node Name         Current       Configured    Required      Status        Comment       ----------------  ------------  ------------  ------------  ------------  ------------  vastdata4         4096          4096          4096          passed            vastdata3         4096          4096          4096          passed          Result: Kernel parameter check passed for "shmmni" Check: Kernel parameter for "shmall"   Node Name         Current       Configured    Required      Status        Comment       ----------------  ------------  ------------  ------------  ------------  ------------  vastdata4         4294967296    4294967296    2097152       passed            vastdata3         4294967296    4294967296    2097152       passed          Result: Kernel parameter check passed for "shmall" Check: Kernel parameter for "file-max"   Node Name         Current       Configured    Required      Status        Comment       ----------------  ------------  ------------  ------------  ------------  ------------  vastdata4         6815744       6815744       6815744       passed            vastdata3         6815744       6815744       6815744       passed          Result: Kernel parameter check passed for "file-max" Check: Kernel parameter for "ip_local_port_range"   Node Name         Current       Configured    Required      Status        Comment       ----------------  ------------  ------------  ------------  ------------  ------------  vastdata4         between 9000.0 & 65500.0  between 9000.0 & 65500.0  between 9000.0 & 65500.0  passed            vastdata3         between 9000.0 & 65500.0  between 9000.0 & 65500.0  between 9000.0 & 65500.0  passed          Result: Kernel parameter check passed for "ip_local_port_range" Check: Kernel parameter for "rmem_default"   Node Name         Current       Configured    Required      Status        Comment       ----------------  ------------  ------------  ------------  ------------  ------------  vastdata4         262144        262144        262144        passed            vastdata3         262144        262144        262144        passed          Result: Kernel parameter check passed for "rmem_default" Check: Kernel parameter for "rmem_max"   Node Name         Current       Configured    Required      Status        Comment       ----------------  ------------  ------------  ------------  ------------  ------------  vastdata4         4194304       4194304       4194304       passed            vastdata3         4194304       4194304       4194304       passed          Result: Kernel parameter check passed for "rmem_max" Check: Kernel parameter for "wmem_default"   Node Name         Current       Configured    Required      Status        Comment       ----------------  ------------  ------------  ------------  ------------  ------------  vastdata4         262144        262144        262144        passed            vastdata3         262144        262144        262144        passed          Result: Kernel parameter check passed for "wmem_default" Check: Kernel parameter for "wmem_max"   Node Name         Current       Configured    Required      Status        Comment       ----------------  ------------  ------------  ------------  ------------  ------------  vastdata4         1048576       1048576       1048576       passed            vastdata3         1048576       1048576       1048576       passed          Result: Kernel parameter check passed for "wmem_max" Check: Kernel parameter for "aio-max-nr"   Node Name         Current       Configured    Required      Status        Comment       ----------------  ------------  ------------  ------------  ------------  ------------  vastdata4         1048576       1048576       1048576       passed            vastdata3         1048576       1048576       1048576       passed          Result: Kernel parameter check passed for "aio-max-nr" Check: Package existence for "binutils"   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     binutils-2.20.51.0.2-5.36.el6  binutils-2.20.51.0.2      passed      vastdata3     binutils-2.20.51.0.2-5.36.el6  binutils-2.20.51.0.2      passed    Result: Package existence check passed for "binutils" Check: Package existence for "compat-libcap1"   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     compat-libcap1-1.10-1     compat-libcap1-1.10       passed      vastdata3     compat-libcap1-1.10-1     compat-libcap1-1.10       passed    Result: Package existence check passed for "compat-libcap1" Check: Package existence for "compat-libstdc++-33(x86_64)"   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     compat-libstdc++-33(x86_64)-3.2.3-69.el6  compat-libstdc++-33(x86_64)-3.2.3  passed      vastdata3     compat-libstdc++-33(x86_64)-3.2.3-69.el6  compat-libstdc++-33(x86_64)-3.2.3  passed    Result: Package existence check passed for "compat-libstdc++-33(x86_64)" Check: Package existence for "libgcc(x86_64)"   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     libgcc(x86_64)-4.4.7-4.el6  libgcc(x86_64)-4.4.4      passed      vastdata3     libgcc(x86_64)-4.4.7-4.el6  libgcc(x86_64)-4.4.4      passed    Result: Package existence check passed for "libgcc(x86_64)" Check: Package existence for "libstdc++(x86_64)"   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     libstdc++(x86_64)-4.4.7-4.el6  libstdc++(x86_64)-4.4.4   passed      vastdata3     libstdc++(x86_64)-4.4.7-4.el6  libstdc++(x86_64)-4.4.4   passed    Result: Package existence check passed for "libstdc++(x86_64)" Check: Package existence for "libstdc++-devel(x86_64)"   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     libstdc++-devel(x86_64)-4.4.7-4.el6  libstdc++-devel(x86_64)-4.4.4  passed      vastdata3     libstdc++-devel(x86_64)-4.4.7-4.el6  libstdc++-devel(x86_64)-4.4.4  passed    Result: Package existence check passed for "libstdc++-devel(x86_64)" Check: Package existence for "sysstat"   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     sysstat-9.0.4-22.el6      sysstat-9.0.4             passed      vastdata3     sysstat-9.0.4-22.el6      sysstat-9.0.4             passed    Result: Package existence check passed for "sysstat" Check: Package existence for "gcc"   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     gcc-4.4.7-4.el6           gcc-4.4.4                 passed      vastdata3     gcc-4.4.7-4.el6           gcc-4.4.4                 passed    Result: Package existence check passed for "gcc" Check: Package existence for "gcc-c++"   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     gcc-c++-4.4.7-4.el6       gcc-c++-4.4.4             passed      vastdata3     gcc-c++-4.4.7-4.el6       gcc-c++-4.4.4             passed    Result: Package existence check passed for "gcc-c++" Check: Package existence for "ksh"   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     ksh-20120801-10.el6       ksh-20100621              passed      vastdata3     ksh-20120801-10.el6       ksh-20100621              passed    Result: Package existence check passed for "ksh" Check: Package existence for "make"   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     make-3.81-20.el6          make-3.81                 passed      vastdata3     make-3.81-20.el6          make-3.81                 passed    Result: Package existence check passed for "make" Check: Package existence for "glibc(x86_64)"   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     glibc(x86_64)-2.12-1.132.el6  glibc(x86_64)-2.12        passed      vastdata3     glibc(x86_64)-2.12-1.132.el6  glibc(x86_64)-2.12        passed    Result: Package existence check passed for "glibc(x86_64)" Check: Package existence for "glibc-devel(x86_64)"   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     glibc-devel(x86_64)-2.12-1.132.el6  glibc-devel(x86_64)-2.12  passed      vastdata3     glibc-devel(x86_64)-2.12-1.132.el6  glibc-devel(x86_64)-2.12  passed    Result: Package existence check passed for "glibc-devel(x86_64)" Check: Package existence for "libaio(x86_64)"   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     libaio(x86_64)-0.3.107-10.el6  libaio(x86_64)-0.3.107    passed      vastdata3     libaio(x86_64)-0.3.107-10.el6  libaio(x86_64)-0.3.107    passed    Result: Package existence check passed for "libaio(x86_64)" Check: Package existence for "libaio-devel(x86_64)"   Node Name     Available                 Required                  Status      ------------  ------------------------  ------------------------  ----------  vastdata4     libaio-devel(x86_64)-0.3.107-10.el6  libaio-devel(x86_64)-0.3.107  passed      vastdata3     libaio-devel(x86_64)-0.3.107-10.el6  libaio-devel(x86_64)-0.3.107  passed    Result: Package existence check passed for "libaio-devel(x86_64)" Checking for multiple users with UID value 0Result: Check for multiple users with UID value 0 passed  Check: Current group ID Result: Current group ID check passed Starting check for consistency of primary group of root user  Node Name                             Status                    ------------------------------------  ------------------------  vastdata4                             passed                    vastdata3                             passed                   Check for consistency of root user's primary group passed Checking OCR integrity... OCR integrity check passed Checking Oracle Cluster Voting Disk configuration... Oracle Cluster Voting Disk configuration check passedCheck: Time zone consistency Result: Time zone consistency check passed Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started...Network Time Protocol(NTP) configuration file not found on any of the nodes. Oracle Cluster Time Synchronization Service(CTSS) can be used instead of NTP for time synchronization on the cluster nodesNo NTP Daemons or Services were found to be running Result: Clock synchronization check using Network Time Protocol(NTP) passed Checking to make sure user "grid" is not in "root" group  Node Name     Status                    Comment                   ------------  ------------------------  ------------------------  vastdata4     passed                    does not exist            vastdata3     passed                    does not exist          Result: User "grid" is not part of "root" group. Check passedChecking consistency of file "/etc/resolv.conf" across nodes Checking the file "/etc/resolv.conf" to make sure only one of domain and search entries is definedFile "/etc/resolv.conf" does not have both domain and search entries definedChecking if domain entry in file "/etc/resolv.conf" is consistent across the nodes...domain entry in file "/etc/resolv.conf" is consistent across nodesChecking if search entry in file "/etc/resolv.conf" is consistent across the nodes...search entry in file "/etc/resolv.conf" is consistent across nodesChecking file "/etc/resolv.conf" to make sure that only one search entry is definedAll nodes have one search entry defined in file "/etc/resolv.conf"Checking all nodes to make sure that search entry is "us.oracle.com" as found on node "vastdata4"All nodes of the cluster have same value for 'search'Checking DNS response time for an unreachable node  Node Name                             Status                    ------------------------------------  ------------------------  vastdata4                             passed                    vastdata3                             passed                  The DNS response time for an unreachable node is within acceptable limit on all nodes File "/etc/resolv.conf" is consistent across nodes Checking integrity of name service switch configuration file "/etc/nsswitch.conf" ...Checking if "hosts" entry in file "/etc/nsswitch.conf" is consistent across nodes...Checking file "/etc/nsswitch.conf" to make sure that only one "hosts" entry is definedMore than one "hosts" entry does not exist in any "/etc/nsswitch.conf" fileCheck for integrity of name service switch configuration file "/etc/nsswitch.conf" passed Pre-check for node addition was successful.

(2) 检查集群内节点状态

[grid@vastdata4 ~]$ olsnodes -tvastdata4 Unpinned

(3) 给新的节点安装 GI 软件

[grid@vastdata4 ~]$ /u01/app/11.2.0/grid/oui/bin/addNode.sh -silent  "CLUSTER_NEW_NODES={vastdata3}" "CLUSTER_NEW_VIRTUAL_HOSTNAMES={vastdata3-vip}" Performing pre-checks for node addition  Checking node reachability...Node reachability check passed from node "vastdata4" Checking user equivalence...User equivalence check passed for user "grid" Checking CRS integrity... Clusterware version consistency passed CRS integrity check passed Checking shared resources... Checking CRS home location..."/u01/app/11.2.0/grid" is sharedShared resources check for node addition passed Checking node connectivity... Checking hosts config file... Verification of the hosts config file successful Check: Node connectivity for interface "eth0"Node connectivity passed for interface "eth0"TCP connectivity check passed for subnet "192.168.0.0"Check: Node connectivity for interface "eth2"Node connectivity passed for interface "eth2"TCP connectivity check passed for subnet "10.0.0.0" Check: Node connectivity for interface "eth3"Checking subnet mask consistency...Subnet mask consistency check passed for subnet "192.168.0.0".Subnet mask consistency check passed for subnet "10.0.0.0".Subnet mask consistency check passed. Node connectivity check passed Checking multicast communication... Checking subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0"...Check of subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0" passed. Checking subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0"...Check of subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0" passed. Check of multicast communication passed.Total memory check passedAvailable memory check passedSwap space check passedFree disk space check failed for "vastdata4:/u01/app/11.2.0/grid,vastdata4:/tmp"Check failed on nodes:         vastdata4Free disk space check passed for "vastdata3:/u01/app/11.2.0/grid,vastdata3:/tmp"Check for multiple users with UID value 1100 passed User existence check passed for "grid"Run level check passedHard limits check passed for "maximum open file descriptors"Soft limits check passed for "maximum open file descriptors"Hard limits check passed for "maximum user processes"Soft limits check passed for "maximum user processes"System architecture check passedKernel version check passedKernel parameter check passed for "semmsl"Kernel parameter check passed for "semmns"Kernel parameter check passed for "semopm"Kernel parameter check passed for "semmni"Kernel parameter check passed for "shmmax"Kernel parameter check passed for "shmmni"Kernel parameter check passed for "shmall"Kernel parameter check passed for "file-max"Kernel parameter check passed for "ip_local_port_range"Kernel parameter check passed for "rmem_default"Kernel parameter check passed for "rmem_max"Kernel parameter check passed for "wmem_default"Kernel parameter check passed for "wmem_max"Kernel parameter check passed for "aio-max-nr"Package existence check passed for "binutils"Package existence check passed for "compat-libcap1"Package existence check passed for "compat-libstdc++-33(x86_64)"Package existence check passed for "libgcc(x86_64)"Package existence check passed for "libstdc++(x86_64)"Package existence check passed for "libstdc++-devel(x86_64)"Package existence check passed for "sysstat"Package existence check passed for "gcc"Package existence check passed for "gcc-c++"Package existence check passed for "ksh"Package existence check passed for "make"Package existence check passed for "glibc(x86_64)"Package existence check passed for "glibc-devel(x86_64)"Package existence check passed for "libaio(x86_64)"Package existence check passed for "libaio-devel(x86_64)"Check for multiple users with UID value 0 passed Current group ID check passed Starting check for consistency of primary group of root user Check for consistency of root user's primary group passed Checking OCR integrity... OCR integrity check passed Checking Oracle Cluster Voting Disk configuration... Oracle Cluster Voting Disk configuration check passedTime zone consistency check passed Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started...No NTP Daemons or Services were found to be running Clock synchronization check using Network Time Protocol(NTP) passed User "grid" is not part of "root" group. Check passedChecking consistency of file "/etc/resolv.conf" across nodes File "/etc/resolv.conf" does not have both domain and search entries defineddomain entry in file "/etc/resolv.conf" is consistent across nodessearch entry in file "/etc/resolv.conf" is consistent across nodesAll nodes have one search entry defined in file "/etc/resolv.conf"The DNS response time for an unreachable node is within acceptable limit on all nodes File "/etc/resolv.conf" is consistent across nodes Checking integrity of name service switch configuration file "/etc/nsswitch.conf" ...Check for integrity of name service switch configuration file "/etc/nsswitch.conf" passed Checking VIP configuration.Checking VIP Subnet configuration.Check for VIP Subnet configuration passed.Checking VIP reachabilityCheck for VIP reachability passed. Pre-check for node addition was successful. Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB.   Actual 6143 MB    PassedOracle Universal Installer, Version 11.2.0.4.0 ProductionCopyright (C) 1999, 2013, Oracle. All rights reserved. Performing tests to see whether nodes vastdata3 are available............................................................... 100% Done..-----------------------------------------------------------------------------Cluster Node Addition SummaryGlobal Settings   Source: /u01/app/11.2.0/grid   New NodesSpace Requirements   New Nodes      vastdata3         /: Required 4.61GB : Available 14.62GBInstalled Products   Product Names      Oracle Grid Infrastructure 11g 11.2.0.4.0       Java Development Kit 1.5.0.51.10       Installer SDK Component 11.2.0.4.0       Oracle One-Off Patch Installer 11.2.0.3.4       Oracle Universal Installer 11.2.0.4.0       Oracle RAC Required Support Files-HAS 11.2.0.4.0       Oracle USM Deconfiguration 11.2.0.4.0       Oracle Configuration Manager Deconfiguration 10.3.1.0.0       Enterprise Manager Common Core Files 10.2.0.4.5       Oracle DBCA Deconfiguration 11.2.0.4.0       Oracle RAC Deconfiguration 11.2.0.4.0       Oracle Quality of Service Management (Server) 11.2.0.4.0       Installation Plugin Files 11.2.0.4.0       Universal Storage Manager Files 11.2.0.4.0       Oracle Text Required Support Files 11.2.0.4.0       Automatic Storage Management Assistant 11.2.0.4.0       Oracle Database 11g Multimedia Files 11.2.0.4.0       Oracle Multimedia Java Advanced Imaging 11.2.0.4.0       Oracle Globalization Support 11.2.0.4.0       Oracle Multimedia Locator RDBMS Files 11.2.0.4.0       Oracle Core Required Support Files 11.2.0.4.0       Bali Share 1.1.18.0.0       Oracle Database Deconfiguration 11.2.0.4.0       Oracle Quality of Service Management (Client) 11.2.0.4.0       Expat libraries 2.0.1.0.1       Oracle Containers for Java 11.2.0.4.0       Perl Modules 5.10.0.0.1       Secure Socket Layer 11.2.0.4.0       Oracle JDBC/OCI Instant Client 11.2.0.4.0       Oracle Multimedia Client Option 11.2.0.4.0       LDAP Required Support Files 11.2.0.4.0       Character Set Migration Utility 11.2.0.4.0       Perl Interpreter 5.10.0.0.2       PL/SQL Embedded Gateway 11.2.0.4.0       OLAP SQL Scripts 11.2.0.4.0       Database SQL Scripts 11.2.0.4.0       Oracle Extended Windowing Toolkit 3.4.47.0.0       SSL Required Support Files for InstantClient 11.2.0.4.0       SQL*Plus Files for Instant Client 11.2.0.4.0       Oracle Net Required Support Files 11.2.0.4.0       Oracle Database User Interface 2.2.13.0.0       RDBMS Required Support Files for Instant Client 11.2.0.4.0       RDBMS Required Support Files Runtime 11.2.0.4.0       XML Parser for Java 11.2.0.4.0       Oracle Security Developer Tools 11.2.0.4.0       Oracle Wallet Manager 11.2.0.4.0       Enterprise Manager plugin Common Files 11.2.0.4.0       Platform Required Support Files 11.2.0.4.0       Oracle JFC Extended Windowing Toolkit 4.2.36.0.0       RDBMS Required Support Files 11.2.0.4.0       Oracle Ice Browser 5.2.3.6.0       Oracle Help For Java 4.2.9.0.0       Enterprise Manager Common Files 10.2.0.4.5       Deinstallation Tool 11.2.0.4.0       Oracle Java Client 11.2.0.4.0       Cluster Verification Utility Files 11.2.0.4.0       Oracle Notification Service (eONS) 11.2.0.4.0       Oracle LDAP administration 11.2.0.4.0       Cluster Verification Utility Common Files 11.2.0.4.0       Oracle Clusterware RDBMS Files 11.2.0.4.0       Oracle Locale Builder 11.2.0.4.0       Oracle Globalization Support 11.2.0.4.0       Buildtools Common Files 11.2.0.4.0       HAS Common Files 11.2.0.4.0       SQL*Plus Required Support Files 11.2.0.4.0       XDK Required Support Files 11.2.0.4.0       Agent Required Support Files 10.2.0.4.5       Parser Generator Required Support Files 11.2.0.4.0       Precompiler Required Support Files 11.2.0.4.0       Installation Common Files 11.2.0.4.0       Required Support Files 11.2.0.4.0       Oracle JDBC/THIN Interfaces 11.2.0.4.0       Oracle Multimedia Locator 11.2.0.4.0       Oracle Multimedia 11.2.0.4.0       Assistant Common Files 11.2.0.4.0       Oracle Net 11.2.0.4.0       PL/SQL 11.2.0.4.0       HAS Files for DB 11.2.0.4.0       Oracle Recovery Manager 11.2.0.4.0       Oracle Database Utilities 11.2.0.4.0       Oracle Notification Service 11.2.0.3.0       SQL*Plus 11.2.0.4.0       Oracle Netca Client 11.2.0.4.0       Oracle Advanced Security 11.2.0.4.0       Oracle JVM 11.2.0.4.0       Oracle Internet Directory Client 11.2.0.4.0       Oracle Net Listener 11.2.0.4.0       Cluster Ready Services Files 11.2.0.4.0       Oracle Database 11g 11.2.0.4.0 ----------------------------------------------------------------------------- Instantiating scripts for add node (Monday, February 25, 2019 1:00:48 AM EST).                                                                 1% Done.Instantiation of add node scripts complete Copying to remote nodes (Monday, February 25, 2019 1:00:54 AM EST)...............................................................................................                                 96% Done.Home copied to new nodes Saving inventory on nodes (Monday, February 25, 2019 1:04:42 AM EST).                                                               100% Done.Save inventory completeWARNING:A new inventory has been created on one or more nodes in this session. However, it has not yet been registered as the central inventory of this system. To register the new inventory please run the script at '/u01/app/oraInventory/orainstRoot.sh' with root privileges on nodes 'vastdata3'.If you do not register the inventory, you may not be able to update or patch the products you installed.The following configuration scripts need to be executed as the "root" user in each new cluster node. Each script in the list below is followed by a list of nodes./u01/app/oraInventory/orainstRoot.sh #On nodes vastdata3/u01/app/11.2.0/grid/root.sh #On nodes vastdata3To execute the configuration scripts:    1. Open a terminal window    2. Log in as "root"    3. Run the scripts in each cluster node    The Cluster Node Addition of /u01/app/11.2.0/grid was successful.Please check '/tmp/silentInstall.log' for more details.

[root@vastdata4 ~]# cat /tmp/silentInstall.logsilentInstall.logsNativeVolName:/u01/app/11.2.0/grid/install/m_asNodeArray:vastdata4,vastdata3m_sLocalNode:vastdata4WARNING: A new inventory has been created on one or more nodes in this session. However, it has not yet been registered as the central inventory of this system. To register the new inventory please run the script at '/u01/app/oraInventory/orainstRoot.sh' with root privileges on nodes 'vastdata3'.If you do not register the inventory, you may not be able to update or patch the products you installed.The following configuration scripts need to be executed as the "root" user in each new cluster node. Each script in the list below is followed by a list of nodes./u01/app/oraInventory/orainstRoot.sh #On nodes vastdata3/u01/app/11.2.0/grid/root.sh #On nodes vastdata3To execute the configuration scripts:    1. Open a terminal window    2. Log in as "root"    3. Run the scripts in each cluster node    The Cluster Node Addition of /u01/app/11.2.0/grid was successful. ------------------------------------------------------------------------------------------------------------------------------------ [root@vastdata3 ~]# /u01/app/oraInventory/orainstRoot.shCreating the Oracle inventory pointer file (/etc/oraInst.loc)Changing permissions of /u01/app/oraInventory.Adding read,write permissions for group.Removing read,write,execute permissions for world. Changing groupname of /u01/app/oraInventory to oinstall.The execution of the script is complete.[root@vastdata3 ~]# /u01/app/11.2.0/grid/root.shPerforming root user operation for Oracle 11g  The following environment variables are set as:    ORACLE_OWNER= grid    ORACLE_HOME=  /u01/app/11.2.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite.The contents of "oraenv" have not changed. No need to overwrite.The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root script.Now product-specific root actions will be performed.Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_paramsUser ignored Prerequisites during installationInstalling Trace File AnalyzerOLR initialization - successfulAdding Clusterware entries to upstartCRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node vastdata4, number 2, and is terminatingAn active cluster was found during exclusive startup, restarting to join the clusterclscfg: EXISTING configuration version 5 detected.clscfg: version 5 is 11g Release 2.Successfully accumulated necessary OCR keys.Creating OCR keys for user 'root', privgrp 'root'..Operation successful.Preparing packages for installation...cvuqdisk-1.0.9-1Configure Oracle Grid Infrastructure for a Cluster ... succeeded

(4) 节点添加后的检查

[grid@vastdata4 ~]$ cluvfy stage -post nodeadd -n vastdata3 -verbose Performing post-checks for node addition  Checking node reachability... Check: Node reachability from node "vastdata4"  Destination Node                      Reachable?                ------------------------------------  ------------------------  vastdata3                             yes                     Result: Node reachability check passed from node "vastdata4"  Checking user equivalence... Check: User equivalence for user "grid"  Node Name                             Status                    ------------------------------------  ------------------------  vastdata3                             passed                  Result: User equivalence check passed for user "grid" Checking cluster integrity...   Node Name                             ------------------------------------  vastdata3                             vastdata4                            Cluster integrity check passed  Checking CRS integrity... Clusterware version consistency passedThe Oracle Clusterware is healthy on node "vastdata4"The Oracle Clusterware is healthy on node "vastdata3" CRS integrity check passed Checking shared resources... Checking CRS home location..."/u01/app/11.2.0/grid" is not sharedResult: Shared resources check for node addition passed  Checking node connectivity... Checking hosts config file...  Node Name                             Status                    ------------------------------------  ------------------------  vastdata4                             passed                    vastdata3                             passed                   Verification of the hosts config file successful  Interface information for node "vastdata4" Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU    ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0   192.168.0.12    192.168.0.0     0.0.0.0         UNKNOWN         08:00:27:EF:F6:15 1500   eth0   192.168.0.33    192.168.0.0     0.0.0.0         UNKNOWN         08:00:27:EF:F6:15 1500   eth0   192.168.0.35    192.168.0.0     0.0.0.0         UNKNOWN         08:00:27:EF:F6:15 1500   eth0   192.168.0.23    192.168.0.0     0.0.0.0         UNKNOWN         08:00:27:EF:F6:15 1500   eth2   10.0.0.4        10.0.0.0        0.0.0.0         UNKNOWN         08:00:27:82:3A:57 1500   eth2   169.254.17.92   169.254.0.0     0.0.0.0         UNKNOWN         08:00:27:82:3A:57 1500   eth3   10.0.0.5        10.0.0.0        0.0.0.0         UNKNOWN         08:00:27:4C:5D:39 1500   eth3   169.254.236.167 169.254.128.0   0.0.0.0         UNKNOWN         08:00:27:4C:5D:39 1500    Interface information for node "vastdata3" Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU    ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0   192.168.0.11    192.168.0.0     0.0.0.0         UNKNOWN         08:00:27:D3:55:CB 1500   eth0   192.168.0.34    192.168.0.0     0.0.0.0         UNKNOWN         08:00:27:D3:55:CB 1500   eth0   192.168.0.22    192.168.0.0     0.0.0.0         UNKNOWN         08:00:27:D3:55:CB 1500   eth2   10.0.0.2        10.0.0.0        0.0.0.0         UNKNOWN         08:00:27:B4:C3:C7 1500   eth2   169.254.73.97   169.254.0.0     0.0.0.0         UNKNOWN         08:00:27:B4:C3:C7 1500   eth3   10.0.0.3        10.0.0.0        0.0.0.0         UNKNOWN         08:00:27:02:BD:9A 1500   eth3   169.254.222.114 169.254.128.0   0.0.0.0         UNKNOWN         08:00:27:02:BD:9A 1500    Check: Node connectivity for interface "eth0"  Source                          Destination                     Connected?        ------------------------------  ------------------------------  ----------------  vastdata4[192.168.0.12]         vastdata4[192.168.0.33]         yes               vastdata4[192.168.0.12]         vastdata4[192.168.0.35]         yes               vastdata4[192.168.0.12]         vastdata4[192.168.0.23]         yes               vastdata4[192.168.0.12]         vastdata3[192.168.0.11]         yes               vastdata4[192.168.0.12]         vastdata3[192.168.0.34]         yes               vastdata4[192.168.0.12]         vastdata3[192.168.0.22]         yes               vastdata4[192.168.0.33]         vastdata4[192.168.0.35]         yes               vastdata4[192.168.0.33]         vastdata4[192.168.0.23]         yes               vastdata4[192.168.0.33]         vastdata3[192.168.0.11]         yes               vastdata4[192.168.0.33]         vastdata3[192.168.0.34]         yes               vastdata4[192.168.0.33]         vastdata3[192.168.0.22]         yes               vastdata4[192.168.0.35]         vastdata4[192.168.0.23]         yes               vastdata4[192.168.0.35]         vastdata3[192.168.0.11]         yes               vastdata4[192.168.0.35]         vastdata3[192.168.0.34]         yes               vastdata4[192.168.0.35]         vastdata3[192.168.0.22]         yes               vastdata4[192.168.0.23]         vastdata3[192.168.0.11]         yes               vastdata4[192.168.0.23]         vastdata3[192.168.0.34]         yes               vastdata4[192.168.0.23]         vastdata3[192.168.0.22]         yes               vastdata3[192.168.0.11]         vastdata3[192.168.0.34]         yes               vastdata3[192.168.0.11]         vastdata3[192.168.0.22]         yes               vastdata3[192.168.0.34]         vastdata3[192.168.0.22]         yes             Result: Node connectivity passed for interface "eth0"  Check: TCP connectivity of subnet "192.168.0.0"  Source                          Destination                     Connected?        ------------------------------  ------------------------------  ----------------  vastdata4:192.168.0.12          vastdata4:192.168.0.33          passed            vastdata4:192.168.0.12          vastdata4:192.168.0.35          passed            vastdata4:192.168.0.12          vastdata4:192.168.0.23          passed            vastdata4:192.168.0.12          vastdata3:192.168.0.11          passed            vastdata4:192.168.0.12          vastdata3:192.168.0.34          passed            vastdata4:192.168.0.12          vastdata3:192.168.0.22          passed          Result: TCP connectivity check passed for subnet "192.168.0.0"  Check: Node connectivity for interface "eth2"  Source                          Destination                     Connected?        ------------------------------  ------------------------------  ----------------  vastdata4[10.0.0.4]             vastdata4[10.0.0.5]             yes               vastdata4[10.0.0.4]             vastdata3[10.0.0.2]             yes               vastdata4[10.0.0.4]             vastdata3[10.0.0.3]             yes               vastdata4[10.0.0.5]             vastdata3[10.0.0.2]             yes               vastdata4[10.0.0.5]             vastdata3[10.0.0.3]             yes               vastdata3[10.0.0.2]             vastdata3[10.0.0.3]             yes             Result: Node connectivity passed for interface "eth2"  Check: TCP connectivity of subnet "10.0.0.0"  Source                          Destination                     Connected?        ------------------------------  ------------------------------  ----------------  vastdata4:10.0.0.4              vastdata4:10.0.0.5              passed            vastdata4:10.0.0.4              vastdata3:10.0.0.2              passed            vastdata4:10.0.0.4              vastdata3:10.0.0.3              passed          Result: TCP connectivity check passed for subnet "10.0.0.0"  Check: Node connectivity for interface "eth3"Checking subnet mask consistency...Subnet mask consistency check passed for subnet "192.168.0.0".Subnet mask consistency check passed for subnet "10.0.0.0".Subnet mask consistency check passed. Result: Node connectivity check passed Checking multicast communication... Checking subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0"...Check of subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0" passed. Checking subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0"...Check of subnet "10.0.0.0" for multicast communication with multicast group "230.0.1.0" passed. Check of multicast communication passed. Checking node application existence... Checking existence of VIP node application (required)  Node Name     Required                  Running?                  Comment     ------------  ------------------------  ------------------------  ----------  vastdata4     yes                       yes                       passed      vastdata3     yes                       yes                       passed    VIP node application check passed Checking existence of NETWORK node application (required)  Node Name     Required                  Running?                  Comment     ------------  ------------------------  ------------------------  ----------  vastdata4     yes                       yes                       passed      vastdata3     yes                       yes                       passed    NETWORK node application check passed Checking existence of GSD node application (optional)  Node Name     Required                  Running?                  Comment     ------------  ------------------------  ------------------------  ----------  vastdata4     no                        no                        exists      vastdata3     no                        no                        exists    GSD node application is offline on nodes "vastdata4,vastdata3" Checking existence of ONS node application (optional)  Node Name     Required                  Running?                  Comment     ------------  ------------------------  ------------------------  ----------  vastdata4     no                        yes                       passed      vastdata3     no                        yes                       passed    ONS node application check passed  Checking Single Client Access Name (SCAN)...  SCAN Name         Node          Running?      ListenerName  Port          Running?      ----------------  ------------  ------------  ------------  ------------  ------------  vastdata-scan.us.oracle.com  vastdata3     true          LISTENER_SCAN1  1521          true          vastdata-scan.us.oracle.com  vastdata4     true          LISTENER_SCAN2  1521          true          vastdata-scan.us.oracle.com  vastdata4     true          LISTENER_SCAN3  1521          true         Checking TCP connectivity to SCAN Listeners...  Node          ListenerName              TCP connectivity?         ------------  ------------------------  ------------------------  vastdata4     LISTENER_SCAN1            yes                       vastdata4     LISTENER_SCAN2            yes                       vastdata4     LISTENER_SCAN3            yes                     TCP connectivity to SCAN Listeners exists on all cluster nodes Checking name resolution setup for "vastdata-scan.us.oracle.com"... Checking integrity of name service switch configuration file "/etc/nsswitch.conf" ...Checking if "hosts" entry in file "/etc/nsswitch.conf" is consistent across nodes...Checking file "/etc/nsswitch.conf" to make sure that only one "hosts" entry is definedMore than one "hosts" entry does not exist in any "/etc/nsswitch.conf" fileCheck for integrity of name service switch configuration file "/etc/nsswitch.conf" passed   SCAN Name     IP Address                Status                    Comment     ------------  ------------------------  ------------------------  ----------  vastdata-scan.us.oracle.com  192.168.0.33              passed                                vastdata-scan.us.oracle.com  192.168.0.34              passed                                vastdata-scan.us.oracle.com  192.168.0.35              passed                               Verification of SCAN VIP and Listener setup passed Checking to make sure user "grid" is not in "root" group  Node Name     Status                    Comment                   ------------  ------------------------  ------------------------  vastdata3     passed                    does not exist          Result: User "grid" is not part of "root" group. Check passed Checking if Clusterware is installed on all nodes...Check of Clusterware install passed Checking if CTSS Resource is running on all nodes...Check: CTSS Resource running on all nodes  Node Name                             Status                    ------------------------------------  ------------------------  vastdata3                             passed                  Result: CTSS resource check passed  Querying CTSS for time offset on all nodes...Result: Query of CTSS for time offset passed Check CTSS state started...Check: CTSS state  Node Name                             State                     ------------------------------------  ------------------------  vastdata3                             Active                  CTSS is in Active state. Proceeding with check of clock time offsets on all nodes...Reference Time Offset Limit: 1000.0 msecsCheck: Reference Time Offset  Node Name     Time Offset               Status                    ------------  ------------------------  ------------------------  vastdata3     0.0                       passed                   Time offset is within the specified limits on the following set of nodes: "[vastdata3]" Result: Check of clock time offsets passed Oracle Cluster Time Synchronization Services check passed Post-check for node addition was successful.

(5) 检查集群内节点状态

[grid@vastdata4 ~]$ olsnodes -tvastdata3 Unpinnedvastdata4 Unpinned

(6) 检查集群资源状态

[grid@vastdata4 ~]$ crsctl stat res -t--------------------------------------------------------------------------------NAME           TARGET  STATE        SERVER                   STATE_DETAILS       --------------------------------------------------------------------------------Local Resources--------------------------------------------------------------------------------ora.DATA.dg               ONLINE  ONLINE       vastdata3                                                   ONLINE  ONLINE       vastdata4                                    ora.FRA.dg               ONLINE  ONLINE       vastdata3                                                   ONLINE  ONLINE       vastdata4                                    ora.LISTENER.lsnr               ONLINE  ONLINE       vastdata3                                                   ONLINE  ONLINE       vastdata4                                    ora.asm               ONLINE  ONLINE       vastdata3                Started                            ONLINE  ONLINE       vastdata4                Started             ora.gsd               OFFLINE OFFLINE      vastdata3                                                   OFFLINE OFFLINE      vastdata4                                    ora.net1.network               ONLINE  ONLINE       vastdata3                                                   ONLINE  ONLINE       vastdata4                                    ora.ons               ONLINE  ONLINE       vastdata3                                                   ONLINE  ONLINE       vastdata4                                    ora.registry.acfs               ONLINE  ONLINE       vastdata3                                                   ONLINE  ONLINE       vastdata4                                    --------------------------------------------------------------------------------Cluster Resources--------------------------------------------------------------------------------ora.LISTENER_SCAN1.lsnr      1        ONLINE  ONLINE       vastdata3                                    ora.LISTENER_SCAN2.lsnr      1        ONLINE  ONLINE       vastdata4                                    ora.LISTENER_SCAN3.lsnr      1        ONLINE  ONLINE       vastdata4                                    ora.cvu      1        ONLINE  ONLINE       vastdata4                                    ora.oc4j      1        ONLINE  ONLINE       vastdata4                                    ora.prod.db      2        ONLINE  ONLINE       vastdata4                Open                ora.scan1.vip      1        ONLINE  ONLINE       vastdata3                                    ora.scan2.vip      1        ONLINE  ONLINE       vastdata4                                    ora.scan3.vip      1        ONLINE  ONLINE       vastdata4                                    ora.vastdata3.vip      1        ONLINE  ONLINE       vastdata3                                    ora.vastdata4.vip      1        ONLINE  ONLINE       vastdata4

三. 新节点安装 DB 软件

[oracle@vastdata4 ~]$ $ORACLE_HOME/oui/bin/addNode.sh  -silent "CLUSTER_NEW_NODES={vastdata3}" Performing pre-checks for node addition  Checking node reachability...Node reachability check passed from node "vastdata4" Checking user equivalence...User equivalence check passed for user "oracle" WARNING: Node "vastdata3" already appears to be part of cluster Pre-check for node addition was successful. Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB.   Actual 6143 MB    PassedOracle Universal Installer, Version 11.2.0.4.0 ProductionCopyright (C) 1999, 2013, Oracle. All rights reserved.  Performing tests to see whether nodes vastdata3 are available............................................................... 100% Done. ..-----------------------------------------------------------------------------Cluster Node Addition SummaryGlobal Settings   Source: /u01/app/oracle/product/11.2.0/db_1   New NodesSpace Requirements   New Nodes      vastdata3         /: Required 4.34GB : Available 10.59GBInstalled Products   Product Names      Oracle Database 11g 11.2.0.4.0       Java Development Kit 1.5.0.51.10       Installer SDK Component 11.2.0.4.0       Oracle One-Off Patch Installer 11.2.0.3.4       Oracle Universal Installer 11.2.0.4.0       Oracle USM Deconfiguration 11.2.0.4.0       Oracle Configuration Manager Deconfiguration 10.3.1.0.0       Oracle DBCA Deconfiguration 11.2.0.4.0       Oracle RAC Deconfiguration 11.2.0.4.0       Oracle Database Deconfiguration 11.2.0.4.0       Oracle Configuration Manager Client 10.3.2.1.0       Oracle Configuration Manager 10.3.8.1.0       Oracle ODBC Driverfor Instant Client 11.2.0.4.0       LDAP Required Support Files 11.2.0.4.0       SSL Required Support Files for InstantClient 11.2.0.4.0       Bali Share 1.1.18.0.0       Oracle Extended Windowing Toolkit 3.4.47.0.0       Oracle JFC Extended Windowing Toolkit 4.2.36.0.0       Oracle Real Application Testing 11.2.0.4.0       Oracle Database Vault J2EE Application 11.2.0.4.0       Oracle Label Security 11.2.0.4.0       Oracle Data Mining RDBMS Files 11.2.0.4.0       Oracle OLAP RDBMS Files 11.2.0.4.0       Oracle OLAP API 11.2.0.4.0       Platform Required Support Files 11.2.0.4.0       Oracle Database Vault option 11.2.0.4.0       Oracle RAC Required Support Files-HAS 11.2.0.4.0       SQL*Plus Required Support Files 11.2.0.4.0       Oracle Display Fonts 9.0.2.0.0       Oracle Ice Browser 5.2.3.6.0       Oracle JDBC Server Support Package 11.2.0.4.0       Oracle SQL Developer 11.2.0.4.0       Oracle Application Express 11.2.0.4.0       XDK Required Support Files 11.2.0.4.0       RDBMS Required Support Files for Instant Client 11.2.0.4.0       SQLJ Runtime 11.2.0.4.0       Database Workspace Manager 11.2.0.4.0       RDBMS Required Support Files Runtime 11.2.0.4.0       Oracle Globalization Support 11.2.0.4.0       Exadata Storage Server 11.2.0.1.0       Provisioning Advisor Framework 10.2.0.4.3       Enterprise Manager Database Plugin -- Repository Support 11.2.0.4.0       Enterprise Manager Repository Core Files 10.2.0.4.5       Enterprise Manager Database Plugin -- Agent Support 11.2.0.4.0       Enterprise Manager Grid Control Core Files 10.2.0.4.5       Enterprise Manager Common Core Files 10.2.0.4.5       Enterprise Manager Agent Core Files 10.2.0.4.5       RDBMS Required Support Files 11.2.0.4.0       regexp 2.1.9.0.0       Agent Required Support Files 10.2.0.4.5       Oracle 11g Warehouse Builder Required Files 11.2.0.4.0       Oracle Notification Service (eONS) 11.2.0.4.0       Oracle Text Required Support Files 11.2.0.4.0       Parser Generator Required Support Files 11.2.0.4.0       Oracle Database 11g Multimedia Files 11.2.0.4.0       Oracle Multimedia Java Advanced Imaging 11.2.0.4.0       Oracle Multimedia Annotator 11.2.0.4.0       Oracle JDBC/OCI Instant Client 11.2.0.4.0       Oracle Multimedia Locator RDBMS Files 11.2.0.4.0       Precompiler Required Support Files 11.2.0.4.0       Oracle Core Required Support Files 11.2.0.4.0       Sample Schema Data 11.2.0.4.0       Oracle Starter Database 11.2.0.4.0       Oracle Message Gateway Common Files 11.2.0.4.0       Oracle XML Query 11.2.0.4.0       XML Parser for Oracle JVM 11.2.0.4.0       Oracle Help For Java 4.2.9.0.0       Installation Plugin Files 11.2.0.4.0       Enterprise Manager Common Files 10.2.0.4.5       Expat libraries 2.0.1.0.1       Deinstallation Tool 11.2.0.4.0       Oracle Quality of Service Management (Client) 11.2.0.4.0       Perl Modules 5.10.0.0.1       JAccelerator (COMPANION) 11.2.0.4.0       Oracle Containers for Java 11.2.0.4.0       Perl Interpreter 5.10.0.0.2       Oracle Net Required Support Files 11.2.0.4.0       Secure Socket Layer 11.2.0.4.0       Oracle Universal Connection Pool 11.2.0.4.0       Oracle JDBC/THIN Interfaces 11.2.0.4.0       Oracle Multimedia Client Option 11.2.0.4.0       Oracle Java Client 11.2.0.4.0       Character Set Migration Utility 11.2.0.4.0       Oracle Code Editor 1.2.1.0.0I       PL/SQL Embedded Gateway 11.2.0.4.0       OLAP SQL Scripts 11.2.0.4.0       Database SQL Scripts 11.2.0.4.0       Oracle Locale Builder 11.2.0.4.0       Oracle Globalization Support 11.2.0.4.0       SQL*Plus Files for Instant Client 11.2.0.4.0       Required Support Files 11.2.0.4.0       Oracle Database User Interface 2.2.13.0.0       Oracle ODBC Driver 11.2.0.4.0       Oracle Notification Service 11.2.0.3.0       XML Parser for Java 11.2.0.4.0       Oracle Security Developer Tools 11.2.0.4.0       Oracle Wallet Manager 11.2.0.4.0       Cluster Verification Utility Common Files 11.2.0.4.0       Oracle Clusterware RDBMS Files 11.2.0.4.0       Oracle UIX 2.2.24.6.0       Enterprise Manager plugin Common Files 11.2.0.4.0       HAS Common Files 11.2.0.4.0       Precompiler Common Files 11.2.0.4.0       Installation Common Files 11.2.0.4.0       Oracle Help for the  Web 2.0.14.0.0       Oracle LDAP administration 11.2.0.4.0       Buildtools Common Files 11.2.0.4.0       Assistant Common Files 11.2.0.4.0       Oracle Recovery Manager 11.2.0.4.0       PL/SQL 11.2.0.4.0       Generic Connectivity Common Files 11.2.0.4.0       Oracle Database Gateway for ODBC 11.2.0.4.0       Oracle Programmer 11.2.0.4.0       Oracle Database Utilities 11.2.0.4.0       Enterprise Manager Agent 10.2.0.4.5       SQL*Plus 11.2.0.4.0       Oracle Netca Client 11.2.0.4.0       Oracle Multimedia Locator 11.2.0.4.0       Oracle Call Interface (OCI) 11.2.0.4.0       Oracle Multimedia 11.2.0.4.0       Oracle Net 11.2.0.4.0       Oracle XML Development Kit 11.2.0.4.0       Oracle Internet Directory Client 11.2.0.4.0       Database Configuration and Upgrade Assistants 11.2.0.4.0       Oracle JVM 11.2.0.4.0       Oracle Advanced Security 11.2.0.4.0       Oracle Net Listener 11.2.0.4.0       Oracle Enterprise Manager Console DB 11.2.0.4.0       HAS Files for DB 11.2.0.4.0       Oracle Text 11.2.0.4.0       Oracle Net Services 11.2.0.4.0       Oracle Database 11g 11.2.0.4.0       Oracle OLAP 11.2.0.4.0       Oracle Spatial 11.2.0.4.0       Oracle Partitioning 11.2.0.4.0       Enterprise Edition Options 11.2.0.4.0 -----------------------------------------------------------------------------  Instantiating scripts for add node (Monday, February 25, 2019 1:20:38 AM EST).                                                                 1% Done.Instantiation of add node scripts complete Copying to remote nodes (Monday, February 25, 2019 1:20:44 AM EST)...............................................................................................                                 96% Done.Home copied to new nodes Saving inventory on nodes (Monday, February 25, 2019 1:26:47 AM EST).                                                               100% Done.Save inventory completeWARNING:The following configuration scripts need to be executed as the "root" user in each new cluster node. Each script in the list below is followed by a list of nodes./u01/app/oracle/product/11.2.0/db_1/root.sh #On nodes vastdata3To execute the configuration scripts:    1. Open a terminal window    2. Log in as "root"    3. Run the scripts in each cluster node    The Cluster Node Addition of /u01/app/oracle/product/11.2.0/db_1 was successful.Please check '/tmp/silentInstall.log' for more details.

[oracle@vastdata4 ~]$ cat /tmp/silentInstall.logsilentInstall.logsNativeVolName:/u01/app/11.2.0/grid/install/m_asNodeArray:vastdata4,vastdata3m_sLocalNode:vastdata4WARNING: A new inventory has been created on one or more nodes in this session. However, it has not yet been registered as the central inventory of this system. To register the new inventory please run the script at '/u01/app/oraInventory/orainstRoot.sh' with root privileges on nodes 'vastdata3'.If you do not register the inventory, you may not be able to update or patch the products you installed.The following configuration scripts need to be executed as the "root" user in each new cluster node. Each script in the list below is followed by a list of nodes./u01/app/oraInventory/orainstRoot.sh #On nodes vastdata3/u01/app/11.2.0/grid/root.sh #On nodes vastdata3To execute the configuration scripts:    1. Open a terminal window    2. Log in as "root"    3. Run the scripts in each cluster node    The Cluster Node Addition of /u01/app/11.2.0/grid was successful. ------------------------------------------------------------------------------------------------------------------------------------[root@vastdata3 ~]# /u01/app/oraInventory/orainstRoot.shCreating the Oracle inventory pointer file (/etc/oraInst.loc)Changing permissions of /u01/app/oraInventory.Adding read,write permissions for group.Removing read,write,execute permissions for world. Changing groupname of /u01/app/oraInventory to oinstall.The execution of the script is complete.[root@vastdata3 ~]# /u01/app/11.2.0/grid/root.shPerforming root user operation for Oracle 11g  The following environment variables are set as:    ORACLE_OWNER= grid    ORACLE_HOME=  /u01/app/11.2.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite.The contents of "oraenv" have not changed. No need to overwrite.The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root script.Now product-specific root actions will be performed.Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_paramsUser ignored Prerequisites during installationInstalling Trace File AnalyzerConfigure Oracle Grid Infrastructure for a Cluster ... succeeded

四 . 给新节点分配实例

1. 检查 RAC 下所有实例状态

[oracle@vastdata4 ~]$ srvctl status database -d PROD -fInstance PROD2 is running on node vastdata4  [oracle@vastdata4 ~]$ srvctl config database -d PROD -aDatabase unique name: PRODDatabase name: PRODOracle home: /u01/app/oracle/product/11.2.0/db_1Oracle user: oracleSpfile: +DATA/PROD/spfilePROD.oraDomain: Start options: openStop options: immediateDatabase role: PRIMARYManagement policy: AUTOMATICServer pools: PRODDatabase instances: PROD2Disk Groups: DATA,FRAMount point paths: Services: Type: RACDatabase is enabledDatabase is administrator managed

2. 为新增节点分配实例

[oracle@vastdata4 ~]$ dbca -silent -addInstance -gdbName PROD -nodelist vastdata3 -instanceName PROD1 -sysDBAUserName sys -sysDBAPassword oracleAdding instance1% complete2% complete6% complete13% complete20% complete26% complete33% complete40% complete46% complete53% complete66% completeCompleting instance management.76% complete100% completeLook at the log file "/u01/app/oracle/cfgtoollogs/dbca/PROD/PROD.log" for further details. [oracle@vastdata4 ~]$ srvctl config database -d PROD -aDatabase unique name: PRODDatabase name: PRODOracle home: /u01/app/oracle/product/11.2.0/db_1Oracle user: oracleSpfile: +DATA/PROD/spfilePROD.oraDomain: Start options: openStop options: immediateDatabase role: PRIMARYManagement policy: AUTOMATICServer pools: PRODDatabase instances: PROD1,PROD2Disk Groups: DATA,FRAMount point paths: Services: Type: RACDatabase is enabledDatabase is administrator managed

3. 检查集群资源状态

[grid@vastdata4 ~]$ crsctl stat res -t--------------------------------------------------------------------------------NAME           TARGET  STATE        SERVER                   STATE_DETAILS       --------------------------------------------------------------------------------Local Resources--------------------------------------------------------------------------------ora.DATA.dg               ONLINE  ONLINE       vastdata3                                                   ONLINE  ONLINE       vastdata4                                    ora.FRA.dg               ONLINE  ONLINE       vastdata3                                                   ONLINE  ONLINE       vastdata4                                    ora.LISTENER.lsnr               ONLINE  ONLINE       vastdata3                                                   ONLINE  ONLINE       vastdata4                                    ora.asm               ONLINE  ONLINE       vastdata3                Started                            ONLINE  ONLINE       vastdata4                Started             ora.gsd               OFFLINE OFFLINE      vastdata3                                                   OFFLINE OFFLINE      vastdata4                                    ora.net1.network               ONLINE  ONLINE       vastdata3                                                   ONLINE  ONLINE       vastdata4                                    ora.ons               ONLINE  ONLINE       vastdata3                                                   ONLINE  ONLINE       vastdata4                                    ora.registry.acfs               ONLINE  ONLINE       vastdata3                                                   ONLINE  ONLINE       vastdata4                                    --------------------------------------------------------------------------------Cluster Resources--------------------------------------------------------------------------------ora.LISTENER_SCAN1.lsnr      1        ONLINE  ONLINE       vastdata3                                    ora.LISTENER_SCAN2.lsnr      1        ONLINE  ONLINE       vastdata4                                    ora.LISTENER_SCAN3.lsnr      1        ONLINE  ONLINE       vastdata4                                    ora.cvu      1        ONLINE  ONLINE       vastdata4                                    ora.oc4j      1        ONLINE  ONLINE       vastdata4                                    ora.prod.db      1        ONLINE  ONLINE       vastdata3                Open                      2        ONLINE  ONLINE       vastdata4                Open                ora.scan1.vip      1        ONLINE  ONLINE       vastdata3                                    ora.scan2.vip      1        ONLINE  ONLINE       vastdata4                                    ora.scan3.vip      1        ONLINE  ONLINE       vastdata4                                    ora.vastdata3.vip      1        ONLINE  ONLINE       vastdata3                                    ora.vastdata4.vip      1        ONLINE  ONLINE       vastdata4

4. 检查新添加节点数据库的运行情况

[oracle@vastdata3 ~]$ sqlplus / as sysdba SQL*Plus: Release 11.2.0.4.0 Production on Mon Feb 25 01:44:56 2019 Copyright (c) 1982, 2013, Oracle.  All rights reserved.  Connected to:Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit ProductionWith the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,Data Mining and Real Application Testing options SQL> select count(*) from tab;   COUNT(*)----------      4987

如有转载，请标明出处。