Oracle 11g RAC集群删除节点和重建一一、前提说明本文章计划删除节点四这一点要注意1、本文章实验在一个4节点RAC集群中删除一个节点实际如果是2节点RAC集群环境步骤基本类似完全可以参考2、实验环境的安装详见我以前的文章《Oracle 11g RAC集群安装_linux7》和《Oracle 11g RAC集群新增节点》3、本文章计划删除节点四4、整个节点删除工作大致围绕三个方面展开删除实例、删除Oracle数据库软件、删除GI软件5、本文章删除节点的方法主要通过GI和oracle自带的卸载工具完成节点删除后续会继续写一篇文章直接通过rm -rf直接删除RAC集群节点6、在删除节点四后会重建集群即节点四重新加入集群7、文章适用于Oracle RAC集群中一个节点故障或损坏需要将该节点从集群中删除并重新替换新增节点8、节点一、节点二、节点三可以统称为保留节点或者存活节点二、背景介绍集群环境配置如下 项目 节点一服务器 节点二服务器 节点三服务器 节点四服务器 主机名 hostrac1 hostrac2 hostrac1 hostrac2 操作系统 centos7.9_x64 centos7.9_x64 centos7.9_x64 centos7.9_x64 集群和数据库软件 Oracle 11.2.0.4 Oracle 11.2.0.4 Oracle 11.2.0.4 Oracle 11.2.0.4 公共网络 192.168.11.22 192.168.11.23 192.168.11.27 192.168.11.28 私有网络 10.10.10.22 10.10.10.23 10.10.10.27 10.10.10.28 虚拟VIP 192.168.11.24 192.168.11.25 192.168.11.29 192.168.11.30 SCANIP 192.168.11.26 备注整个集群两个节点共享一个SCANIP地址。三、准备工作条件允许的情况下强烈建议下面的工作1、备份OCR/OLR集群配置文件2、数据库备份3、软件目录备份四、正式删除节点1、删除数据库实例删除节点四上运行的数据库实例1关闭计划删除的目标实例如果目标实例已经故障没有启动这一步骤可以省略 任意节点执行如下关闭实例命令都需 su - oracle srvctl stop instance -d orcl -i orcl4 -o abort 检查验证 srvctl status database -d orcl Instance orcl1 is running on node hostrac1 Instance orcl2 is running on node hostrac2 Instance orcl3 is running on node hostrac3 Instance orcl4 is not running on node hostrac42删除实例在保留节点上执行节点一、节点二、节点三都可以方法一通过dbca图形界面方式完成删除依次选择 Real Application Clusters -- Instance Management --- Delete Instance 方法二dbca静默方式完成删除实例 删除命令 dbca -silent -deleteInstance [-nodeList node_name] -gdbName gdb_name -instanceName instance_name -sysDBAUserName sysdba -sysDBAPassword password node_name 是删除节点名 gdb_name 是全局数据库名 instance 是删除的实例名 sysdba 是拥有sysdba权限的oracle用户名称 passwd 是sysdba用户的密码 具体执行 dbca -silent -deleteInstance -nodeList hostrac4 -gdbName orcl -instanceName orcl4 -sysDBAUserName sys -sysDBAPassword oracle 输出类似如下 [oraclehostrac1 ~]$ dbca -silent -deleteInstance -nodeList hostrac4 -gdbName orcl -instanceName orcl4 -sysDBAUserName sys -sysDBAPassword oracle Deleting instance 1% complete 2% complete 6% complete 13% complete 20% complete 26% complete 33% complete 40% complete 46% complete 53% complete 60% complete 66% complete Completing instance management. 100% complete Look at the log file /u01/app/oracle/cfgtoollogs/dbca/orcl.log for further details.3检查验证在保留节点上执行节点一、节点二、节点三都可以su - oarcle srvctl status database -d orcl srvctl config database -d orcl2、删除Oracle数据库软件1禁用和停止节点四上的监听在保留节点上执行节点一、节点二、节点三都可以su - grid srvctl disable listener -l LISTENER -n hostrac4 srvctl stop listener -l LISTENER -n hostrac4 验证 crsctl stat res -t2更新inventory目录在待删除的节点四上执行更新inventory命令语法 $ORACLE_HOME/oui/bin/runInstaller -updateNodeList ORACLE_HOMEOracle_home_location CLUSTER_NODES{name_of_node_to_delete} -local 具体如下 su - oracle $ORACLE_HOME/oui/bin/runInstaller -updateNodeList ORACLE_HOME$ORACLE_HOME CLUSTER_NODEShostrac4 -local3卸载Oracle数据库软件在待删除的节点四上执行su - oracle $ORACLE_HOME/deinstall/deinstall -local 备注如果卸载不干净需要人为节点四上手工执行命令rm -rf $ORACLE_HOME/*4更新其他的节点的inventory在保留节点上都需要执行节点一、节点二、节点三都需要更新更新inventory命令 $ORACLE_HOME/oui/bin/runInstaller -updateNodeList ORACLE_HOMEOracle_home_location CLUSTER_NODES{remaining_node_list} 具体如下 su - oracle $ORACLE_HOME/oui/bin/runInstaller -updateNodeList ORACLE_HOME$ORACLE_HOME CLUSTER_NODES{hostrac1,hostrac2,hostrac3} -local3、删除GI软件1 检查被删除节点状态在保留节点上执行节点一、节点二、节点三都可以[gridhostrac3 ~]$ olsnodes -s -n -t hostrac1 1 Active Unpinned hostrac2 2 Active Unpinned hostrac3 3 Active Unpinned hostrac4 4 Active Unpinned2如果节点四是pined状态需要执行以下命令UNPINroot用户执行在保留节点上执行节点一、节点二、节点三都可以类似如下 [roothostrac3 ~]# /u01/app/crs/bin/crsctl unpin css -n hostrac4 CRS-4667: Node hostrac4 successfully unpinned.3停止被删节点HAS服务在待删除的节点四上执行在待删除节点的$ORACLE_HOME/crs/install目录下运行rootcrs.pl脚本root用户执行 su - root /u01/app/crs/crs/install/rootcrs.pl -deconfig -force 输出类似如下 [roothostrac4 ~]# /u01/app/crs/crs/install/rootcrs.pl -deconfig -force Using configuration parameter file: /u01/app/crs/crs/install/crsconfig_params Network exists: 1/192.168.11.0/255.255.255.0/ens33, type static VIP exists: /hostrac1-vip/192.168.11.24/192.168.11.0/255.255.255.0/ens33, hosting node hostrac1 VIP exists: /hostrac2-vip/192.168.11.25/192.168.11.0/255.255.255.0/ens33, hosting node hostrac2 VIP exists: /hostrac3-vip/192.168.11.29/192.168.11.0/255.255.255.0/ens33, hosting node hostrac3 VIP exists: /hostrac4-vip/192.168.11.30/192.168.11.0/255.255.255.0/ens33, hosting node hostrac4 GSD exists ONS exists: Local port 6100, remote port 6200, EM port 2016 CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on hostrac4 CRS-2673: Attempting to stop ora.crsd on hostrac4 CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on hostrac4 CRS-2673: Attempting to stop ora.CRS.dg on hostrac4 CRS-2673: Attempting to stop ora.DATA.dg on hostrac4 CRS-2677: Stop of ora.DATA.dg on hostrac4 succeeded CRS-2677: Stop of ora.CRS.dg on hostrac4 succeeded CRS-2673: Attempting to stop ora.asm on hostrac4 CRS-2677: Stop of ora.asm on hostrac4 succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on hostrac4 has completed CRS-2677: Stop of ora.crsd on hostrac4 succeeded CRS-2673: Attempting to stop ora.mdnsd on hostrac4 CRS-2673: Attempting to stop ora.ctssd on hostrac4 CRS-2673: Attempting to stop ora.evmd on hostrac4 CRS-2673: Attempting to stop ora.asm on hostrac4 CRS-2677: Stop of ora.mdnsd on hostrac4 succeeded CRS-2677: Stop of ora.evmd on hostrac4 succeeded CRS-2677: Stop of ora.ctssd on hostrac4 succeeded CRS-2677: Stop of ora.asm on hostrac4 succeeded CRS-2673: Attempting to stop ora.cluster_interconnect.haip on hostrac4 CRS-2677: Stop of ora.cluster_interconnect.haip on hostrac4 succeeded CRS-2673: Attempting to stop ora.cssd on hostrac4 CRS-2677: Stop of ora.cssd on hostrac4 succeeded CRS-2673: Attempting to stop ora.crf on hostrac4 CRS-2677: Stop of ora.crf on hostrac4 succeeded CRS-2673: Attempting to stop ora.gipcd on hostrac4 CRS-2677: Stop of ora.gipcd on hostrac4 succeeded CRS-2673: Attempting to stop ora.gpnpd on hostrac4 CRS-2677: Stop of ora.gpnpd on hostrac4 succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on hostrac4 has completed CRS-4133: Oracle High Availability Services has been stopped. Removing Trace File Analyzer Successfully deconfigured Oracle clusterware stack on this node检查集群状态在保留节点上执行节点一、节点二、节点三都可以su - grid crsctl stat res -t olsnodes -s -n -t4从集群中删除节点四在保留节点上执行节点一、节点二、节点三都可以在保留节点的$ORACLE_HOME/bin目录下运行如下命令在集群中删除节点root用户执行 获取$ORACLE_HOME目录 su - grid env|grep ORACLE_HOME ORACLE_HOME/u01/app/crs root用户执行 /u01/app/crs/bin/crsctl delete node -n hostrac4 输出如下 [roothostrac1 ~]# /u01/app/crs/bin/crsctl delete node -n hostrac4 CRS-4661: Node hostrac4 successfully deleted. 验证 olsnodes -s -n -t5更新inventory目录在待删除的节点四上执行更新inventory命令语法 $ORACLE_HOME/oui/bin/runInstaller -updateNodeList ORACLE_HOMEOracle_home_location CLUSTER_NODES{remaining_nodes_list} CRSTRUE 具体如下 su - grid $ORACLE_HOME/oui/bin/runInstaller -updateNodeList ORACLE_HOME$ORACLE_HOME CLUSTER_NODEShostrac4 CRSTRUE -local6卸载 GI 软件在待删除的节点四上执行$ORACLE_HOME/deinstall/deinstall -local该步骤的输出在文章最后总结部分有详细说7更新inventory目录在保留节点上执行节点一、节点二、节点三都可以更新inventory命令语法 $ORACLE_HOME/oui/bin/runInstaller -updateNodeList ORACLE_HOMEOracle_home_location CLUSTER_NODES{name_of_node_to_delete} 具体如下 su - grid $ORACLE_HOME/oui/bin/runInstaller -updateNodeList ORACLE_HOME$ORACLE_HOME CLUSTER_NODES{hostrac1,hostrac2,hostrac3} CRSTRUE8验证节点是否删除成功这个步骤关系以后是否可以增加节点到集群中在保留节点上执行节点一、节点二、节点三都可以su - grid cluvfy stage -post nodedel -n hostrac4 -verbose 输出如下 [gridhostrac1 ~]$ cluvfy stage -post nodedel -n hostrac4 -verbose Performing post-checks for node removal Checking CRS integrity... Clusterware version consistency passed The Oracle Clusterware is healthy on node hostrac1 The Oracle Clusterware is healthy on node hostrac3 The Oracle Clusterware is healthy on node hostrac2 CRS integrity check passed Result: Node removal check passed Post-check for node removal was successful.9备份OCR在保留节点上执行节点一、节点二、节点三都可以 su - grid which ocrconfig /u01/app/crs/bin/ocrconfig //获取ocrconfig可执行文件的路径 root用户执行备份 /u01/app/crs/bin/ocrconfig -manualbackup /u01/app/crs/bin/ocrconfig -local -manualbackup 验证备份 /u01/app/crs/bin/ocrconfig -showbackup /u01/app/crs/bin/ocrconfig -local -showbackup4、验证GI集群目录和Oracle数据库软件目录已经清空在待删除的节点四上执行通过下面命令获取GI集群和Oracle数据库软件的基本目录 su - oracle env|grep ORACLE_BASE su - grid env|grep ORACLE_HOME 验证结果 [gridhostrac4 oracle]$ ls -ltr /u01/app total 0 drwxrwxr-x. 2 oracle oinstall 6 Jun 18 11:30 oracle [gridhostrac4 oracle]$ ls -ltr /u01/app/oracle total 0 备注如果还有残余的需要手工rm -rf方式来彻底删除至此完成集群节点删除四、将删除的节点四重新添加至集群准备工作一由于是将删除的节点重新加入下面的准备工作1-17步骤理论上都在安装阶段完成过无需执行1、配置主机名 2、/etc/hosts文件配置 3、集群GI和Oracle数据库安装用户创建 4、创建目录 5、oracle用户和grid用户ssh互信配置 6、共享磁盘配置共享设备与ASM磁盘映射 7、配置ntp时间同步 8、操作系统依赖包安装 9、关闭防火墙和selinux安全策略 10、停止和禁用linux7版本上的一些服务器 11、可选设置linux多用户模式即纯文本模式没有图形界面 12、Linux7版本上关闭透明大页和NUMA 13、grid和oracle用户环境变量配置 14、修改/etc/profile配置文件 15、修改内核参数、操作系统用户限制、安全策略 16、可选内存大于32G建议设置大页,可以参考以前的文章《Oracle11g 安装完毕后的配置优化步骤》 17、重启生效二18-22阶段的5个步骤需要手工执行18、源有环境备份数据库可选强烈建议19、集群配置文件备份OCR/OLR这个步骤必须执行节点一完成 su - grid which ocrconfig /u01/app/crs/bin/ocrconfig //获取ocrconfig可执行文件的路径 root用户执行备份 /u01/app/crs/bin/ocrconfig -manualbackup /u01/app/crs/bin/ocrconfig -local -manualbackup 验证备份 /u01/app/crs/bin/ocrconfig -showbackup /u01/app/crs/bin/ocrconfig -local -showbackup20、/etc/oracle目录下的配置文件备份源有集群三个节点都执行 tar -zcvf /etc/oracle.tar.gz /etc/oracle21、备份数据库GI集群、DB数据库软件这个步骤强烈建议执行22、cluvfy验证添加节点的条件是否满足节点一完成以grid用户执行 su - grid cluvfy comp peer -n hostrac1,hostrac2,hostrac3,hostrac4 -verbose cluvfy stage -pre nodeadd -n hostrac4 -verbose正式添加节点1、添加GI集群节点节点一grid用户执行su - grid export IGNORE_PREADDNODE_CHECKSY $ORACLE_HOME/oui/bin/addNode.sh -silent CLUSTER_NEW_NODES{hostrac4} CLUSTER_NEW_VIRTUAL_HOSTNAMES{hostrac4-vip} CLUSTER_NEW_PRIVATE_NODE_NAMES{hostrac4-priv} 最后提示在新节点节点四root用户执行root.sh两个脚本。这个提示只需要执行一个脚本 /u01/app/crs/root.sh2、检查集群状态可以每个节点都执行检查crsctl stat res -t3、添加DB节点节点一执行以oracle用户执行添加节点到Oracle RAC数据库软件 su - oracle export IGNORE_PREADDNODE_CHECKSY $ORACLE_HOME/oui/bin/addNode.sh -silent CLUSTER_NEW_NODES{hostrac4} 最后提示在新节点节点四root用户执行root.sh两个脚本 /u01/app/oracle/product/11.2/db_1/root.sh4.添加数据库实例节点一执行oracle用户查看数据库配置节点一执行 su - oracle dbca -silent -addInstance -nodeList hostrac4 -gdbName orcl -instanceName orcl4 -sysDBAUserName sys -sysDBAPassword oracle 实例添加后验证: srvctl config database -d orcl|grep Database instances Database instances: orcl1,orcl2,orcl3,orcl45、为新增的节点四数据库实例添加undo及redo如果用使用DBCA可以自动完成创建Undo表空间、Redo日志及配置初始化参数等工作在节点一上执行su - oracle sqlplus /as sysdba CREATE UNDO TABLESPACE UNDOTBS4 DATAFILE DATA/orcl/datafile/undotbs04.dbf SIZE 50M AUTOEXTEND ON; ALTER DATABASE ADD LOGFILE THREAD 4 GROUP 7 (DATA/orcl/redo07.log) SIZE 50M; ALTER DATABASE ADD LOGFILE THREAD 4 GROUP 8 (DATA/orcl/redo08.log) SIZE 50M; ALTER DATABASE enable THREAD 4; 备注如果当时删除节点时上面的redo、undo都保留且日志线程enable则上面的无需执行6、启动新增的节点四数据库实例启动新的数库实例 srvctl start instance -d orcl -i orcl4 备注条件允许的情况下建议直接将数据库整体来一次重启操作 srvctl stop database -d orcl srvctl start database -d orcl7、最终验证[gridhostrac3 ~]$ crsctl stat res -t至此完成删除节点的重新加入五、总结1、卸载 GI 软件需要根据提示进行交互最后确认删除gi软件并且过程中会提示让root手工执行一个脚本最后还需要以root用户rm -rf方式删除几个残余的文件和目录 具体输出如下 [gridhostrac4 ~]$ $ORACLE_HOME/deinstall/deinstall -local Checking for required files and bootstrapping ... Please wait ... Location of logs /tmp/deinstall2026-06-18_11-59-36AM/logs/ ############ ORACLE DEINSTALL DECONFIG TOOL START ############ ######################### CHECK OPERATION START ######################### ## [START] Install check configuration ## Checking for existence of the Oracle home location /u01/app/crs Oracle Home type selected for deinstall is: Oracle Grid Infrastructure for a Cluster Oracle Base selected for deinstall is: /u01/app/grid Checking for existence of central inventory location /u01/app/oraInventory Checking for existence of the Oracle Grid Infrastructure home The following nodes are part of this cluster: hostrac4 Checking for sufficient temp space availability on node(s) : hostrac4 ## [END] Install check configuration ## Traces log file: /tmp/deinstall2026-06-18_11-59-36AM/logs//crsdc.log Enter an address or the name of the virtual IP used on node hostrac4[hostrac4-vip] /////这里直接回车即可 The following information can be collected by running /sbin/ifconfig -a on node hostrac4 Enter the IP netmask of Virtual IP 192.168.11.30 on node hostrac4[255.255.255.0] /////这里直接回车即可 Enter the network interface name on which the virtual IP address 192.168.11.30 is active /////这里直接回车即可 Enter an address or the name of the virtual IP[] /////这里直接回车即可 Network Configuration check config START Network de-configuration trace file location: /tmp/deinstall2026-06-18_11-59-36AM/logs/netdc_check2026-06-18_12-02-24-PM.log Specify all RAC listeners (do not include SCAN listener) that are to be de-configured [LISTENER]: /////这里直接回车即可 Network Configuration check config END Asm Check Configuration START ASM de-configuration trace file location: /tmp/deinstall2026-06-18_11-59-36AM/logs/asmcadc_check2026-06-18_12-02-53-PM.log ######################### CHECK OPERATION END ######################### ####################### CHECK OPERATION SUMMARY ####################### Oracle Grid Infrastructure Home is: The cluster node(s) on which the Oracle home deinstallation will be performed are:hostrac4 Since -local option has been specified, the Oracle home will be deinstalled only on the local node, hostrac4, and the global configuration will be removed. Oracle Home selected for deinstall is: /u01/app/crs Inventory Location where the Oracle home registered is: /u01/app/oraInventory Following RAC listener(s) will be de-configured: LISTENER Option -local will not modify any ASM configuration. Do you want to continue (y - yes, n - no)? [n]: y 这里输入y表示继续删除gi软件 A log of this session will be written to: /tmp/deinstall2026-06-18_11-59-36AM/logs/deinstall_deconfig2026-06-18_11-59-39-AM.out Any error messages from this session will be written to: /tmp/deinstall2026-06-18_11-59-36AM/logs/deinstall_deconfig2026-06-18_11-59-39-AM.err ######################## CLEAN OPERATION START ######################## ASM de-configuration trace file location: /tmp/deinstall2026-06-18_11-59-36AM/logs/asmcadc_clean2026-06-18_12-02-57-PM.log ASM Clean Configuration END Network Configuration clean config START Network de-configuration trace file location: /tmp/deinstall2026-06-18_11-59-36AM/logs/netdc_clean2026-06-18_12-02-57-PM.log De-configuring RAC listener(s): LISTENER De-configuring listener: LISTENER Stopping listener on node hostrac4: LISTENER Warning: Failed to stop listener. Listener may not be running. Listener de-configured successfully. De-configuring Naming Methods configuration file... Naming Methods configuration file de-configured successfully. De-configuring backup files... Backup files de-configured successfully. The network configuration has been cleaned up successfully. Network Configuration clean config END ---------------------------------------- The deconfig command below can be executed in parallel on all the remote nodes. Execute the command on the local node after the execution completes on all the remote nodes. Run the following command as the root user or the administrator on node hostrac4. /////这里根据提示root用户执行下面的脚本 /tmp/deinstall2026-06-18_11-59-36AM/perl/bin/perl -I/tmp/deinstall2026-06-18_11-59-36AM/perl/lib -I/tmp/deinstall2026-06-18_11-59-36AM/crs/install /tmp/deinstall2026-06-18_11-59-36AM/crs/install/rootcrs.pl -force -deconfig -paramfile /tmp/deinstall2026-06-18_11-59-36AM/response/deinstall_Ora11g_gridinfrahome1.rsp Press Enter after you finish running the above commands ---------------------------------------- Remove the directory: /tmp/deinstall2026-06-18_11-59-36AM on node: Setting the force flag to false Setting the force flag to cleanup the Oracle Base Oracle Universal Installer clean START Detach Oracle home /u01/app/crs from the central inventory on the local node : Done Delete directory /u01/app/crs on the local node : Done Delete directory /u01/app/oraInventory on the local node : Done Delete directory /u01/app/grid on the local node : Done Oracle Universal Installer cleanup was successful. Oracle Universal Installer clean END ## [START] Oracle install clean ## Clean install operation removing temporary directory /tmp/deinstall2026-06-18_11-59-36AM on node hostrac4 ## [END] Oracle install clean ## ######################### CLEAN OPERATION END ######################### ####################### CLEAN OPERATION SUMMARY ####################### Following RAC listener(s) were de-configured successfully: LISTENER Oracle Clusterware is stopped and successfully de-configured on node hostrac4 Oracle Clusterware is stopped and de-configured successfully. Successfully detached Oracle home /u01/app/crs from the central inventory on the local node. Successfully deleted directory /u01/app/crs on the local node. Successfully deleted directory /u01/app/oraInventory on the local node. Successfully deleted directory /u01/app/grid on the local node. Oracle Universal Installer cleanup was successful. Run rm -rf /etc/oraInst.loc as root on node(s) hostrac4 at the end of the session. Run rm -rf /opt/ORCLfmap as root on node(s) hostrac4 at the end of the session. Run rm -rf /etc/oratab as root on node(s) hostrac4 at the end of the session. 根据上面的提示在待删除的节点四root用户执行rm -rf文件删除 rm -rf /etc/oraInst.loc rm -rf /opt/ORCLfmap rm -rf /etc/oratab Oracle deinstall tool successfully cleaned up temporary directories. ####################################################################### ############# ORACLE DEINSTALL DECONFIG TOOL END ############# 让root用户执行的脚本输出如下 [roothostrac4 ~]# /tmp/deinstall2026-06-18_11-59-36AM/perl/bin/perl -I/tmp/deinstall2026-06-18_11-59-36AM/perl/lib -I/tmp/deinstall2026-06-18_11-59-36AM/crs/install /tmp/deinstall2026-06-18_11-59-36AM/crs/install/rootcrs.pl -force -deconfig -paramfile /tmp/deinstall2026-06-18_11-59-36AM/response/deinstall_Ora11g_gridinfrahome1.rsp Using configuration parameter file: /tmp/deinstall2026-06-18_11-59-36AM/response/deinstall_Ora11g_gridinfrahome1.rsp ****Unable to retrieve Oracle Clusterware home. Start Oracle Clusterware stack and try again. CRS-4047: No Oracle Clusterware components configured. CRS-4000: Command Stop failed, or completed with errors. ################################################################ # You must kill processes or reboot the system to properly # # cleanup the processes started by Oracle clusterware # ################################################################ Either /etc/oracle/olr.loc does not exist or is not readable Make sure the file exists and it has read and execute access Either /etc/oracle/olr.loc does not exist or is not readable Make sure the file exists and it has read and execute access Failure in execution (rc-1, 256, No such file or directory) for command /etc/init.d/ohasd deinstall error: package cvuqdisk is not installed Successfully deconfigured Oracle clusterware stack on this node