OS-Upgrade/-Patching of an Oracle Cluster

2021-05-09 Off By Markus Flechtner

Patching the operating system of your database servers should be regular task of your system administrators. Security issues need to be fixed etc. In the context of database servers, especially database servers running oracle Grid Infrastructure, additional should be considered after patching or upgrading the operating system.

When you upgrade or patch your OS, relinking of the Oracle executables is recommended. For a system without Grid Infrastructure, this is easy:

  1. Stop database instances and listener
  2. Relink the Oracle Software
  3. Start listener and  database instances

In a DataGuard environment a switchover to the standby system is a good idea before relinking the primary system, but I’ll leave such a configuration aside in the context of this post.

In an environment with Oracle Grid Infrastructure in place, either a cluster or Oracle Restart, some more steps are required.

Here’s my preferred way:

Step 1: Disable Autostart of the Grid Infrastructure

Personally, I prefer to reboot a server after an OS upgrade, it may not always be required, but I consider it helpful.

In the first step, I disable the automatic start of the Grid Infrastructure after a reboot.

grid@tick:~/ [+ASM1] su

grid@tick:~/ [+ASM1] su
Password:
root@tick:/home/grid/ [+ASM1] crsctl disable crs
CRS-4621: Oracle High Availability Services autostart is disabled.

root@tick:/home/grid/ [+ASM1] cat /etc/oracle/scls_scr/tick/root/ohasdstr
disable

root@tick:/home/grid/ [+ASM1] exit
exit

Step 2: Upgrade/Patch the OS

The next step is to upgrade the OS. In my case, it’s a simple “yum upgrade -y”:

root@tick:/home/grid/ [+ASM1] yum upgrade -y
Loaded plugins: langpacks, ulninfo
[..]
---> Package kernel-uek.x86_64 0:4.1.12-124.50.2.el7uek will be installed
---> Package kernel-uek-devel.x86_64 0:4.1.12-124.50.2.el7uek will be installed
---> Package kernel-uek-firmware.noarch 0:4.1.12-124.50.2.el7uek will be installed
[…]

As you can see, a kernel upgrade is included in this upgrade.

Step 3: Reboot the server

Rebooting the server is a simple “shutdown -r now”:

root@tick:/home/grid/ [+ASM1] shutdown -r now
Connection to tick closed by remote host.
Connection to tick closed.

Step 4: Relink the Oracle Software

As the automatic start of the Grid Infrastructure is disabled, all Oracle components are down after the reboot and you can relink the Oracle software.

Relink Grid Infrastructure

As most of the files of the GI are owned by root after the installation, this has to be reverted to the owner of the GI software (in my case “grid”) before relinking:

grid@tick:~/ [+ASM1] echo $ORACLE_HOME
/u00/app/grid/product/19.0.0.0
grid@tick:~/ [+ASM1] su
Password:
root@tick:/home/grid/ [+ASM1] $ORACLE_HOME/crs/install/rootcrs.sh -unlock
Using configuration parameter file: /u00/app/grid/product/19.0.0.0/crs/install/crsconfig_params
The log of current session can be found at:
  /u00/app/oracle/crsdata/tick/crsconfig/crsunlock_tick_2021-05-09_12-34-33AM.log
2021/05/09 12:34:34 CLSRSC-347: Successfully unlock /u00/app/grid/product/19.0.0.0

Of course, it’s a good idea to check the log file afterwards.

Now you can relink the Grid Infrastructure software as the GI owner:

grid@tick:~/ [+ASM1] cd $ORACLE_HOME/bin
grid@tick:/u00/app/grid/product/19.0.0.0/bin/ [+ASM1] ./relink all

After relinking, you have to change the ownership of the files back to root:

root@tick:/home/grid/ [+ASM1] $ORACLE_HOME/rdbms/install/rootadd_rdbms.sh

root@tick:/home/grid/ [+ASM1] $ORACLE_HOME/crs/install/rootcrs.sh -lock
Using configuration parameter file: /u00/app/grid/product/19.0.0.0/crs/install/crsconfig_params
The log of current session can be found at:
  /u00/app/oracle/crsdata/tick/crsconfig/crslock_tick_2021-05-09_12-44-22AM.log
2021/05/09 12:44:25 CLSRSC-329: Replacing Clusterware entries in file 'oracle-ohasd.service'

Relink RDBMS Software

Do not forget to relink the software in all your RDBMS homes on the cluster:

grid@tick:~/ [+ASM1] su - oracle
Password:

oracle@tick:~/ [rdbms19000] echo $ORACLE_HOME
/u00/app/oracle/product/19.0.0.0

oracle@tick:~/ [rdbms19000] $ORACLE_HOME/bin/relink all
writing relink log to: /u00/app/oracle/product/19.0.0.0/install/relinkActions2021-05-09_12-46-39PM.log
oracle@tick:~/ [rdbms19000] exit
logout

Please check the oracle executable after relinking the software. Typically, the group should be “asmadmin” to enable the RDBMS software to access the ASM disks. If it’s still “oinstall”, you have to change it as root. For newer versions of the RDBMS software (at least 18c and 19c), this is done automatically, but especially for 11.2 software, you have to do it manually (I must admit, I did not test it for 12.1 and 12.2).

Oracle provides a script “setasmgid” for this purpose:

oracle@tick:~/ [rdbms19] ls -al /u00/app/oracle/product/11.2.0.4/bin/oracle
-rwsr-s--x 1 oracle oinstall 448599912 May 19 20:20 /u00/app/oracle/product/11.2.0.4/bin/oracle
oracle@tick:~/ [rdbms19]

oracle@tick:~/ [rdbms19] su
Password:
root@tick:/home/oracle/ [rdbms19] cd /etc/oracle
root@tick:/etc/oracle/ [rdbms19] ls -al
total 4124
drwxr-xr-x    6 root oinstall     151 May 19 12:56 .
drwxr-xr-x. 145 root root        8192 Jun 11 14:03 ..
drwxrwx---    2 root oinstall      56 May 19 13:02 lastgasp
drwxrwxrwt    2 root oinstall       6 May 19 12:55 maps
-rw-r--r--    1 root oinstall     163 May 19 13:01 ocr.loc
-rw-r--r--    1 root root           0 May 19 12:56 ocr.loc.orig
-rw-r--r--    1 root oinstall      93 May 19 12:56 olr.loc
-rw-r--r--    1 root root           0 May 19 12:56 olr.loc.orig
drwxrwxr-x    5 root oinstall      44 May 19 12:55 oprocd
drwxr-xr-x    3 root oinstall      18 May 19 12:55 scls_scr
-rws--x---    1 root oinstall 4199656 May 19 12:55 setasmgid
root@tick:/etc/oracle/ [rdbms19]

root@tick:/etc/oracle/ [rdbms19] ./setasmgid oracle_binary_path=/u00/app/oracle/product/11.2.0.4/bin/oracle
root@tick:/etc/oracle/ [rdbms19] exit
exit
oracle@tick:~/ [rdbms19] ls -al /u00/app/oracle/product/11.2.0.4/bin/oracle
-rwsr-s--x 1 oracle asmadmin 448599912 May 19 20:20 /u00/app/oracle/product/11.2.0.4/bin/oracle

Step 5: Re-enable Autostart of the Grid Infrastructure

grid@tick:~/ [+ASM1] su
Password:
root@tick:/home/grid/ [+ASM1] echo $ORACLE_HOME
/u00/app/grid/product/19.0.0.0
root@tick:/home/grid/ [+ASM1] crsctl enable crs
CRS-4622: Oracle High Availability Services autostart is enabled.
root@tick:/home/grid/ [+ASM1] cat /etc/oracle/scls_scr/tick/root/ohasdstr
enable

Step 6: Restart of the  Grid Infrastructure

root@tick:/home/grid/ [+ASM1] crsctl start crs
CRS-4123: Oracle High Availability Services has been started.

Step 7: Check the status of the Oracle components on the node

It will take some time for the Oracle stack (GI + RDBMS) to start and then you can check the status of your node with the tool “crsstat”:

grid@tick:~/ [+ASM1] crsctl status resource -t

That’s it. It’s not really difficult, but there are some more steps to consider than in a simple RDBMS only environment.

Hope this help.

MOS-Notes:

  • Relinking Oracle Home FAQ ( Frequently Asked Questions) (Doc ID 1467060.1)
  • RAC: Frequently Asked Questions (RAC FAQ) (Doc ID 220970.1)
  • How To Relink The Oracle Grid Infrastructure Standalone (Restart) Installation Or Oracle Grid Infrastructure RAC/Cluster Installation (11.2 to 18.c). (Doc ID 1536057.1)

Amazon Partner-Link

CategoryCommon
TagsCluster Clusterware OS-Upgrade RAC Real Application Cluster