oracledba.help
Install

19c Combo OJVM-GI-RU

TOC

Overview

The following covers applying the OJVM-GI Release Update (RU) combo patch to a RAC database. Patch 33248471 is used in these examples (AKA October 2021).

This document is only intended for where the GI Home and the Database Homes are not shared and ACFS file system is not configured.

🠊 If you have a new unpatched 19c RAC perform this workaround before patching. 🠈

Combo Patch Fun Facts!

For both the Database RU and the OJVM RU

  • opatchauto brings down the services, applys the patch (and subpatches) then restarts the services on the node.
  • opatchauto only patches the individual node.
    So you have to run it on all nodes.
  • opatchauto will patch both the GRID_HOME and ORACLE_HOME on the node.
    So you don't have to explicitly execute it on the GI home then the DB home.
  • opatchauto updates the SQL in the database.
    So you don't need to run datapatch -verbose afterward.
  • The GI RU opatchauto must be run from the GI Home as the root user.

OJVM RU

  • OJVM must be applied as a Nonrolling Patch unless specified otherwise.
  • The OJVM must be run from the DB Home as the root user.
  • To implement the OJVM PSU opatchauto apply is run on all nodes but the datapatch operation (AKA Load Modified SQL Files) is just run once from one node.

While it can vary, plan on each node taking 60-90 minutes.

Prerequisites

  • If using VMware or VirtualBox, take a snapshot before beginning.
  • Ensure you have the latest OPatch.
  • Confirm the Oracle inventory is OK.
    $ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME
  • Download Combo patch (Doc ID 2118136.2):
    1. What would you like to download? OJVM Update/PSU/Bundle Patches
    2. Please select the version: 19.0.0.0
    3. Download: OJVM + GI Update (33248471)
    4. As root user copy patch .zip file to your patches dir.
      • cp /media/sf_sw/Oracle/patches/19/2021-Oct/p33248471_190000_Linux-x86-64.zip /u01/orasw/patches/
      • chown grid:oinstall /u01/orasw/patches/*
  • Run My Oracle Support's Conflict Checker to detect and resolve any conflicts.
  • Unzip the downloaded combo patch zip file.

su - grid
cd /u01/orasw/patches
unzip p33248471_190000_Linux-x86-64.zip

A sub directory with the matching patch number (33248471) will be created.
  • List included major patch sets.

cd /u01/orasw/patches/33248471
ls -l
33182768 (GI RU)
33192694 (OJVM RU)

There is a README.html in each top-level sub dir that identifies type (GI or OJVM).

Corresponding README.html Example Entry:

  • <title>Oracle Database Patch 33182768 - GI Release Update 19.nnn</title>
  • <title>Oracle Database Patch 33192694 - Oracle JavaVM Component Release Update 19.nnn</title>

One of the biggest reasons patches fail is lack of disk space in your ORACLE_HOME. Ensure you have 25gb or more free.

GI RU

Perform this on each node before doing OJVM RU. 

 su -
 If Linux 8 run: export CV_ASSUME_DISTID=OEL8.1
 export PATH=$PATH:/u01/app/19.3.0.0.0/grid/OPatch
 cd /u01/app/19.3.0.0.0/grid/OPatch 
 ./opatchauto apply /u01/orasw/patches/33248471/33182768

 Session Log
 /u01/app/19.3.0.0.0/grid/cfgtoollogs/opatchauto/opatchauto<DateTime>.log
 Current Cmds Log
 /u01/app/19.3.0.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch<DateTime>_1.log

If this RAC created with only 1 node use the -nonrolling option: ./opatchauto apply /<PathToPatchNumber> -nonrolling.

🠊 Apply above GI RU on all nodes before continuing.

Check dba_registry

 COL version     FORMAT a10
 COL action      FORMAT a10
 COL status      FORMAT a10
 COL action_time FORMAT a30
 COL description FORMAT a65
 SELECT patch_id,patch_type,action,status,action_time,description FROM dba_registry_sqlpatch;

  PATCH_ID    VERSION     ACTION  STATUS   ACTION_TIME                    DESCRIPTION 
  ----------  ----------  ------  -------  -----------------------------  -------------------------------------------------------------- 
  33182768    19.x    APPLY   SUCCESS  DD-MON-YY nn.nn.nn.nnnnnn AA   DATABASE PATCH SET UPDATE 19.x.nnnnnnn

OJVM RU

  • After the GI RU completed for all nodes, open a new console to apply the OJVM RU from node 1 then other nodes.
  • This will bring down DB instance, apply OJVM PSU, then restart DB instance. 
 su -
 If Linux 8 run: export CV_ASSUME_DISTID=OEL8.1
 export ORACLE_HOME=/u01/app/oracle/product/19.3.0.0.0/dbhome_1
 export PATH=$ORACLE_HOME/bin:$PATH
 export PATH=$PATH:/usr/ccs/bin
 export PATH=$PATH:$ORACLE_HOME/OPatch
 cd $ORACLE_HOME/OPatch
 ./opatchauto apply /u01/orasw/patches/33248471/33192694 -oh $ORACLE_HOME -nonrolling
  • OJVM Log Path: $ORACLE_HOME/cfgtoollogs/opatchautodb/systemconfig<DateTime>.log
  • For Oracle-Restart run the the OJVM without the -nonrolling option.

Load Modified SQL Files Into the Database

  • Perform this on just node 1 once. Do not run on other nodes.
  • Ensure ORACLE_SID set for instance. 
 a. su - oracle

 b. Create pfile From spfile
    oracle> sqlplus / as sysdba
    SQL> create pfile from spfile;
    oracle> ls -l $ORACLE_HOME/dbs

 c. Stop DB
    OS> srvctl stop database -d oradb

 d. Startup Upgrade
    oracle> sqlplus / as sysdba
    SQL> startup
    SQL> alter system set cluster_database=false scope=spfile;
    SQL> exit

    oracle> srvctl stop database -d oradb

    oracle> sqlplus / as sysdba
    SQL> startup upgrade;
    SQL> exit

 e. Run Datapatch 
    oracle> cd $ORACLE_HOME/OPatch
    oracle> ./datapatch -verbose

 f. Start DB Back to Normal Mode
    oracle> sqlplus / as sysdba
    SQL> alter system set cluster_database=true scope=spfile;
    SQL> shutdown
    SQL> exit

    oracle> srvctl start database -d oradb

 g. Test
    SELECT instance_name,status,database_status,active_state 
    FROM gv$instance ORDER BY instance_name;

    INSTANCE_NAME    STATUS       DATABASE_STATUS   ACTIVE_ST
    ---------------- ------------ ----------------- ---------
    oradb1	     OPEN	  ACTIVE             NORMAL
    oradb2	     OPEN	  ACTIVE             NORMAL
  • Make sure when you test that the database is not still in UPGRADE mode (AKA OPEN MIGRATE).
  • OJVM checks.

Post Combo Patch Actions

  • Ensure Oracle Inventory for the oracle home shows applied patches.
    OS> $ORACLE_HOME/OPatch/opatch lsinventory|more
  • Perform database status checks. 
   SELECT instance_name,status,database_status,active_state FROM gv$instance ORDER BY instance_name;
    INSTANCE_NAME    STATUS       DATABASE_STATUS   ACTIVE_ST 
    ---------------- ------------ ----------------- --------- 
    mydb1            OPEN         ACTIVE            NORMAL    
    mydb2            OPEN         ACTIVE            NORMAL
  • Check dba_registry 
 COL version     FORMAT a10
 COL action      FORMAT a10
 COL status      FORMAT a10
 COL action_time FORMAT a30
 COL description FORMAT a65
 SELECT patch_id,patch_type,action,status,action_time,description FROM dba_registry_sqlpatch;

   PATCH_ID PATCH_TYPE ACTION     STATUS     ACTION_TIME                    DESCRIPTION 
  --------- ---------- ---------- ---------- ------------------------------ ---------------------------------------------------- 
   29517242 RU         APPLY      SUCCESS    31-JUL-19 02.55.38.683597 PM   Database Release Update : 19.3.0.0.190416 (29517242) 
   29834717 RU         APPLY      SUCCESS    02-AUG-19 10.39.07.471379 AM   Database Release Update : 19.4.0.0.190716 (29834717) 
   33192694 INTERIM   APPLY      SUCCESS    dd-MON-YY hh.mm.ss.nnnnnn AA  OJVM RELEASE UPDATE: 19.4.0.0.190716 (29774421)

References

This page has been derived from various MOS resources including: Doc ID 2285040.1, 3-20637596581, 3-16179254071, 3-16515328131, 3-17593686831, 3-18616506451.

Rollback Example Session

opatchauto rollback <UNZIPPED_PATCH_LOCATION>/<PATCH_NUMBER> [-nonrolling] 

 -- Get Sub Patches
 cd /u01/orasw/patches/33248471 
 ls -l 
 33182768 (GI RU) 
 33192694 (OJVM RU)

 -- Rollback: RU
 export ORACLE_HOME=/u01/app/oracle/product/19.3.0.0.0/dbhome_1
 export PATH=$ORACLE_HOME/bin:$PATH
 export PATH=$PATH:$ORACLE_HOME/OPatch
 cd $ORACLE_HOME/OPatch
 ./opatchauto rollback /u01/orasw/patches/33248471/33182768 -oh $ORACLE_HOME

 -- Rollback: OJVM
 export ORACLE_HOME=/u01/app/oracle/product/19.3.0.0.0/dbhome_1
 export PATH=$ORACLE_HOME/bin:$PATH
 export PATH=$PATH:$ORACLE_HOME/OPatch
 cd $ORACLE_HOME/OPatch
 ./opatchauto rollback /u01/orasw/patches/33248471/33192694 -oh $ORACLE_HOME -nonrolling

 -- QC
 $ORACLE_HOME/OPatch/opatch lsinventory

Common Errors

opatchauto apply Results java.io.FileNotFoundException: <oraInventory>/ContentsXML/oui-patch.xml (Permission denied) Error in Non-OUI Nodes

Per Doc ID 2582139.1

Background

During 19.x Grid Infrastructure (GI) installation, the file 'oui-patch.xml' will be created under the central inventory directory on the OUI node (node where gridSetup.sh was invoked) but not on the remote nodes.

🠊 This is a bug. It WILL effect you unless you do the workaround. 🠈

Workaround

Workaround is to have the OUI-PATCH.XML file copied in the central inventory of Non-OUI node from the OUI node with permission for GRID_HOME owner and re-run opatchauto apply on Non-OUI node.

Example Session

Node 1 (as root) 

 cp /u01/app/oraInventory/ContentsXML/oui-patch.xml /media/sf_sw/xfer/

Change path (/media/sf_sw/xfer) to match your environment.

Node 2 (as root) 

  cp /media/sf_sw/xfer/oui-patch.xml /u01/app/oraInventory/ContentsXML
  chown grid:oinstall /u01/app/oraInventory/ContentsXML/oui-patch.xml
  chmod 660 /u01/app/oraInventory/ContentsXML/oui-patch.xml
  ls -la /u01/app/oraInventory/ContentsXML/oui-patch.xml

[WARNING] The database instance 'oradb1' from '/u01/.../dbhome_1', in host 'lnx01' is not running.

Occurs after running: opatchauto apply on node. 

[WARNING] The database instance 'oradb1' from '/u01/...dbhome_1',
in host'lnx01' is not running. SQL changes, if any,  will not be applied.
To apply. the SQL changes, bring up the database instance and run the 
command manually from any one node (run as oracle).

Refer to the readme to get the correct steps for applying the sql changes.

OPatchauto session completed at Thu Oct 21 14:40:08 2021
Time taken to complete the session 53 minutes, 33 seconds

Workaround Session

su -
srvctl status database -d oradb
srvctl start instance -d oradb -i oradb1
srvctl status database -d oradb

su - oracle
cd $ORACLE_HOME/OPatch
oracle@lnx01 OPatch]$ ./datapatch -verbose

SQL Patching tool version 19.13.0.0.0 Production on Thu Oct 21 14:46:23 2021
Copyright (c) 2012, 2021, Oracle.  All rights reserved.

Log file for this invocation: /u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_83566_2021_10_21_14_46_23/sqlpatch_invocation.log

Connecting to database...OK
Gathering database info...done
Bootstrapping registry and package to current versions...done
Determining current state...done

Current state of interim SQL patches:
Interim patch 32876380 (OJVM RELEASE UPDATE: 19.12.0.0.210720 (32876380)):
  Binary registry: Installed
  SQL registry: Applied successfully on 29-SEP-21 08.08.28.516824 AM

Current state of release update SQL patches:
  Binary registry:
    19.13.0.0.0 Release_Update 211004165050: Installed
  SQL registry:
    Applied 19.12.0.0.0 Release_Update 210716141810 successfully on 28-SEP-21 04.33.45.384075 PM

Adding patches to installation queue and performing prereq checks...done
Installation queue:
  No interim patches need to be rolled back
  Patch 33192793 (Database Release Update : 19.13.0.0.211019 (33192793)):
    Apply from 19.12.0.0.0 Release_Update 210716141810 to 19.13.0.0.0 Release_Update 211004165050
  No interim patches need to be applied

Installing patches...
Patch installation complete.  Total patches installed: 1

Validating logfiles...done
Patch 33192793 apply: SUCCESS
  logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/33192793/24462514/33192793_apply_ORADB_2021Oct21_14_47_48.log (no errors)
SQL Patching tool complete on Thu Oct 21 14:51:47 2021

Patch Summary Example

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:lnx01
RAC Home:/u01/app/oracle/product/19.3.0.0.0/dbhome_1
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /u01/orasw/patches/32900083/32895426/32915586
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /u01/orasw/patches/32900083/32895426/32585572
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /u01/orasw/patches/32900083/32895426/32918050
Reason: This patch is not applicable to this specified target type - "rac_database"


==Following patches were SUCCESSFULLY applied:

Patch: /u01/orasw/patches/32900083/32895426/32904851
Log: /u01/app/oracle/product/19.3.0.0.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2021-09-28_14-33-37PM_1.log

Patch: /u01/orasw/patches/32900083/32895426/32916816
Log: /u01/app/oracle/product/19.3.0.0.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2021-09-28_14-33-37PM_1.log


Host:lnx01
CRS Home:/u01/app/19.3.0.0.0/grid
Version:19.0.0.0.0
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /u01/orasw/patches/32900083/32895426/32585572
Log: /u01/app/19.3.0.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-09-28_14-44-54PM_1.log

Patch: /u01/orasw/patches/32900083/32895426/32904851
Log: /u01/app/19.3.0.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-09-28_14-44-54PM_1.log

Patch: /u01/orasw/patches/32900083/32895426/32915586
Log: /u01/app/19.3.0.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-09-28_14-44-54PM_1.log

Patch: /u01/orasw/patches/32900083/32895426/32916816
Log: /u01/app/19.3.0.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-09-28_14-44-54PM_1.log

Patch: /u01/orasw/patches/32900083/32895426/32918050
Log: /u01/app/19.3.0.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-09-28_14-44-54PM_1.log


OPatchauto session completed at Tue Sep 28 15:06:59 2021
Time taken to complete the session 40 minutes, 34 seconds