Thursday, March 31, 2016

How to Apply Critical Patch Update (CPU) on RAC

http://khalidali-oracledba.blogspot.com/2012/04/how-to-apply-critical-patch-update-cpu.html

How to Apply Critical Patch Update (CPU) on RAC

Applying CPU Patche "CPUJULY2011" for 10.2.0.4 Database on LINUX

Note: 
Before continue the following step please read the README file from the patch.
Critical patch update (CPU) patches are cumulative, which means fixes from previous Oracle security alerts and critical patch updates are included.
It is not required to have previous security patches applied before applying the CPUJul2011 patches. However, you must be on the stated patch set level for a given product home before applying the CPUJul2011 patches for that release.
The Critical Patch Update (CPU) patches are rolling up gradation,
Do one of the following, depending on whether this is a RAC environment:
If this is a RAC environment, choose one of the patch installation methods provided by OPatch (rolling, all node, or minimum downtime), and shutdown instances and listeners as appropriate for the installation method selected.
This CPU patch is rolling RAC installable, Please refer to My Oracle Support Note244241.1.
If this is not a RAC environment, shut down all instances and listeners associated with the Oracle home that you are updating.

Step by step CPU patches Applying:

If you are wish to applying rolling CPU patch on RAC, then the following steps must be following.
Rolling patch (no downtime)
-Shutdown the Oracle instance on node 1
-Apply the patch to the RAC home on node 1
-Start the Oracle instance on node 1
-Shutdown the Oracle instance on node 2
-Apply the patch to the RAC home on node 2
-Start the Oracle instance on node 2
-Shutdown the Oracle instance on node 3
-Apply the patch to the RAC home on node 3
-Start the Oracle instance on node 3
1.Download the CPU patch p12419249_10204_Linux-x86 from Metalink.
2.Change the owner of the patch file to oracle user.
# chown –R oracle: install p12419249_10204_Linux-x86.zip
3.Set the PATH variable to locate the opatch utility.
$ export PATH=$PATH: $ORACLE_HOME/OPatch
4.unzip the patch and go the unzipped directory
$unzip p12419249_10204_Linux-x86.zip
5.Fine the Opatch version
$ opatch version
Invoking OPatch 10.2.0.4.2
OPatch Version: 10.2.0.4.2
$ opatch lsinventory
Note: if you want check the CPU patch is whether rolling support or not, follow the steps.
-go to the patch directory
Cd /oracle/12419249
[oracle@rac1 12419249]$ opatch query -all
Invoking OPatch 10.2.0.4.2
Oracle Interim Patch Installer version 10.2.0.4.2
Copyright (c) 2007, Oracle Corporation.  All rights reserved.
Oracle Home                  : /oracle/product/10.2.0/rdbms
Central Inventory           : /oracle/product/10.2.0/oraInventory
from                             : /etc/oraInst.loc
OPatch version               : 10.2.0.4.2
OUI version                    : 10.2.0.4.0
OUI location                   : /oracle/product/10.2.0/rdbms/oui
Log file location              : /oracle/product/10.2.0/rdbms/cfgtoollogs/opatch/opatch2011-05-01_08-55-20AM.log
-------------------------------------------------------------------------------- 
Patch created on 20 May 2011, 03:02:21 hrs PST8PDT
Need to shutdown Oracle instances: false      (<--hear false mean we no need to down the database)
 Patch is roll-backable: true
 Patch is a rolling patch: true
 Patch has sql related actions: false
 Patch is an online patch: false
 Patch is a portal patch: false
 List of platforms supported:
 46: Linux Intel
 List of bugs to be fixed:
 8534387: CPUJUL2009 DATABASE 10.2.0.4
 8290506: CPUAPR2009 DATABASE 10.2.0.4
 7375644: MLR BUG FOR 10.2.0.4 FOR CPUOCT2008
 9352191: CPUAPR2010 DATABASE 10.2.0.4
 9655017: CPUJUL2010 DATABASE 10.2.0.4
 7150470: MLR BUG FOR 10.2.0.4 FOR CPUJUL2008
 7592346: CPUJAN2009 DATABASE 10.2.0.4
 9952272: CPUOCT2010 DATABASE 10.2.0.4
 9119226: CPUJAN2010 DATABASE 10.2.0.4
 11725015: CPUAPR2011 DATABASE 10.2.0.4
 12419249: CPUJUL2011 DATABASE 10.2.0.4
 8836308: CPUOCT2009 DATABASE 10.2.0.4
 10249540: CPUJAN2011 DATABASE 10.2.0.4
 List of optional components:
 oracle.rdbms.rsf  :  10.2.0.4.0
 oracle.rdbms       :  10.2.0.4.0
6.Backup the oraInventory  and Opatch directory
$cp -R oraInventory old_oraInventory
$cp -R opatch old_opatch
7.If you are Applying on RAC, follow the below steps:
Shut down the instance one of node
$ srvctl stop instance -d racdb –i racdb1
Shut down the ASM instanc respected node
$ srvctl stop asm -n rac1
Shut down all Nodeapps services of the node
$ srvctl stop ndoeapps -n rac1
8.Go to the Patch Directory and invoke opatch apply.
$ cd 12419249
$opatch apply or $opatch napply -skip_subset -skip_duplicate
9.Verify Patches are applied
$opatch lsinventory -detail -oh $ORACLE_HOME
10.Now start the Node1 and repeat the same 1 to 10 steps on Node2
$ srvctl start nodeapps –n rac1
$srvctl start asm –n rac1
$srvctl start instance –d racdb –i racdb1
Note: if the database on rac1 located, now relocate to node2
crs_relocate ora.racdb.db
11.Now stop Instance,asm and nodeapps on node2
$ srvctl stop instance –d racdb –i racdb2
$ srvctl stop asm –n rac2
$ srvctl stop nodeapps –n rac2
12.Go to the Patch Direcotry and invoke the opatch apply on node2
$ cd 12419249
$ opatch apply or opatch napply -skip_subset -skip_duplicate 
     13.Verify Patches are applied 
          $opatch lsinventory -detail -oh $ORACLE_HOME
     14.Start  the Instance,Asm and Nodeapps on node2
$srvctl start instance –d racdb –i racdb2
$srvctl start asm –n rac2
$srvctl start nodeapps –n rac2
$crs_stat –t 

Post CPU installation Steps:

For each database instance running on the Oracle home being patched, connect to the database using SQL*Plus on each node.
Connect as SYSDBA and run the catbundle.sql script as follows:
On node1 and node2:
cd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> STARTUP
SQL> @catbundle.sql cpu apply
SQL> @utlrp.sql
SQL> QUIT
The catbundle.sql execution is reflected in the dba_registry_history view by a row associated with bundle series CPU.
For information about the catbundle.sql script, see My Oracle Support Note 605795.1Introduction to Oracle Database catbundle.sql.
Check the following log files in $ORACLE_HOME/cfgtoollogs/catbundle for any errors:
catbundle_CPU__APPLY_.log
catbundle_CPU__GENERATE_.log
Recompiling Views in the Database
You may skip this section if you have recompiled views for this database during the installation of a previous CPU.
The time required to recompile the views and related objects depends on the total number of objects and on your system configuration. 
In one internal Oracle test with approximately 2000 views and 4000 objects, the total execution time for view_recompile_jan2008cpu.sql and utlrp.sql was about 30 minutes.
If you want to check whether view recompilation has already been performed for the database, execute the following statement.
SELECT * FROM registry$history where ID = '6452863';
If the view recompilation has been performed, this statement returns one or more rows. If the view recompilation has not been performed, this statement returns no rows. If no rows returns then go the following steps.
The following steps recompile the views in the database. For a RAC environment, perform these steps on only one node.
1. Run the pre-check script (so named because it was initially released in CPUJan2008), which reports the maximum number of views and objects that may be recompiled:
cd $ORACLE_HOME/cpu/view_recompile
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> @recompile_precheck_jan2008cpu.sql
SQL> QUIT
The purpose of this step is to help you determine whether view recompilation should be done at the same time as the CPU install, or scheduled later.
Note:
If the database is not in a RAC environment(if Single Instance), perform this step and skip the next step 2. (If the database is in a RAC environment, go to the next step2.)
Run the view recompilation script, note that this script is run with the database in upgrade mode, which restricts connections as SYSDBA.
cd $ORACLE_HOME/cpu/view_recompile
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> SHUTDOWN IMMEDIATE
SQL> STARTUP UPGRADE
SQL> @view_recompile_jan2008cpu.sql
SQL> SHUTDOWN;
SQL> STARTUP;
SQL> QUIT
2.If the database is in a RAC environment, run the view recompilation script as follows, note that this script is run with the database in upgrade mode, which restricts connections as SYSDBA. Stop all instances except the one where the view recompilation is being executed.
cd $ORACLE_HOME/cpu/view_recompile
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> STARTUP NOMOUNT
SQL> ALTER SYSTEM SET CLUSTER_DATABASE=FALSE SCOPE=spfile;
SQL> SHUTDOWN
SQL> STARTUP UPGRADE
SQL> @?/ cpu/view_recompile /view_recompile_jan2008cpu.sql
SQL> SHUTDOWN;
SQL> STARTUP NOMOUNT;
Set the CLUSTER_DATABASE initialization parameter to TRUE:
SQL> ALTER SYSTEM SET CLUSTER_DATABASE=TRUE SCOPE=spfile;
Restart the database:
SQL> QUIT
cd $CRS_HOME/bin
srvctl start database -d racdb
If any invalid objects were reported, run the utlrp.sql script as follows:
cd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> @utlrp.sql
Then, manually recompile any invalid objects. For example:
SQL> alter package schemaname.packagename compile;
4. Verify Patches are applied.
$opatch lsinventory -detail -oh $CRS_HOME #if you have CRS_HOME
$opatch lsinventory -detail -oh $ORACLE_HOME #if you have both ORACLE_HOME
  The CPU patch was successfully applied.

Wednesday, March 30, 2016

Patching: How to verify Oracle database patchset?

Patching: How to verify Oracle database patchset?


1. How to verify Oracle database patchset?
$ORACLE_HOME/OPatch/opatch lsinventory






2. How to find more details of patchset?
$ORACLE_HOME/OPatch/opatch lsinventory -details


3. How to check the version of all the components installed in DB?
sqlplus / as sysdba -- be sure to login as sys as dba_registry shows components owned by “current” user.

set linesize 200
set pagesize 200
col comp_name format a50
select comp_name,version,status from dba_registry;


4. How to get information about upgrades, downgrades, and critical patch updates that have been performed on the database?
set linesize 200
set pagesize 200
col action_time format a30
col action format a10
col namespace format a12
col version format a12
col bundle_series format a25
col comments format a20
select * from dba_registry_history;



5. Other helpful Oracle support Document IDs
No.
Description
Oracle Document ID
1
Quick Reference to Patch Numbers for Database PSU, SPU(CPU), Bundle Patches and Patchsets
1454618.1
2
Introduction to Oracle Recommended Patches
756388.1
3
Oracle Recommended Patches — Oracle Database
756671.1
4
Quick Reference to Patch Numbers for Database PSU, SPU(CPU), Bundle Patches and Patchsets
1454618.1
5
Release Schedule of Current Database Releases
742060.1

Applying CPUJan2011 Patch on 11.2.0.1.0/Linux(64 bit)

Applying CPUJan2011 Patch on 11.2.0.1.0/Linux(64 bit)





                                     Applying CPUJan2011 Patch on 11.2.0.1.0/Linux(64 bit)

1. Database version
SQL> select * from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
PL/SQL Release 11.2.0.1.0 - Production
CORE     11.2.0.1.0             Production
TNS for Linux: Version 11.2.0.1.0 - Production
NLSRTL Version 11.2.0.1.0 - Production


2. OS Version
[root@mydb Desktop]# uname -a
Linux mydb.mydomain.com 2.6.39-400.17.1.el6uek.x86_64 #1 SMP Fri Feb 22 18:16:18 PST 2013 x86_64 x86_64 x86_64 GNU/Linux
[root@mydb Desktop]#

3. Download CPUJan2012 patch for 11.2.0.1
 Patch No 10248516 from support.oracle.com

4. Opatch Version
To apply CPUJan2012, OPatch utility version 11.2.0.1.0 or later to apply this patch. Oracle recommends that you use
the latest released OPatch 11.2, which is available for download from My Oracle Support patch 6880880 by
selecting the 11.2.0.0.0 release

[oracle@mydb ~]$ export PATH=$ORACLE_HOME/OPatch:$PATH
[oracle@mydb ~]$ opatch lsinventory
Oracle Interim Patch Installer version 11.2.0.3.5
Copyright (c) 2013, Oracle Corporation.  All rights reserved.

Oracle Home       : /u01/app/oracle/product/11.2.0/db_1
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/oracle/product/11.2.0/db_1/oraInst.loc
OPatch version    : 11.2.0.3.5
OUI version       : 11.2.0.1.0
Log file location : /u01/app/oracle/product/11.2.0/db_1/cfgtoollogs/opatch/opatch2013-08-29_20-57-33PM_1.log

Lsinventory Output file location : /u01/app/oracle/product/11.2.0/db_1/cfgtoollogs/opatch/lsinv/lsinventory2013-08-29_20-57-33PM.txt
--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle Database 11g                                                  11.2.0.1.0
There are 1 products installed in this Oracle Home.
There are no Interim patches installed in this Oracle Home.
--------------------------------------------------------------------------------
OPatch succeeded.


5. Sessions Status
Check How Many sesion are ACTIVE, If any found Ask Application team to bring down all Applications/Processes.

SQL> select username,count(*) from v$session where username is not null group by username;
USERNAME                                        COUNT(*)
------------------------------ ----------
SYS                                                                         1


6. Invalid Objects
SQL>  select owner,object_type,count(*) from dba_objects where status='INVALID' group by owner,object_type order by owner,object_type;

OWNER                                                       OBJECT_TYPE                   COUNT(*)
------------------------------ ------------------- ----------
SCOTT                                          FUNCTION                       1
SCOTT                                          MATERIALIZED VIEW                   1
SCOTT                                          PACKAGE                                         2
SCOTT                                          PACKAGE BODY                             2
SCOTT                                          PROCEDURE                                    1


7. Status of Oracle Services
[oracle@mydb ~]$ ps -ef | grep smon
oracle   25478  5892  0 21:05 pts/0    00:00:00 grep smon
[oracle@mydb ~]$ ps -ef | grep tns
oracle   25482  5892  0 21:05 pts/0    00:00:00 grep tns

8. Take Cold Backup of Database & Backup of (ORACLE_HOME & Inventory)
--Preferred to have a user-managed backup

9. Apply Opatch
[oracle@mydb ~]$ export PATH=$ORACLE_HOME/OPatch:$PATH
[oracle@mydb 10248516]$ opatch apply
Oracle Interim Patch Installer version 11.2.0.3.5
Copyright (c) 2013, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/oracle/product/11.2.0/db_1
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/oracle/product/11.2.0/db_1/oraInst.loc
OPatch version    : 11.2.0.3.5
OUI version       : 11.2.0.1.0
Log file location : /u01/app/oracle/product/11.2.0/db_1/cfgtoollogs/opatch/10248516_Aug_29_2013_21_13_26/apply2013-08-29_21-13-25PM_1.log

Applying interim patch '10248516' to OH '/u01/app/oracle/product/11.2.0/db_1'
Verifying environment and performing prerequisite checks...
All checks passed.
Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name: tyagi.ankur07@gmail.com
Provide your My Oracle Support password to receive security updates via your My Oracle Support account.
Password (optional):          

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/u01/app/oracle/product/11.2.0/db_1')

Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files...
Patching component oracle.rdbms.rsf, 11.2.0.1.0...

Patching component oracle.rdbms.dbscripts, 11.2.0.1.0...

Patching component oracle.rdbms, 11.2.0.1.0...

Patching component oracle.oraolap, 11.2.0.1.0...

Patching component oracle.rdbms.deconfig, 11.2.0.1.0...

Patching component oracle.javavm.server, 11.2.0.1.0...

Patching component oracle.precomp.common, 11.2.0.1.0...

Patching component oracle.network.rsf, 11.2.0.1.0...

Patching component oracle.network.listener, 11.2.0.1.0...

Patching component oracle.rdbms.dv.oc4j, 11.2.0.1.0...

Patching component oracle.sdo.locator, 11.2.0.1.0...

Patching component oracle.sysman.console.db, 11.2.0.1.0...

OPatch found the word "warning" in the stderr of the make command.
Please look at this stderr. You can re-run this make command.
Stderr output:
ins_precomp.mk:19: warning: overriding commands for target `pcscfg.cfg'
/u01/app/oracle/product/11.2.0/db_1/precomp/lib/env_precomp.mk:2091: warning: ignoring old commands for target `pcscfg.cfg'
/u01/app/oracle/product/11.2.0/db_1/precomp/lib/ins_precomp.mk:19: warning: overriding commands for target `pcscfg.cfg'
/u01/app/oracle/product/11.2.0/db_1/precomp/lib/env_precomp.mk:2091: warning: ignoring old commands for target `pcscfg.cfg'



Verifying the update...
Patch 10248516 successfully applied
OPatch Session completed with warnings.
Log file location: /u01/app/oracle/product/11.2.0/db_1/cfgtoollogs/opatch/10248516_Aug_29_2013_21_13_26/apply2013-08-29_21-13-25PM_1.log

OPatch completed with warnings.

Note:- As per Metalink ID:  Opatch warning: overriding commands for target xxxx (Doc ID 1448337.1), we can safely ignore these warnings.


10. Post Installation
Database instance running on the Oracle home being patched, connect to the database using SQL*Plus using SYSDBA and run the catbundle.sql script as follows:

[oracle@mydb ~]$ cd /u01/app/oracle/product/11.2.0/db_1/rdbms/admin/
[oracle@mydb admin]$ ls catbundle.sql
catbundle.sql
[oracle@mydb admin]$ sqlplus '/as sysdba'
SQL*Plus: Release 11.2.0.1.0 Production on Thu Aug 29 21:29:07 2013
Copyright (c) 1982, 2009, Oracle.  All rights reserved.
Connected to an idle instance.

SQL> startup
ORACLE instance started.
Total System Global Area  634679296 bytes
Fixed Size                                2216024 bytes
Variable Size                        486543272 bytes
Database Buffers               142606336 bytes
Redo Buffers                         3313664 bytes
Database mounted.
Database opened.

SQL> @catbundle.sql psu apply


11. Check the status from registry$history
SQL> select action, version, id, comments, bundle_series from registry$history where bundle_series like '%PSU%';
ACTION          VERSION                 ID COMMENTS        BUNDLE_SERIES
--------------- --------------- ---------- --------------- ---------------
APPLY           11.2.0.1                 4 PSU 11.2.0.1.4  PSU


12. Compile Invalid objects by executing “utlrp.sql”.
[oracle@mydb admin]$ ls utlrp*
utlrp.sql
[oracle@mydb admin]$ sqlplus '/as sysdba'

SQL*Plus: Release 11.2.0.1.0 Production on Thu Aug 29 21:39:48 2013

Copyright (c) 1982, 2009, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> @utlrp.sql

Now, check invalid objects after patching and recompile

SQL> select owner,object_type,count(*) from dba_objects where status='INVALID' group by owner,object_type order by owner,object_type;

OWNER                                                       OBJECT_TYPE                   COUNT(*)
------------------------------ ------------------- ----------
SCOTT                                          PACKAGE                                         1
SCOTT                                          PACKAGE BODY                             1


13. OPatch status
[oracle@mydb ~]$ opatch lsinventory|grep 10248516
Patch  10248516     : applied on Thu Aug 29 21:20:48 IST 2013
     8588519, 8783738, 8834425, 9454385, 8856497, 8890026, 8721315, 10248516