Tuesday, March 2, 2010

cross platform (AIX to HP) single instance database to RAC using RCONFIG tool

converting a cross platform (AIX to HP) single instance database to 2 node RAC using the RCONFIG tool.


If you were following my blog, you might aware that we have a plan of implementing 16 node production RAC on HP UX Superdom Itanium servers using EMC DMX4 SAN storage.
As part of production implementation, last week, I have configured initial 8 node RAC successfully and the immediate plans were to migrate the databases across operating systems (AIX to HP ux) and convert a single instance database to RAC. I have done this exercise on the development setup sucessfully, the only change this time is that I have used RCONFIG utility to convert single instance database to RAC. Earlier I have used DBCA method to do the same.

Oracle provides following methods to convert a single instance database to RAC:

Grid Control
DBCA
Manual
RCONFIG(from 10gR2)

I must say, each DBA may choose a different method according to his/her convineance to achieve the result. The following illustrates how I have migrated a database from AIX to HP UX and from a single instance to RAC (on 2 node).

Database and OS details

Source:

Database 10gR2 (10.2.0.4)
OS : AIX
non-ASM

Target:

Databae 10gR2 (10.2.0.4)
OS : HP UX Superdom Itaninum
ASM

RCONFIG brief
RCONFIG tool is introuced in Oracle 10gR2 and the main functionality of this tool is to convert single instance database to RAC.
The tool(rconfig) can be found under $ORACLE_HOME/bin directry.
The xml input file (ConvertToRAC.xml) resides under $ORACLE_HOME/assistants/rconfig/sampleXMLs (it is recommended to copy the file before using it).
Log files(rconfig.log and etc) can be found under $ORACLE_HOME/cfgtoollogs/rconfig

The Convert verify option in the ConvertToRAC.xml file has three options:

  • Convert verify="YES": rconfig performs checks to ensure that the prerequisites for single-instance to RAC conversion have been met before it starts conversion
  • Convert verify="NO": rconfig does not perform prerequisite checks, and starts conversion
  • Convert verify="ONLY" : rconfig only performs prerequisite checks; it does not start conversion after completing prerequisite checks

Before you actually starts the convert process, you can run the rconfig with convert verify=ONLY option which basically check all the prerequisites. At the end, the result code either would be 0 or 1. Where 0 stands for operation success and 1 stand for operation failures and in case of failure, it will also gives you the reason of failure.

Following RMAN command converts the database from AIX to HP UX platform.

RMAN> convert database new database 'ORCL'

transport script '/tmp_backup/tempdb/transport.sql'

to platform 'HP-UX IA (64-bit)'

db_file_name_convert

'+DG_ORCL/ORCL/datafile','/tmp_backup/tempdb';

Move all the converted datafiles to target machine and follow the ML NOTE: 414878.1 for further details on how to start the on target machine.

Following illustrate how to convert single instance database to RAC using the RCONFIG tool:

cp $ORACLE_HOME/assistants/rconfig/sampleXMLs/ConvertToRAC.xml convertdb.xml

modify the convertdb.xml file according to your environment. Following is the example:

==
convert_t24uat.xml 46 lines, 2964 characters
?xml version=1.0 encoding=UTF-8?
RConfig xmlns:n=http://www.oracle.com/rconfig
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance
xsi:schemaLocation=http://www.oracle.com/rconfig
ConvertToRAC
Verify does a precheck to ensure all pre-requisites are met, before the conversion is attempted. Allowable values are: YES|NO|O
NLY
Convert verify=NO
Specify current OracleHome of non-rac database for SourceDBHome
SourceDBHome /u00/app/oracle/product/10.2.0/db_1 SourceDBHome
Specify OracleHome where the rac database should be configured. It can be same as SourceDBHome
TargetDBHome/u00/app/oracle/product/10.2.0/db_1 TargetDBHome
Specify SID of non-rac database and credential. User with sysdba role is required to perform conversion
SourceDBInfo SID=ORCL
Credentials
User sys User
Password password Password
RolesysdbaRole
Credentials
SourceDBInfo
ASMInfo element is required only if the current non-rac database uses ASM Storage
ASMInfo SID=+ASM1
Credentials
User sys User
Password passwordasm Password
RolesysdbaRole
Credentials
ASMInfo
Specify the list of nodes that should have rac instances running. LocalNode should be the first node in this nodelist.
NodeList
Node name=rac01
Node name=rac02
NodeList
Specify prefix for rac instances. It can be same as the instance name for non-rac database or different. The instance number will be attached to this prefix.
InstancePrefix ORCL InstancePrefix
!--Specify port for the listener to be configured for rac database.If port=, alistener existing on localhost will be used for rac
database.The listener will be extended to all nodes in the nodelist --
Listener port=""
Specify the type of storage to be used by rac database. Allowable values are CFS|ASM. The non-rac database should have same storage type.
SharedStorage type=ASM
Specify Database Area Location to be configured for rac database.If this field is left empty, current storage will be used for rac database. For CFS, this field will have directory path.
TargetDatabaseArea+DG_ORCL TargetDatabaseArea
Specify Flash Recovery Area to be configured for rac database. If this field is left empty, current recovery area of non-rac dat
abase will be configured for rac database. If current database is not using recovery Area, the resulting rac database will not have
a recovery area.
TargetFlashRecoveryArea +DG_ORCL Target FlashRecoveryArea
SharedStorage
Convert
ConvertToRAC
RConfig

===
Once you modify the convert.xml file according to your environment, use the following command to run the tool:

rconfig convertdb.xml

Oracle does the following during conversion:
oracle@usogp01: /u00/app/oracle/product/10.2.0/db_1/bin # ./rconfig convertdb.xml
Converting Database ORCL. to Cluster Database. Target Oracle Home : /u00/app/oracle/product/10.2.0/db_1.
Setting Data Files and Control Files
Adding Database Instances
Adding Redo Logs
Enabling threads for all Database Instances
Setting TEMP tablespace
Adding UNDO tablespaces
Adding Trace files
Setting Flash Recovery Area
Updating Oratab
Creating Password file(s)
Configuring Listeners
Configuring related CRS resources
Adding NetService entries
Starting Cluster Database
Starting Listeners
Operation Succeeded
/u00/app/oracle/product/10.2.0/db_1
ORCL1<\SID>
ORCL2<\SID>
<\SIDList>
oracle@usogp01: /u00/app/oracle/product/10.2.0/db_1/bin #

==
As you can see the exit code is 0, which indicates the success of the conversion process.


Uphills faced during the conversion process

The only error which I got during the conversion was missing listener.ora under ORACLE_HOME/network/admin directory as we had created listener under the ASM home (we have separate RDBMS and ASM homes). Problem has been resolved by creating the link under the RDBMS network admin directory of ASM listener.ora file.

We have contacted one of the developer of rconfig utiity to clarify the two changes (following) to make sure it will work:

non-ASM to ASM
single instance to RAC.

He said that the main purpose of this utility is to convert single instance database to RAC and optinally few non-ASM of target database to ASM while converting to RAC. But, he is not sure about moving the entire non-ASM to ASM while conversion. As they don't have any test case. However, asked us to go ahed and try. Well, we were amazed that it worked for us and we delightly informed the developer that two changes are worked for us and you can have our reference.

Our next migration/conversion is our EBusiness suit and of course, I am going to share my expereince here.

References

http://download-west.oracle.com/docs/cd/B19306_01/install.102/b14201/cvrt2rac.htm#sthref1273


Important Meta Link Notes:

ML Note:387046.1 RCONFIG : Frequently Asked Questions
ML Note:371519.1 RCONFIG Error When Converting From Single Instance To RAC
ML Note:375472.1 RCONFIG FAILS with an null pointer EXCEPTION
ML Note:391297.1 RCONFIG fails when using ASM as Shared Storage
ML Note:388577.1 Using Oracle 10g Release 2 Real Application Clusters and
Automatic Storage Management with Oracle E-Business Suite
Release 12


Happy Reading,



RMAN> convert database new database 'ORCL'
transport script '/tmp_backup/tempdb/transport.sql'
to platform 'HP-UX IA (64-bit)'
db_file_name_convert
'+DG_ORCL/ORCL/datafile','/tmp_backup/tempdb';

this command means convert an ASM database on AIX to filesystem on HPUX?


How we faster the process of converting a non-ASM single-instance database to RAC database with ASM using RCONFIG tool?

I have been given with a challenging task to convert one of our critical production databases, which is of 1 TB (Terabyte) in size, to Oracle 10g RAC with ASM storage option. Even though, there are many methods and tools available to perform this activity, I have preferred to use the RCONFIG tool.

We prepared an input XML file required for RCONFIG tool, and run the RCONFIG utility as follows:

$ cd /oracle/ora102/db_1/assistants/rconfig/sampleXMLs
$ rconfig ConverToRAC.xml
When we start the RCONFIG tool to convert the database to RAC, the RCONFIG tool initially moves all the non-ASM database files to ASM disk files, for this RCONFIG tool internally invokes RMAN utility to backup the target database to the ASM disk groups, eventually the database is converted to RAC using RCONFIG.

The conversion took almost 9 hours to complete the process, because during the conversion RMAN used only one channel per data file to backup to ASM disks. There was no chance of improving the RMAN copy process by allocating more channels in the input XML file, and also Oracle doesn’t recommend doing other changes in the input XML file.

One thing was observed during the RMAN copy that RMAN is using target database control file instead of recovery catalog, and also using the RMAN default preconfigured settings for that database.

To know the RMAN default preconfigured settings for the database:

$ export ORACLE_SID=MYPROD
$ rman target /
Recovery Manager: Release 10.2.0.4.0 - Production on Wed Aug 5 10:21:05 2009

Copyright (c) 1982, 2007, Oracle. All rights reserved.

connected to target database: MYPROD (DBID=1131234567)

RMAN> show all;

using target database control file instead of recovery catalog
RMAN configuration parameters are:
CONFIGURE RETENTION POLICY TO REDUNDANCY 1; # default
CONFIGURE BACKUP OPTIMIZATION OFF; # default
CONFIGURE DEFAULT DEVICE TYPE TO DISK; # default
CONFIGURE CONTROLFILE AUTOBACKUP OFF; # default
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '%F'; # default
CONFIGURE DEVICE TYPE DISK PARALLELISM 1 BACKUP TYPE TO BACKUPSET;
CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE MAXSETSIZE TO UNLIMITED; # default
CONFIGURE ENCRYPTION FOR DATABASE OFF; # default
CONFIGURE ENCRYPTION ALGORITHM 'AES128'; # default
CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default
CONFIGURE SNAPSHOT CONTROLFILE NAME TO '/oracle/ora102/db_1/dbs/snapcf_T24MIG1.f'; # default
Here we see that the PARALLELISM is 1 (default), that’s why the RMAN using only one channel during backing up the non-ASM datafiles to ASM Disk Groups, and were taking 9 hours to complete the backup.

We have changed the PRALLELISM count to 6 (it depends upon number of CPUs you have in the server).

Solution:

RMAN> CONFIGURE DEVICE TYPE DISK PARALLELISM 6;

old RMAN configuration parameters:
CONFIGURE DEVICE TYPE DISK PARALLELISM 1 BACKUP TYPE TO BACKUPSET;
new RMAN configuration parameters:
CONFIGURE DEVICE TYPE DISK PARALLELISM 6 BACKUP TYPE TO BACKUPSET;
new RMAN configuration parameters are successfully stored

RMAN> show all;

RMAN configuration parameters are:
CONFIGURE RETENTION POLICY TO REDUNDANCY 1; # default
CONFIGURE BACKUP OPTIMIZATION OFF; # default
CONFIGURE DEFAULT DEVICE TYPE TO DISK; # default
CONFIGURE CONTROLFILE AUTOBACKUP OFF; # default
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '%F'; # default
CONFIGURE DEVICE TYPE DISK PARALLELISM 6 BACKUP TYPE TO BACKUPSET;
CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE MAXSETSIZE TO UNLIMITED; # default
CONFIGURE ENCRYPTION FOR DATABASE OFF; # default
CONFIGURE ENCRYPTION ALGORITHM 'AES128'; # default
CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default
CONFIGURE SNAPSHOT CONTROLFILE NAME TO '/oracle/ora102/db_1/dbs/snapcf_T24MIG1.f'; # default
After changing the PARALLELISM count to 6, the RMAN has allocated 6 channels and the conversion process has improved greatly and reduced the downtime drastically to 4 Hours 30 minutes.

Following is the extract of rconfig.log file, this file is located under:

$ORACLE_HOME/db_1/cfgtoolslogs/rconfig
............................................................................
............................................................................
............................................................................
[17:17:16:43] Log RMAN Output=RMAN> backup as copy database to destination '+DATA_DG';
[17:17:16:53] Log RMAN Output=Starting backup at 04-AUG-09
[17:17:16:258] Log RMAN Output=using target database control file instead of recovery catalog
[17:17:16:694] Log RMAN Output=allocated channel: ORA_DISK_1
[17:17:16:698] Log RMAN Output=channel ORA_DISK_1: sid=866 devtype=DISK
[17:17:17:9] Log RMAN Output=allocated channel: ORA_DISK_2
[17:17:17:13] Log RMAN Output=channel ORA_DISK_2: sid=865 devtype=DISK
[17:17:17:324] Log RMAN Output=allocated channel: ORA_DISK_3
[17:17:17:327] Log RMAN Output=channel ORA_DISK_3: sid=864 devtype=DISK
[17:17:17:637] Log RMAN Output=allocated channel: ORA_DISK_4
[17:17:17:641] Log RMAN Output=channel ORA_DISK_4: sid=863 devtype=DISK
[17:17:17:967] Log RMAN Output=allocated channel: ORA_DISK_5
[17:17:17:971] Log RMAN Output=channel ORA_DISK_5: sid=862 devtype=DISK
[17:17:18:288] Log RMAN Output=allocated channel: ORA_DISK_6
[17:17:18:293] Log RMAN Output=channel ORA_DISK_6: sid=861 devtype=DISK
[17:17:20:416] Log RMAN Output=channel ORA_DISK_1: starting datafile copy
[17:17:20:427] Log RMAN Output=input datafile fno=00053 name=/oradata/MYPROD/users_01.dbf
[17:17:20:532] Log RMAN Output=channel ORA_DISK_2: starting datafile copy
[17:17:20:544] Log RMAN Output=input datafile fno=00021 name=/oradata/MYPROD/ users_02.dbf
[17:17:20:680] Log RMAN Output=channel ORA_DISK_3: starting datafile copy
[17:17:20:694] Log RMAN Output=input datafile fno=00022 name=/oradata/MYPROD/ users_03.dbf
[17:17:20:786] Log RMAN Output=channel ORA_DISK_4: starting datafile copy
[17:17:20:800] Log RMAN Output=input datafile fno=00023 name=/oradata/MYPROD/ users_04.dbf
[17:17:20:855] Log RMAN Output=channel ORA_DISK_5: starting datafile copy
[17:17:20:868] Log RMAN Output=input datafile fno=00024 name=/oradata/MYPROD/ users_05.dbf
[17:17:20:920] Log RMAN Output=channel ORA_DISK_6: starting datafile copy
[17:17:20:930] Log RMAN Output=input datafile fno=00011 name=/oradata/MYPROD/ users_06.dbf
............................................................................
............................................................................
............................................................................
[21:29:5:518] Log RMAN Output=Finished backup at 04-AUG-09
............................................................................
............................................................................
............................................................................

[21:39:10:723] [NetConfig.startListenerResources:5] started Listeners associated with database MYPROD
[21:39:10:723] [Step.execute:255] STEP Result=Operation Succeeded
[21:39:10:724] [Step.execute:284] Returning result:Operation Succeeded
[21:39:10:724] [RConfigEngine.execute:68] bAsyncJob=false
[21:39:10:725] [RConfigEngine.execute:77] Result= < version="1.1">


&ltConvertToRAC>
&ltConvert>
&ltResponse>
&ltResult code="0" >
Operation Succeeded


&ltReturnValue type="object">
&ltOracle_Home>
/oracle/ora102/db_1

&ltSIDList>
&ltSID&gtMYPROD1<\SID>
&ltSID&gtMYPROD2<\SID>
<\SIDList>


Note: For the sake of look and feel format, the above output has been trimmed neatly. You can also observer that 6 channels were being allocated, timings of backup start and end, and the success code end of the rconfig.log file.

References:

To know more about RCONFIG tool and other Metalink references on it, please take a look at the below blog post written by Mr. Syed Jaffar Hussain.

http://jaffardba.blogspot.com/2008/09/my-experience-of-converting-cross.html

Oracle 10g R2 Documentation information on RCONFIG:

http://download.oracle.com/docs/cd/B19306_01/install.102/b14205/cvrt2rac.htm#BABBAAEH