Troubleshooting Grid Control/ Grid Log files

Troubleshooting Grid Control/ Grid Log files
http://oracledbasupport.co.uk

When troubleshooting problems in the Grid Control framework, it is important to remember that Grid Control is a J2EE application deployed to the Oracle Application Server 10g J2EE and Web Cache installation using an Oracle database as a repository.

When faced with a problem, first localize the problem to the affected component

1. Troubleshooting the OMR
2. Troubleshooting the OMS
3. Troubleshooting the OMA










1. Troubleshooting the OMR ( Repository/Database )

Connectivity: Access to the OMR is through the database listener. Please locate listener log file at ORACLE_HOME/network/log. The log file contains record of every connection and connection request received by the listener. Listener errors are recorded in the log file with a TNS-xxxxx format. If necessary, enable tracing for listener to record more diagnostic information. Trace files for the listener can be found in ORACLE_HOME/network/trace

Availability: If OMR database is unavailable for some other reason, check database alert log and trace files to determine the root cause.

Space usage: Space problems may occur in the OMR database if the OMR tablespaces are unable to accept new information due to lack of free space in the data files.

Performance: OMR database performance problems will normally trigger Grid Control alerts as metric threshold values are crossed.

2. Troubleshooting the OMS ( Management Service )

OMS log files are produced for – 1> Oracle Web Cache, 2> Oracle HTTP Server, 3> Oracle Application Server Containers for J2EE, and 4> the Oracle Process Monitor and Notification subcomponents.

1> Oracle Web Cache Log files may be found in ORACLE_HOME/webcache/logs. There are two key log files for Web Cache: the access_log (which records every connection to the Web Cache) and the event_log (which contains data about Web Cache availability and errors).

2> The Oracle HTTP Server provides the access_log and error_log in ORACLE_HOME/Apache/Apache/logs. As with Web Cache, the access_log can grow very large and is therefore “rotated” every 12 hours. Older logs are preserved in ORACLE_HOME/Apache/Apache/logs.

3> Oracle Application Server Containers for J2EE for generated log files for Grid Control may be found in ORACLE_HOME/j2ee/OC4J_EM/log/OC4J_EM_default_island_1. OC4J generates several log files that provide diagnostic information, including:

default-web-access.log: Contains information about each request received by the component. Information includes the IP address of the requester, date and time of the request, the URL that was specified in the request, and the result code. All requests should come from the OHS; all result codes should indicate success (200). This information is valuable when troubleshooting connection difficulties between the OHS and OC4J.

em-application.log: Contains information about all events, errors, and exceptions associated with the EM application. This is excellent information for troubleshooting Java errors.

global-application.log: Contains information about events, errors, and exceptions relating to the OC4J JVM that are not specific to the EM application. This is also a good source of information for troubleshooting Java errors.

server.log: Includes availability information for the OC4J_EM component, including start and stop times.

4> The Oracle Process Monitor and Notification (OPMN) system provides logs for each of the OMS components in ORACLE_HOME/opmn/logs. Key log files include:

HTTP_Server: In cases where the OHS will not start, this log file often contains pertinent error messages that can help diagnose the problem.

OC4J~OC4J_EM~default_island~1: Contains any errors received while starting the OC4J_EM component. This can be helpful in troubleshooting Java errors and global configuration problems.

3. Troubleshooting the OMA ( Management Agent )

Connectivity between the OMA and OMS: When a single management agent is unable to connect to the OMS, the problem will normally be found on the OMA’s server. If multiple agents are unable to connect, the problem may lie with the OMS or underlying network. Check AGENT_HOME/sysman/config/emd.properties and verify that the repository URL is correct.

Ensure you can ping the host identified as the repository URL. attempt to telnet to the OMS host.

Upload throughput as the OMA reports information on metric targets through the OMS to the OMR: OMA logs may be found in AGENT_HOME/sysman/log. OMA upload errors are recorded in AGENT_HOME/sysman/log/emdctl.trc.

Target discovery as new targets are added to a server: Use emctl config listtargets or check AGENT_HOME/sysman/emd/targets.xml to determine which targets are monitored by the agent. Remember to make a backup copy of the targets.xml file prior to any modifications. Errors with target discovery will be reported in AGENT_HOME/sysman/log/agentca.log.

Oracle Enterprise Manager 10g Grid Control comprises three major components:
Oracle Management Repository (OMR)
Oracle Management Service (OMS)
Oracle Management Agent (OMA)

•

Oracle Enterprise Manager 10g Grid Control comprises three major components:

• Oracle Management Repository (OMR)
• Oracle Management Service (OMS)
• Oracle Management Agent (OMA)

Key configuration files for the OMA include:

–./sysman/emd/targets.xml

–./sysman/config/emd.properties

–./sysman/config/emagentlogging.properties •

To start the Grid Control framework, do the following:
1. Start the OMR database listener
2. Start the OMR database
3. Start all OMSs
4. Start the OMA on the OMS/OMR server
5. Start the OMA on managed servers

To stop the Grid Control framework:
1. Stop the OMA on managed servers (optional).
2. Stop the OMA on the OMS/OMR server.
3. Stop all OMSs.
4. Stop the OMR database.
5. Stop the OMR database listener.

Oracle Management Agent : Must be installed on each managed host, Must be in its own ORACLE_HOME , Communicates with the OMS via HTTP or HTTPS

Oracle Management Service : Includes member components:, Oracle HTTP Server (OHS), Oracle Application Server Containers for J2EE (OC4J), OracleAS Web Cache, Is a J2EE application deployed on Oracle Application Server 10g, Connects to the OMR by using Java Database Connectivity

Oracle Management Repository : Resides in an Oracle database, Includes schema objects belonging to SYSMAN, Can be installed in a preexisting database, Can be configured to contain other management data, including:, Oracle Application Server infrastructure database, Oracle Recovery Manager catalog

Oracle Process Monitor and Notification Control utility $ opmnctl startall $ opmnctl stopall $ opmnctl startproc ias-component=OC4J $ opmnctl stopproc process-type=OC4J_EM $ opmnctl status -l

Enterprise Manager Control Utility (emctl) $ emctl start oms $ emctl stop oms $ emctl status oms $ emctl start iasconsole $ emctl stop iasconsole

Distributed Configuration Manager Control (dcmctl) $ dcmctl start $ dcmctl start –ct WebCache $ dcmctl stop $ dcmctl getstate $ dcmctl listcomponents

Example Output of Commands

$ /opt/oracle/product/oms10g/opmn/bin/opmnctl status -l

Processes in Instance: EnterpriseManager0.test
-------------------+--------------------+---------+----------+------------+----------+-----------+------
ias-component      | process-type       |     pid | status   |        uid |  memused |    uptime | ports
-------------------+--------------------+---------+----------+------------+----------+-----------+------
DSA                | DSA                |     N/A | Down     |        N/A |      N/A |       N/A | N/A
LogLoader          | logloaderd         |     N/A | Down     |        N/A |      N/A |       N/A | N/A
HTTP_Server        | HTTP_Server        |     749 | Alive    | 1325924656 |   194208 | 123:54:07 | http1:7778,http2:7200,https1:4444,https2:1159,http3:4889
dcm-daemon         | dcm-daemon         |     629 | Alive    | 1325924655 |    24956 | 123:54:39 | N/A
OC4J               | home               |   23906 | Alive    | 1325924658 |    33148 | 121:24:21 | ajp:12502,rmi:12402,jms:12602
OC4J               | OC4J_EMPROV        |   23907 | Alive    | 1325924659 |    57724 | 121:24:21 | ajp:12503,rmi:12403,jms:12603
OC4J               | OC4J_EM            |   12150 | Alive    | 1325924667 |   242044 | 1193:01:~ | ajp:12501,rmi:12401,jms:12601
WebCache           | WebCache           |   23908 | Alive    | 1325924660 |   106924 | 121:24:21 | http:7777,invalidation:9401,statistics:9402
WebCache           | WebCacheAdmin      |   23909 | Alive    | 1325924661 |    15652 | 121:24:21 | administration:9400

$ /opt/oracle/product/oms10g/opmn/bin/opmnctl status

Processes in Instance: EnterpriseManager0.test
-------------------+--------------------+---------+---------
ias-component      | process-type       |     pid | status
-------------------+--------------------+---------+---------
DSA                | DSA                |     N/A | Down
LogLoader          | logloaderd         |     N/A | Down
HTTP_Server        | HTTP_Server        |     749 | Alive
dcm-daemon         | dcm-daemon         |     629 | Alive
OC4J               | home               |   23906 | Alive
OC4J               | OC4J_EMPROV        |   23907 | Alive
OC4J               | OC4J_EM            |   12150 | Alive
WebCache           | WebCache           |   23908 | Alive
WebCache           | WebCacheAdmin      |   23909 | Alive

To Start and stop individual ias components use : opmnctl startproc ias-component=OC4J (See IAS-Component Above)

To Start and stop individual processes use : opmnctl startproc process-type=OC4J_EMPROV (See Process-Type Above)

$ /opt/oracle/product/oms10g/opmn/bin/opmnctl stopall
opmnctl: stopping opmn and all managed processes...
================================================================================
opmn id=test:6201
5 of 6 processes stopped. ias-instance id=EnterpriseManager0.test
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
--------------------------------------------------------------------------------
ias-component/process-type/process-set:
OC4J/OC4J_EM/default_island

Error
--> Process (pid=12150)
time out while waiting for a managed process to stop
Log:
/opt/oracle/product/oms10g/opmn/logs/OC4J~OC4J_EM~default_island~1

opmnctl: graceful stop of processes failed, trying forceful shutdown...

$ /opt/oracle/product/oms10g/opmn/bin/opmnctl status
Unable to connect to opmn.
Opmn may not be up.

EM Agent Deploy Log Files

Posted By Sagar Patil

Connectivity Log File	Location
prereq.log	$OMS_HOME/sysman/prov/agentpush//prereqs/local
prereq.out	$OMS_HOME/sysman/prov/agentpush//prereqs/local
prereq.err	$OMS_HOME/sysman/prov/agentpush//prereqs/local

Prerequisite Log File	Location
prereq.log	$OMS_HOME/sysman/prov/agentpush//prereqs/
prereq.out	$OMS_HOME/sysman/prov/agentpush//prereqs/
prereq.err	$OMS_HOME/sysman/prov/agentpush//prereqs/

Logs	Location
EMAgentPush.log	/sysman/prov/agentpush/logs/ : :Agent Deploy application logs.
remoteInterfaces.log	/sysman/prov/agentpush/logs/ :Logs of the remote interfaces layer
install.log/.err	/sysman/prov/agentpush//logs// :Log or error of the new agent installation or new cluster agent installation
upgrade.log/.err	/sysman/prov/agentpush//logs// : Log or error of the upgrade operation using Agent Deploy
nfsinstall.log/err	/sysman/prov/agentpush//logs//: Log or error of the agent installation using the Shared Agent Home option in Agent Deploy.
clusterUpgrade.log/err	/sysman/prov/agentpush//logs//:Log or error of the cluster upgrade operation using Agent Deploy
sharedClusterUpgradeConfig.log/err	/sysman/prov/agentpush//logs//:Log or error of the config operation in case of upgrade on a shared cluster
config.log/err	/sysman/prov/agentpush//logs//:Log or error of the configuration of shared cluster in case of an agent installation on a shared cluster.
preinstallscript.log/.err	/sysman/prov/agentpush//logs//:Log/error of the running of preinstallation script, if specified Clone Agent Configuration If you have a working EM grid console configuration, you can TAR a client configuration and clone it at other nodes. Tar the working agent at node $ cd /opt/oracle/product $ ls -l total 8 drwxr-xr-x 65 oracle9 oinstall 2048 Oct 10 2006 9.2.0 drwxrwx--- 3 oracle9 oinstall 96 Nov 19 13:29 agent_10g $ tar –cvf agent.tar agent_10g Set the ORACLE_HOME to /opt/oracle/product/agent10g< ?xml:namespace prefix = o /> $export ORACLE_HOME=/opt/oracle/product/agent10g 4. Run the installer as shown below - $cd $ORACLE_HOME/oui/bin Or $cd /opt/oracle/product/agent10g/oui/bin $./runInstaller -clone -forceClone ORACLE_HOME=$ORACLE_HOME ORACLE_HOME_NAME=agent10g -noconfig –silent 5. Run the root.sh as root user /opt/oracle/product/agent10g/agent10g/root.sh 6. Amend the REPOSITORY_URL and emdWalletSrcUrl parameters in the /sysman/config/emp.properties configuration file to make the agent point to right grid server 7. Run the agent configuration assistant $cd $ORACLE_HOME/bin or $cd /opt/oracle/product/agent10g/bin $./agentca -f 8. Secure the agent $./emctl secure agent 9. Start the agent, if not already $./emctl start agent

Replacing Jinitiator with JRE for Oracle Apps 11i

Replacing Jinitiator with JRE for Oracle Apps 11i
----------------------------------------------------

Download Interoperability patch 6863618 (Metalink)

Download the Java Runtime Environment (JRE) 6 plug-in at, http://java.sun.com/javase/downloads/index.jsp

Downloaded JRE Native plug-in file from jre-6_uX-windows-i586-p.exe and rename to j2se160x.exe
(Where X = the version number. Using jre-6u5 as an example this would be j2se16005.exe)


Move the j2se1600X.exe file to the web application tier and place it in [COMMON_TOP]/util/jinitiator


Apply Developer 6i Patchset 18 ( 4948577 )

unzip patch to $ORACLE_HOME
cd $ORACLE_HOME/developer6i_patch18
./patch_install.sh 2>&1 | tee patch_install_p18.log

cd $ORACLE_HOME/procbuilder60/lib; make -f ins_procbuilder.mk install
cd $ORACLE_HOME/forms60/lib; make -f ins_forms60w.mk install
cd $ORACLE_HOME/graphics60/lib; make -f ins_graphics60w.mk install
cd $ORACLE_HOME/reports60/lib; make -f ins_reports60w.mk install

cp developer6i_patch18/bin/genshlib $ORACLE_HOME/bin


Additional Patches
------------------
5713544

sh patch.sh
adrelink.sh force=y "fnd f60webmx"


4261542

cd $ORACLE_HOME
unzip p4261542_600_GENERIC.zip

cp -r $ORACLE_HOME/forms60/java/oracle/forms/handler/AlertDialog.class $ORACLE_HOME/forms60/java/oracle/forms/handler/AlertDialog.class.PRE_BUG4261542
cp -r $ORACLE_HOME/forms60/java/oracle/forms/engine/Main.class $ORACLE_HOME/forms60/java/oracle/forms/engine/Main.class.PRE_BUG4261542

cd $ORACLE_HOME/4261542/oracle/forms/engine
cp Main.class $ORACLE_HOME/forms60/java/oracle/forms/engine/Main.class
cd $ORACLE_HOME/4261542/oracle/forms/handler
cp AlertDialog.class $ORACLE_HOME/forms60/java/oracle/forms/handler/AlertDialog.class

Generate Product JAR Files ( adadmin )

5216496
sh patch.sh

5753922
sh patch.sh
adrelink.sh force=y "fnd f60webmx"


6195758
cd $ORACLE_HOME
unzip p6195758_600_GENERIC.zip
cp -r $ORACLE_HOME/forms60/java/oracle/forms/handler/UICommon.class $ORACLE_HOME/forms60/java/oracle/forms/handler/UICommon.class.PRE_BUG6195758
cp -r $ORACLE_HOME/forms60/java/oracle/forms/handler/ComponentItem.class $ORACLE_HOME/forms60/java/oracle/forms/handler/ComponentItem.class.PRE_BUG6195758
cd $ORACLE_HOME/6195758/oracle/forms/handler
cp UICommon.class $ORACLE_HOME/forms60/java/oracle/forms/handler/UICommon.class
cp ComponentItem.class $ORACLE_HOME/forms60/java/oracle/forms/handler/ComponentItem.class
Generate Product JAR Files ( adadmin )

5938515
sh patch.sh
adrelink.sh force=y "fnd f60webmx"

3830807
sh patch.sh

4586086
cd $ORACLE_HOME
unzip .zip
cd $ORACLE_HOME/forms60/lib
mv env_forms60.mk env_forms60.mk.PRE_BUG4586086
cp $ORACLE_HOME/bug4586086/env_forms60.mk $ORACLE_HOME/forms60/lib
cd $ORACLE_HOME/forms60/lib
make -f cus_forms60w.mk libso_install
adrelink.sh force=y "fnd f60webmx"

relink - adadmin
select 'Maintain Applications Files Menu' and then select 'Relink Applications Program'
Enter list of products to link ('all' for all products)[all] : fnd
Generate specific executables for each selected product [No] ? y
Relink with debug information [No] ? n
Enter executables to relink, or enter 'all' [all] : f60webmx ar60run ar60runb ar60rund *

4888294 (adpatch )


5884875

cd $ORACLE_HOME
unzip p5884875_600_GENERIC.zip
cp -r $ORACLE_HOME/forms60/java/oracle/forms/engine/Main.class $ORACLE_HOME/forms60/java/oracle/forms/engine/Main.class.PRE_BUG5884875
cp -r $ORACLE_HOME/forms60/java/oracle/forms/handler/AlertDialog.class $ORACLE_HOME/forms60/java/oracle/forms/handler/AlertDialog.class.PRE_BUG5884875
cd $ORACLE_HOME/5884875/oracle/forms/engine
cp Main.class $ORACLE_HOME/forms60/java/oracle/forms/engine/Main.class
cd $ORACLE_HOME/5884875/oracle/forms/handler
cp AlertDialog.class $ORACLE_HOME/forms60/java/oracle/forms/engine/AlertDialog.class
Run adadmin "Generate JAR Files"


(6863618)
---------

Apply the Patch Driver using adpatch
Run the txkSetPlugin.sh script from the /6863618/fnd/bin/

txkSetPlugin.sh 16005


Start Oracle Applicatons 11.5.10.2 and start using Java instead of Jinititor for Applets

Here we are going to create a schema similar to the APPS schema but has only read-only prviliges.

Here we are going to create a schema similar to the APPS schema but has only read-only prviliges.


Steps:

1. Create the user for the required schema:

SQL> connect system/manager
SQL> create user identified by default tablespace
temporary tablespace temp;

2. Grant connect and resource privileges to your user:

SQL> connect system/manager
SQL> grant connect, resource to ;

3. Use the following select statement to generate a script that will grant privileges on APPS
objects to your user.
Note the following:

A. This select statement should generate a script that will grant almost all required permissions
to a user called MYUSER.
B. This select statement is a guide only. You should work with your DBA to enhance it according to
your requirements.

SELECT 'GRANT '
|| DECODE(O.OBJECT_TYPE,'TABLE','SELECT',
'VIEW','SELECT',
'EXECUTE')
|| ' ON '
|| DECODE(O.Owner,'PUBLIC','',O.Owner || '.')
|| '"'
|| O.OBJECT_NAME
|| '"'
|| ' TO MYUSER;' COMMAND
FROM ALL_OBJECTS O
WHERE O.OBJECT_TYPE IN ('TABLE','PACKAGE','PACKAGE BODY','PROCEDURE', 'VIEW','FUNCTION')
UNION
SELECT 'GRANT '
|| DECODE (O2.object_type, 'TABLE', 'SELECT',
'VIEW', 'SELECT',
'EXECUTE')
|| ' ON '
|| DECODE(O.Owner,'PUBLIC','',O.Owner || '.')
|| '"'
|| O.object_name
|| '"'
|| ' TO MYUSER;' COMMAND
FROM ALL_OBJECTS O
, ALL_OBJECTS O2
, DBA_SYNONYMS S
WHERE O.object_type = 'SYNONYM'
AND O.object_name = S.synonym_name
AND O2.object_name = S.table_name
AND O2.OBJECT_TYPE IN ('TABLE','PACKAGE','PACKAGE BODY','PROCEDURE', 'VIEW','FUNCTION')

4. Use the following select statement to generate a script that will create synonyms in
schema for all objects owned by APPS.

SELECT 'CREATE SYNONYM MYUSER.'
|| O.OBJECT_NAME
|| ' FOR APPS.'
|| O.OBJECT_NAME
|| ';' COMMAND
FROM DBA_OBJECTS O
WHERE O.Owner = 'APPS'

5. Run the above two scripts as SYS user.

Migrate Users From E-business Suite R12 to Oracle Internet Directory

Migrate Users From E-business Suite R12 to Oracle Internet Directory

Tags: AppsUserExport, bulk user export apps, bulk user import, bulkload, disable provisioning profile, Implement SSO in R12, ldapadd, ldifimport, ldifmigrate, ldifmigrator, migrate users from e-business suite 11i R12 to OID, oidctl, oidprovtool, SSO AND OID, sso and r12 user synchronization, user propogation bidirectional OID SSO R12 R11i

Hi All,
After Integrating the Existing E-business suite R12 Instance with Single Sign On(OID)
we will find that the existing users are NOT automatically migrated to the Oracle Internet Directory.

So I will explain Here How to Migrate the existing Users from E-Business Suite to Oracle Internet Directory.
Later on depending upon the provisioning profile It will synchronize accordingly.(default setting is bi-directional).

step I.Use AppsUserExport to export apps user information from R12 E-Business Suite…

$java oracle.apps.fnd.oid.AppsUserExport -v -dbc $INST_TOP/appl/fnd/12.0.0/secure/VIS.dbc -o usersr12.txt -pwd apps -g -l usersr12.log

Step II.Convert Intermediate LDIF file to Final LDIF File from OID Server…

Transfer the file usersr12.txt which we got from AppsUserExport to OID Server and
Execute the following command

$ldifmigrator “input_file=usersr12.txt” “output_file=usersr12.ldif”
“s_UserContainerDN=cn=users,dc=vectorconsulting,dc=co.uk”
“s_UserNicknameAttribute=uid”

Output

Migration of LDIF data completed.All entries are successfully migrated…

Step III.Loading Final LDIF File into Oracle Internet Directory..

a. disable the provisioning profile with oidprovtool..

$oidprovtool operation=disable ldap_host=sso.vectorconsulting.co.uk ldap_port=369
ldap_user_dn=cn=orcladmin ldap_user_password=welcome123 application_dn=”orclApplicationCommonName=VIS,
cn=EBusiness,cn=Products,cn=OracleContext,dc=vectorconsulting,dc=co.uk” profile_mode=BOTH

b. Stop OID Server using $ORACLE_HOME/opmn/bin/opmnctl stopall

c. Incase you used oidmon or oidctl then check using ldapcheck whether they are stopped..

d. Shutdown any other running OID processes manually by
oidctl connect=VIS server= instance=3 stop

and now grep the procesess and ensure that no OID processes are running..

e. Finally coming to the actual loading part.

we use bulkload for loading but before loading we should use the -check and -generate option
as follows to check duplicates and if duplicates are found in the logfile ,manually edit the LDIF file
and remove those user entries like follows

$bulkload connect=”IASDB” check=true generate=true file=”usersr12.ldif”

Enter OID password(orcladmin password)
the resultant logfile will be bulkload.log in $ORACLE_HOME/ldap/log directory.
After ensuring that you encounter no errors in the bulkload.log file proceed to the next step..

f.bulkload connect=’IASDB’ load=true file=”usersr12.ldif”

Now You can Start the OID Server then enable the provisiong profile using the following
$oidprovtool operation=enable ldap_host=sso.vectorconsulting.co.uk ldap_port=369
ldap_user_dn=cn=orcladmin ldap_user_password=welcome123 application_dn=”orclApplicationCommonName=VIS,
cn=EBusiness,cn=Products,cn=OracleContext,dc=vectorconsulting,dc=co.uk” profile_mode=BOTH

and check whether populated users are successfully able
to login to E-business Suite R12 Via Single-Sign On….

Application tier processes are broadly classified in to five type of Services

Type of Application Services
Application tier processes are broadly classified in to five type of Services

i) Root Service - Oracle Process Manager and Notification (OPMN) is part of Oracle Application Server 10G R2/R3. OPMN is used for inter process communication (between OC4J, HTTP Server), death detection and start/stop of Services for 10g R3.

ii) Web Entry Point Services - HTTP Server is part of Oracle Application Server 10G R3 and built on Apache (aka Oracle Apache. All web request to R12 first hit HTTP Server.

iii) Web Application Services - OACORE OC4J, Forms OC4J, OAFM OC4J - These are set of 3 OC4J (Oracle Container for J2EE) to run application.

iv) Batch Processing Services - This set of service consist of Apps Listener (FNDFS & FNDSM), Concurrent Manager, Fulfillment Server and Interaction Server

v) Other Services - Oracle Forms Service, Forms Metric Client/Server and Mobile Web Application (MWA) Service

.

Process —> Services –> Startup Script –> Context parameter

a) Oracle Process Manager - Root Service - adopmnctl.sh- s_root_status, s_opmnstatus
b) Oracle HTTP Server - Web Entry Point Service - adapcctl.sh- s_web_entry_status, s_oacorestatus
c) OACORE OC4J - Web Application Services - adoacorectl.sh- s_web_applications_status s_oacorectrl
d) FORMS OC4J - Web Application Services - adformsctl.sh- s_web_applications_status, s_formsstatus
e) OAFM OC4J - Web Application Services - adoafmctl.sh- s_web_applications_status, s_oafmstatus
f) Concurrent Manager - Batch Processing Service - adcmctl.sh- s_batch_status, s_concstatus
g) Apps Listener - Batch Processing Service - adalnctl.sh- s_batch_status, s_tnsstatus
h) Interaction Center (ICSM) - Batch Processing Service - ieoicsm.sh- s_batch_status, s_icsmctrl
i) Fulfillment Server - Batch Processing Service - jtffmctl.sh- s_batch_status, s_jtffsstatus
j) Forms Server - Other Service - adformsrvctl.sh- s_other_service_group_status , s_formsserver_status
k) Metric Client - Other Service - adfmcctl.sh- s_other_service_group_status , s_metcstatus
l) Metric Server - Other Service - adfmsctl.sh- s_other_service_group_status , s_metsstatus
m) Mobile Web Application (MWA) - Other Service - mwactlwrpr.sh- s_other_service_group_status , s_mwastatus

.

Key points to note

1) When you select Root services, Web Services are automatically selected i.e. First three Services Root, Web Entry Point and Web Application Services can only be enabled or disabled together.
If you want Web Entry Point Services, Web Application Services, and Root Services to be installed on different nodes then use autoconfig after installation.

2. In R12 with introduction of Services there is NO concept of Forms node or Web Node.

3. With default installation all services Root, Web Entry Point, Web Application Services, Batch Processing and Other services are enabled.

.

References

* 406558.1 Configuring Applications Node Services in Oracle Applications Release 12 (Metalink Note)
* Chapter 2, Page 59-61 of R12 Installation Guide

Script to find and kill runaway processes

Script to find and kill runaway processes (concurrent requests) after termination
April 9, 2009 at 1:09 pm | In Oracle Apps | No Comments
Tags: clear runaway processes, concurrent request status "terminating", kill processes conc requests, script to find runaway processes, sql to find pending and running concurrent requests

Hello Guys,

This script is useful for finding the SID and serial number for a specific 11i/R12 concurrent request that has been terminated in the applications, but has not been cleaned up by PMON.
Sometimes in 11i, when a long-running concurrent request is terminated, it will still be active in the database for long periods taking up resources. With this script, you can find the SID, serial number and server PID for a terminated concurrent request so that the process can be killed at the database and OS level. The input parameter for this script is the terminated concurrent request ID.

After running this script you can then do an “alter system kill session” and a “kill-9″ using the server pid to stop the processing faster.
This script has been tested on 8.1.7.4, and on 11.5.1 through 11.5.9.

rem fndreqidx.sql
rem
SET LINESIZE 132
rem
TTITLE ‘Deleted Concurrent Manager Requests’
rem
COL user_concurrent_queue_name FORMAT a15 HEADING ‘QUEUE NAME’ trunc
COL request_id FORMAT 9999999 HEADING ‘REQUEST|ID’
COL phase_code FORMAT a1 HEADING ‘P’
COL status_code FORMAT a1 HEADING ‘S’
COL actual_start_date FORMAT a11 HEADING ‘START DATE’
COL user_name FORMAT a15 HEADING ‘USER|NAME’
COL spid FORMAT a6 HEADING ‘SERVER|PID’ headsep ‘|’
COL program FORMAT a40 HEADING ‘PROGRAM’
COL duration FORMAT a10 HEADING ‘DURATION’
COL sidserial FORMAT a11 HEADING ‘SID,SERIAL#’
–
SELECT qt.user_concurrent_queue_name
, fcr.Request_Id Request_id
, fu.User_name
, p.spid
, s.sid ||’, ‘|| s.serial# SIDSERIAL
, substr( Fcpv.Concurrent_Program_Name ||’ – ‘|| Fcpv.User_Concurrent_Program_Name, 1,46) Program
, to_char( fcr.actual_start_date, ‘mm/dd hh24:mi’ ) actual_start_date
, phase_code, status_code
, to_char( trunc(sysdate) + ( sysdate – fcr.actual_start_date )
, ‘hh24:mi:ss’ ) duration
FROM apps.Fnd_Concurrent_Queues Fcq
, apps.fnd_concurrent_queues_tl qt
, apps.Fnd_Concurrent_Requests Fcr
, apps.Fnd_Concurrent_Programs Fcp
, apps.Fnd_User Fu
, apps.Fnd_Concurrent_Processes Fpro
, v$session s
, v$process p
, apps.Fnd_Concurrent_Programs_Vl Fcpv
WHERE phase_code = ‘C’
AND status_Code = ‘X’
AND s.paddr = p.addr
AND fcr.requested_by = user_id
AND fcq.application_id = qt.application_id
AND fcq.concurrent_queue_id = qt.concurrent_queue_id
AND userenv(’lang’) = qt.language
AND fcr.os_process_id = s.process
AND fcr.Controlling_Manager = Concurrent_Process_Id
AND (fcq.concurrent_queue_id = fpro.concurrent_queue_id
AND fcq.application_id = fpro.queue_application_id )
AND (fcr.concurrent_program_id = fcp.concurrent_program_id
AND fcr.program_application_id = fcp.application_id )
AND (fcr.concurrent_program_id = fcpv.concurrent_program_id
AND fcr.program_application_id = fcpv.application_id )
ORDER BY fcr.actual_start_date;

Cheat sheet for Apps DBA!

Search FND_PROFILES for Hard-Coded Path
As part of Windows to Linux Upgrade/Migrate project. I wrote the following query to pull all the profiles that has a hard-coded Windows Path

-- Start Script

SELECT fpo.profile_option_name PROFILE, fpov.profile_option_value VALUE,
DECODE (fpov.level_id,10001, 'SITE', 10002, 'APPLICATION',10003, 'RESPONSIBILITY',10004, 'USER') "LEVEL",
fa.application_short_name app, fr.responsibility_name responsibility,
fu.user_name "USER"
FROM apps.fnd_profile_option_values fpov,
apps.fnd_profile_options fpo,
apps.fnd_application fa,
apps.fnd_responsibility_vl fr,
apps.fnd_user fu,
apps.fnd_logins fl
WHERE fpo.profile_option_id = fpov.profile_option_id
AND fa.application_id(+) = fpov.level_value
AND fr.application_id(+) = fpov.level_value_application_id
AND fr.responsibility_id(+) = fpov.level_value
AND fu.user_id(+) = fpov.level_value
AND fl.login_id(+) = fpov.last_update_login
AND ( LOWER (fpov.profile_option_value) LIKE 'f:%'
OR LOWER (fpov.profile_option_value) LIKE 'd:%'
OR LOWER (fpov.profile_option_value) LIKE 'c:%'
)
ORDER BY 2

-- End Script
Posted by Shyam Enuganti at 9:49 PM 0 comments
Search LONG Column for a string
I am working on Windows to Linux upgrade project. As part of it I was looking for Hard-Coded Paths in Profiles, Packages, Procedures, Functions, Views and Triggers.

Searching for the hard-coded path in DBA_VIEWS and DBA_TRIGGERS posed some challenges. So described is the SQL which I used created to search DBA_VIEWS, same thing can be modified for DBA_TRIGGERS.

(Gave up with formating the code)

-- START Script
SELECT 'Views containing WINDOWS path' WINPATH FROM DUAL;

SET SERVEROUTPUT ON

DECLARE
/* This program will retrive the Hard-coded WINDOWS path used in dba_views; The same program can be used to search in dba_triggers by changing SELECT clause. This script can also be used to search a LONG datatype column by changing required clauses.
*/
v_path varchar2(10);
p_path varchar2(10) := ' ';
v_view dba_views.view_name%TYPE;
v_own dba_views.owner%TYPE;

-- For search in long datatype
cursor1 INTEGER;
ignore INTEGER;
STATEMENT VARCHAR2 (1000);
out_val VARCHAR2 (1000);
out_length INTEGER;
num_bytes INTEGER := 1000;
offset INTEGER;
num_fetches INTEGER;
row_count INTEGER;
out_view VARCHAR2 (30);
out_own VARCHAR2 (30);

CURSOR c1
IS
SELECT 'F:' winpath FROM dual Union
SELECT 'C:' winpath FROM dual Union
SELECT 'D:' winpath FROM dual;

BEGIN
BEGIN
FOR c1_rec IN c1
LOOP
v_path := c1_rec.winpath;

--Search in TRIGGER Body
BEGIN
STATEMENT := 'select text ,view_name,owner from dba_views where owner not in (''SYS'',''SYSTEM'')';
cursor1 := DBMS_SQL.open_cursor;
DBMS_SQL.parse (cursor1, STATEMENT, DBMS_SQL.native);
DBMS_SQL.define_column_long (cursor1, 1);
DBMS_SQL.define_column (cursor1, 2, v_view, 30);
DBMS_SQL.define_column (cursor1, 3, v_own, 30);
ignore := DBMS_SQL.EXECUTE (cursor1);

LOOP
IF DBMS_SQL.fetch_rows (cursor1) > 0
THEN
row_count := row_count + 1;
offset := 0; num_fetches := 1;
DBMS_SQL.COLUMN_VALUE (cursor1, 2, out_view);
DBMS_SQL.COLUMN_VALUE (cursor1, 3, out_own);

LOOP
DBMS_SQL.column_value_long (cursor1, 1, num_bytes, offset, out_val,out_length );

IF (INSTR (UPPER (out_val), v_path) > 0)
THEN
IF (p_path = v_path)
THEN
NULL;
ELSE
DBMS_OUTPUT.put_line ( 'Windows path '
|| v_path
|| ' found in view '
|| out_own
|| '.'
|| out_view
);
p_path := v_path;
END IF;

offset := offset + num_bytes;
num_fetches := num_fetches + 1;
ELSE
EXIT;
END IF;

IF out_length < num_bytes
THEN
EXIT;
END IF;
END LOOP;
ELSE
EXIT;
END IF;
END LOOP;

DBMS_SQL.close_cursor (cursor1);
EXCEPTION
WHEN OTHERS
THEN
DBMS_OUTPUT.put_line (SQLERRM);

IF DBMS_SQL.is_open (cursor1)
THEN
DBMS_SQL.close_cursor (cursor1);
END IF;
END;
END LOOP; -- End of c1_rec loop
END;
END;
/


--END Script
Posted by Shyam Enuganti at 9:30 PM 0 comments
Labels: dba_triggers, dba_view, long, search long
Tuesday, September 26, 2006
UPGRADING wanna reduce DOWNTIME by couple of Hours!
In May we upgraded from 11.5.9 to 11.5.10.2. It took only 18 Hrs for Patching. (800+GB Database, 3 Nodes with 6 NLS languages, followed STAGED Patching). Never heard of STAGED Patching then check Metalink Note:242480.1

During our practice builds we were looking for time consuming jobs. And the following 2 jobs took couple of Hours. By following the below PRE-TASKS we saved couple of Hours and 60+GB of disk space.


Pre Task FIX for Job ICXDLTMP.sql
******
truncate table icx.icx_sessions;
truncate table icx.icx_session_attributes;
truncate table icx.icx_text;
truncate table icx.icx_requisitioner_info;
truncate table icx.icx_transactions;

begin
fnd_bc4j_cleanup_pkg.delete_transaction_rows(SYSDATE - 4/24);
fnd_bc4j_cleanup_pkg.delete_control_rows(SYSDATE - 4/24);
end;
/
******
Pre Task FIX for Job afupdfmt.sql
******
update applsys.fnd_lobs set FILE_FORMAT = 'IGNORE'
where nvl(program_name,'attach') != 'FND_HELP';

sql> @$FND_TOP/sql/aflobbld.sql APPLSYS APPS
******

Bump the values of ADJREOPTS & ADJRIOPTS tags to atlease 512MB or 1GB in contextfile "XML file" and run ADCONFIG - During patching ADJAVA cmd will load XML Files, Jar Files into DB and setting these to higher values will reduce the downtime.


Good Luck and Cheers!


Insight into AutoConfig
I am going to give you some insight or inner working of this tool. Thanks to Harminder Singh (Try at your own risk)

For detailed information on AUTOCONFIG refer to the following metalink Note:165195.1, Note:218089.1, Note:270519.1 and Note:217368.1.

Wanna create CONTEXT file manually, try this ... and then copy the generated XML to $APPL_TOP/admin/host_SID.xml
adbldxml.pl tier=apps appsuser=apps appspass=pswd log=$HOME/admin/log/adconfig_`date '+%m%d%y_%H%M%S'`.log out=$HOME/admin/SID_`date '+%m%d%y_%H%M%S'`.xml servername=nodename

Wanna Update the tags in CONTEXT File, try this to update tag s_appsEnvName to SHYAM
java -classpath "${CLASSPATH}:${CMDDIR}/java/adconfig.zip" oracle.apps.ad.context.UpdateContext ${HOME}/admin/SID.xml s_appsEnvName "SHYAM"

Wanna try to instantiate configuration files from custom driver and templates files based on the values from CONTEXT FILE.
java -classpath "${CLASSPATH}:${CMDDIR}/java/adconfig.zip" oracle.apps.ad.autoconfig.InstantiateFile -e $HOME/admin/${TWO_TASK}.xml -d /d01/sid/admin/custom.drv -log /d01/sid/admin/$CONTEXT_NAME.xxcinstantiate.log -bacdir /d01/sid/admin/bak/$CONTEXT_NAME -pwd NOP




Check if Concurent Managers can start without really Starting!

Do you want to know, without starting Concurrent managers if they can start without any issues. I personally ran into this issue many times both in the single node and in multi-node (Parallel Concurrent Processing) Solaris environments.

The following command can be used to check if the Concurrent Managers can be started without really starting. If it comes up with " Internal heap Dump" then there will be no issues, but if it core dumps or hangs then there is some issue with LISTENER. Try stopping listener it hangs, only way is to kill the process.

sqlplus apps/pswd@FNDSM_NODE1_SID
sqlplus apps/pswd@FNDSM_NODE2_SID
sqlplus apps/pswd@FNDSM_NODE3_SID

When we are on 11.5.9 I had to really push Oracle to fix bug #3617132. In the manager log file you normally see the following error "CONC-SM TNS FAIL". I even saw the same issue in 11.5.10 instance also.

Only way is modifying the listener.ora on the appsTier OR start listener with debug mode enabled.
APPS_SID =
(ADDRESS_LIST =
(ADDRESS= (PROTOCOL= TCP)(Host= hostname)(Port= 1522))
(ADDRESS= (PROTOCOL= IPC)(Host= hostname)(Port= 15221)(KEY=SIDKEY1))
(ADDRESS= (PROTOCOL= IPC)(Host= hostname)(Port= 15222)(KEY=SIDKEY2))
(ADDRESS= (PROTOCOL= IPC)(Host= hostname)(Port= 15223)(KEY=SIDKEY3))
)

Need more details, shoot me an email

Good Luck and Cheers!


For detailed discussion about SSLConfigTool please refer to Oracle Doc

If you are installing Infrastructure in a cluster environment you have an option to enable SSL during installation. In Non-Cluster environment you have to enable SSL using manual method or by using SSLConfigTool(Introduced in 10.1.2.)
Where as on the Middle Tier you don't have an option to install in SSL mode. We have to use SSLConfigTool to enable SSL (For single node or multi node)

Enabling SSL manually is bit tricky and also takes lot of time. Using this tool it can be enabled in less than 5 min.

The SSLConfigTool command is used as follows:

SSLConfigTool ( -config_w_prompt
| -config_w_file
| -config_w_default
| -rollback )
[-dry_run]
[-wc_for_infra]
[-secure_admin]
[-opwd ]
[-ptl_dad ]
[-ptl_inv_pwd ]

Commands I used to Enable Rollback SSL.

On Infrastructure Tier:

SSLConfigTool -config_w_prompt -opwd orcladminpswd

On Middle Tier:
1) SSLConfigTool -config_w_prompt -opwd orcladminpswd -ptl_inv_pwd orcladminpswd
2) Go to Webcacheadmin and change the settings in Site Definitions (Make SSL site Default) & in Site-to-Server Mapping change the Origin Server to to use SSL Port
To Rollback:
SSLConfigTool -rollback -opwd orcladminpswd


Good Luck and Cheers!

Convert openssl to oracle wallet certificate Oracle Wallet certificate is different from openssl certificate. If you already have a openssl certificate for yourcompany.com and want to use the same certificate for 10gAS then follow the steps to convert openssl certificate to Oracle Wallet certificate. Thanks to my colleague Jim Kennedy he has put the steps together.

For detailed explanation refer to metalink Note:340178.1

1) Copy the following files to a directory (e.g /var/opt/oracle/walletcert)

server.crt = Standard server certificate (received from Verisign)
server.key = Standard key (generated with openssl - 1024.key)
ca.crt = Combination of intermediate certificate and root certificate concatenated


2) Generate wallet certificate

$IAS_HOME/Apache/open_ssl/bin/openssl pkcs12 -export -descert -in server.crt -inkey server.key -certfile ca.crt -name server_wallet.crt(can be anyname) -out ewallet.p12


3) Copy/ftp the walletcert directory to the server where 10gAS is installed.

4) Open and save the certificate using owm executable.

Good Luck and Cheers
Posted by Shyam Enuganti at 3:51 PM 3 comments
Wanna DUMP JVM threads or check GC size
This scripts will be handy if you wanna check the Garbage Collection Size of OACore JVMs or DUMP them to check for any locking/waiting issue


#
# scripts Check the Garbage Collection Size of OACore JVMs
#
echo " GC size This one -| on "`date`
echo " V"
for i in `ls -1 $HOME/ora/iAS/Apache/Jserv/logs/jvm/OA*out`;do
which_jproces=`fuser $i 2>/dev/null `
which_jvm=`basename $i|awk -F\. '{print $2}'`
echo "Jvm # $which_jvm running on pid$which_jproces" "\c"; tail -1 $i
done|sort +9 -n

#
# scripts to dump OACore JVMs
#

for pids in `ps -fu $LOGNAME|grep -i verbose|grep -v grep|awk '{print $2}'`;do
echo "Running 3 samples 5 sec apart for pids : $pids "
(kill -3 $pids ; sleep 5 ; kill -3 $pids ; sleep 5 ; kill -3 $pids ) &
done


JVM & OC4J Size in 11i Apps & 10gAS The default values for the JVM/OC4J do not suffice and are not ideal for performance, these values are typically too low and may not be applicable for medium or large scale applications.

After lot of internal discussions and with Oracle Support opinion we had sized our JVM and OC4J. These values are set in our environment for Oracle Apps running on 11.5.10.2 and 10gAS 10.1.2.0.2

In jserv.properties

wrapper.bin.parameters=-verbose:gc -Xmx1024M -Xms256M -XX:MaxPermSize=256M -XX:NewRatio=2 -XX:+PrintGCTimeStamps -XX:+UseTLAB -XX:+PrintClassHistogram -XX:-UseParallelGC


In jserv.conf (By default only 1 JVM is started, so edit or change the value in Context file and run autoconfig)

ApJServGroup OACoreGroup 8 1 /d01/sid/ora/iAS/Apache/Jserv/etc/jserv.properties



In 10gAS for Portal on Middle Tier set the value by editing in opmn.xml or using enterprise-manager console. The values has to be set for java-options and numprocs. Ref:Oracle Doc

The following values need to be editied in OC4J_Portal section. It starts with " process-type id="OC4J_Portal" module-id="OC4J"

"-server -Dnetworkaddress.cache.ttl=30 -Djava.security.policy=/d01/sid/as10g/j2ee/OC4J_Portal/config/java2.policy -Djava.awt.headless=true -Xms512m -Xmx1024m"
...
process-set id="default_island" numprocs="4"