Re-configuration of the Enterprise Manager in RAC Database for 11gR2
Posted by Mir Sayeed Hassan on May 13th, 2019
Re-configuration of the Enterprise Manager in RAC Database for 11gR2
Get the current status of the enterprise manager of RAC database
Check the status of the Enterprise Manager in database
Error:
[oracle(rac_1)@trac1 bin$./emctl status dbconsole OC4J Configuration issue. /u01/app/oracle/product/11.2.0/db_1/oc4j/j2ee/OC4J_DBConsole_An error occurred while attempting to change log file permissions. Logging may not be active for this process._rac not found
Login to the database & change the of the sys/system/sysman/dbsnmp – test
Verify
SQL> connect sys/test as sysdba Connected
SQL> connect system/test Connected
SQL> connect dbsnmp/test Connected.
SQL> connect sysman/test Connected.
[oracle(rac_1)@trac1 ~$cd /u01/app/oracle/product/11.2.0/db_1/dbs [oracle(rac_1)@trac1 dbs$cp orapwrac orapwrac_bkp [oracle(rac_1)@trac1 dbs$rm -rf orapwrac [oracle(rac_1)@trac1 dbs$orapwd file=orapwrac password=test entries=20
[oracle(rac_1)@trac1 dbs$scp orapwrac root@10.20.0.91:/u01/app/oracle/product/11.2.0/db_1/dbs root@10.20.0.91's password: ****
[oracle(rac_2)@trac2 dbs$ifconfig eth1 Link encap:Ethernet HWaddr 00:50:56:B1:23:F7 inet addr:10.20.0.91 Bcast:10.20.0.255 Mask:255.255.255.0 inet6 addr: fe80::250:56ff:feb1:23f7/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:3738698 errors:0 dropped:4728 overruns:0 frame:0 TX packets:3140219 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:463145670 (441.6 MiB) TX bytes:559918958 (533.9 MiB)
[oracle(rac_2)@trac2 ~$cd /u01/app/oracle/product/11.2.0/db_1/dbs [oracle(rac_2)@trac2 dbs$ll -rw-r-----. 1 root root 3584 Apr 9 17:18 orapwrac
Change the group ownership to oracle & oinstall
[root@trac2 dbs]# chown -R oracle:oinstall orapwrac [root@trac2 dbs]# ll -rw-r-----. 1 oracle oinstall 3584 Apr 9 17:18 orapwrac
Also copy the same orapwrac from primary database to standby database & change the name
[oracle(rac_1)@trac1 dbs$scp orapwrac root@192.168.120.253:/u01/app/oracle/product/11.2.0/db_1/dbs root@192.168.120.253's password: **** orapwrac 100% 3584 3.5KB/s 00:00
[root@racstd dbs]# cd /u01/app/oracle/product/11.2.0/db_1/dbs [root@racstd dbs]# ll -rw-r-----. 1 root root 3584 Apr 9 17:24 orapwrac
[root@racstd dbs]# mv orapwrac orapwstby_rac
[root@racstd dbs]# chown -R oracle:oinstall orapwstby_rac
Also verify the orapw in primary & standby database
[oracle(rac_2)@trac2 dbs$cksum orapwrac 3930077537 3584 orapwrac -- its should be the samein both primary & standby database
Also verify the sys password from primary & standby database
Primary database
[oracle(rac_1)@trac1 bin$sqlplus sys/test@rac_1 as sysdba SQL> exit - ok
[oracle(rac_1)@trac1 bin$sqlplus sys/test@rac_2 as sysdba SQL> exit – ok
[oracle(rac_1)@trac1 bin$sqlplus sys/test@stby_rac as sysdba SQL> exit – ok
Standby database
[oracle@racstd dbs]$ echo $ORACLE_SID stby_rac
[oracle@racstd dbs]$ sqlplus sys/test@stby_rac as sysdba SQL> exit – ok
[oracle@racstd dbs]$ sqlplus sys/test@rac_1 as sysdba SQL> exit – ok
[oracle@racstd dbs]$ sqlplus sys/test@rac_2 as sysdba SQL> exit – ok
EM issue
[oracle(rac_1)@trac1 ~$cd $ORACLE_HOME/bin
[oracle(rac_1)@trac1 bin$./emctl status dbconsole OC4J Configuration issue. /u01/app/oracle/product/11.2.0/db_1/oc4j/j2ee/OC4J_DBConsole_An error occurred while attempting to change log file permissions. Logging may not be active for this process._rac not found.
Error while droping the exiting EM
[oracle(rac_1)@trac1 bin$emca -deconfig dbcontrol db -repos drop -cluster STARTED EMCA at Apr 9, 2019 5:59:21 PM EM Configuration Assistant, Version 11.2.0.3.0 Production Copyright (c) 2003, 2011, Oracle. All rights reserved. Enter the following information: Database unique name: rac Database unique name: Service name: rac Listener ORACLE_HOME [ /u01/app/11.2.0/grid_1 ]: y Password for SYS user: Apr 9, 2019 5:59:32 PM oracle.sysman.emcp.util.GeneralUtil initSQLEngineRemotely WARNING: Error during db connection : ORA-12545: Connect failed because target host or object does not exist Password for SYSMAN user: ---------------------------------------------------------------------- WARNING : While repository is dropped the database will be put in quiesce mode. ---------------------------------------------------------------------- Do you wish to continue? [yes(Y)/no(N)]: y Apr 9, 2019 5:59:42 PM oracle.sysman.emcp.EMConfig perform INFO: This operation is being logged at /u01/app/oracle/cfgtoollogs/emca/rac/emca_2019_04_09_17_59_20.log. Apr 9, 2019 5:59:47 PM oracle.sysman.emcp.ParamsManager checkListenerStatusForDBControl WARNING: ORA-12545: Connect failed because target host or object does not exist Apr 9, 2019 5:59:47 PM oracle.sysman.emcp.EMConfig perform SEVERE: Scan Listener is not up or database service is not registered with it. Start the Scan Listener and register database service and run EM Configuration Assistant again . Refer to the log file at /u01/app/oracle/cfgtoollogs/emca/rac/emca_2019_04_09_17_59_20.log for more details. Could not complete the configuration. Refer to the log file at /u01/app/oracle/cfgtoollogs/emca/rac/emca_2019_04_09_17_59_20.log for more details.
Note:
Correct the remote_listener parameter setting at the database by logging into the database as SYS user and executing:
[oracle(rac_1)@trac1:~]$ srvctl config scan_listener SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521,1522 SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1521,1522 SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1521,1522
SQL> alter system set remote_listener='rac-cluster:1521' scope=both sid='*'; SQL> alter system register;
Login with grid user & issue the below command
[root@trac1 ~]# su - grid [grid(+ASM1)@trac1:~]$ srvctl modify scan_listener -p 1521
If the user password was unknown, we can create or alter the password
Verify
[oracle(rac_1)@trac1:~]$ srvctl config scan SCAN name: ractest-scan, Network: 1/10.20.0.0/255.255.255.0/eth1 SCAN VIP name: scan1, IP: /ractest-scan/10.20.0.98 SCAN VIP name: scan2, IP: /ractest-scan/10.20.0.94 SCAN VIP name: scan3, IP: /ractest-scan/10.20.0.95
[oracle(rac_1)@trac1:~]$ srvctl config scan_listener SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521 SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1521 SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1521
grid(+ASM1)@trac1 ~$sqlplus / as sysasm SQL>
SQL> create user ASM identified by test; User created.
SQL> grant SYSASM, SYSOPER to ASM; Grant succeeded.
SQL> alter user ASMDBA identified by test; User altered.
SQL> select * from v$pwfile_users; USERNAME SYSDB SYSOP SYSAS ------------------------------ ----- ----- ----- SYS TRUE TRUE FALSE ASM FALSE TRUE TRUE
SQL> select * from gv$pwfile_users order by inst_id; INST_ID USERNAME SYSDB SYSOP SYSAS --------- ------------------------------ ----- ----- ----- 1 ASM FALSE TRUE TRUE 1 SYS TRUE TRUE FALSE 1 ASMDBA FALSE TRUE TRUE 2 ASM FALSE TRUE TRUE 2 SYS TRUE TRUE FALSE 2 ASMDBA FALSE TRUE TRUE 6 rows selected.
SQL> create user asmsnmp identified by test; User created.
SQL> grant SYSASM, SYSOPER to asmsnmp; Grant succeeded.
SQL> select * from v$pwfile_users; USERNAME SYSDB SYSOP SYSAS ------------------------------ ----- ----- ----- SYS TRUE TRUE FALSE ASM FALSE TRUE TRUE ASMDBA FALSE TRUE TRUE ASMSNMP FALSE TRUE TRUE
[oracle(rac_1)@trac1 ~$srvctl stop listener [oracle(rac_1)@trac1 ~$srvctl start listener [oracle(rac_1)@trac1 ~$srvctl status listener Listener LISTENER is enabled Listener LISTENER is running on node(s): trac1, trac2
Find out the cluster name for the rac database
Login to the grid & get the cluster name
[root@trac1 ~]# su - grid grid(+ASM1)@trac1 ~$cd $GRID_HOME/bin grid(+ASM1)@trac1 bin$ ./olsnodes -c test-cluster
or
grid(+ASM1)@trac1 bin$cd $GRID_HOME/bin grid(+ASM1)@trac1 bin$cemutlo -n
[root@trac1 dbs]# su – grid grid(+ASM1)@trac1 ~$sqlplus / as sysasm
SQL> connect sys/test as sysasm Connected
Also verify the Permission of Oracle binary for Oracle & Grid users – Refer
Finally try to execute the emca for the RAC database for creation of EM.
[oracle(rac_1)@trac1:/u01/app/oracle/product/11.2.0/db_1/bin]$ ./emca -deconfig dbcontrol db -repos recreate -cluster STARTED EMCA at May 13, 2019 11:52:04 AM EM Configuration Assistant, Version 11.2.0.3.0 Production Copyright (c) 2003, 2011, Oracle. All rights reserved. Enter the following information: Database unique name: rac Service name: rac Listener port number: 1521 Listener ORACLE_HOME [ /u01/app/11.2.0/grid_1 ]: Password for SYS user: Incorrect usage. Run 'emca -help' for more details. [oracle(rac_1)@trac1:/u01/app/oracle/product/11.2.0/db_1/bin]$ [oracle(rac_1)@trac1:/u01/app/oracle/product/11.2.0/db_1/bin]$ [oracle(rac_1)@trac1:/u01/app/oracle/product/11.2.0/db_1/bin]$ [oracle(rac_1)@trac1:/u01/app/oracle/product/11.2.0/db_1/bin]$ [oracle(rac_1)@trac1:/u01/app/oracle/product/11.2.0/db_1/bin]$ ./emca -config dbcontrol db -repos recreate -cluster STARTED EMCA at May 13, 2019 11:52:24 AM EM Configuration Assistant, Version 11.2.0.3.0 Production Copyright (c) 2003, 2011, Oracle. All rights reserved. Enter the following information: Database unique name: rac Service name: rac Listener port number: 1521 Listener ORACLE_HOME [ /u01/app/11.2.0/grid_1 ]: Password for SYS user: Password for DBSNMP user: Password for SYSMAN user: Cluster name: test-cluster Email address for notifications (optional): Outgoing Mail (SMTP) server for notifications (optional): ASM ORACLE_HOME [ /u01/app/11.2.0/grid_1 ]: ASM port [ 1521 ]: ASM username [ ASMSNMP ]: ASM user password: May 13, 2019 11:52:48 AM oracle.sysman.emcp.util.GeneralUtil initSQLEngineRemotely WARNING: Error during db connection : ORA-12514: TNS:listener does not currently know of service requested in connect descriptor -------Note (As per the Oracle support we can ignore this warning & continue) ----------------------------------------------------------------- You have specified the following settings Database ORACLE_HOME ................ /u01/app/oracle/product/11.2.0/db_1 Database instance hostname ................ Listener ORACLE_HOME ................ /u01/app/11.2.0/grid_1 Listener port number ................ 1521 Cluster name ................ test-cluster Database unique name ................ rac Email address for notifications ............... Outgoing Mail (SMTP) server for notifications ............... ASM ORACLE_HOME ................ /u01/app/11.2.0/grid_1 ASM port ................ 1521 ASM user role ................ SYSDBA ASM username ................ ASMSNMP ---------------------------------------------------------------------- WARNING : While repository is dropped the database will be put in quiesce mode. ---------------------------------------------------------------------- Do you wish to continue? [yes(Y)/no(N)]: y May 13, 2019 11:52:56 AM oracle.sysman.emcp.EMConfig perform INFO: This operation is being logged at /u01/app/oracle/cfgtoollogs/emca/rac/emca_2019_05_13_11_52_24.log. May 13, 2019 11:53:02 AM oracle.sysman.emcp.EMReposConfig invoke INFO: Dropping the EM repository (this may take a while) ... May 13, 2019 11:53:03 AM oracle.sysman.emcp.EMReposConfig invoke INFO: Repository successfully dropped May 13, 2019 11:53:04 AM oracle.sysman.emcp.EMReposConfig createRepository INFO: Creating the EM repository (this may take a while) ... May 13, 2019 12:01:53 PM oracle.sysman.emcp.EMReposConfig invoke INFO: Repository successfully created May 13, 2019 12:02:04 PM oracle.sysman.emcp.EMReposConfig uploadConfigDataToRepository INFO: Uploading configuration data to EM repository (this may take a while) ... May 13, 2019 12:03:35 PM oracle.sysman.emcp.EMReposConfig invoke INFO: Uploaded configuration data successfully May 13, 2019 12:03:36 PM oracle.sysman.emcp.EMDBCConfig instantiateOC4JConfigFiles INFO: Propagating /u01/app/oracle/product/11.2.0/db_1/oc4j/j2ee/OC4J_DBConsole_trac1_rac to remote nodes ... May 13, 2019 12:03:40 PM oracle.sysman.emcp.EMDBCConfig instantiateOC4JConfigFiles INFO: Propagating /u01/app/oracle/product/11.2.0/db_1/oc4j/j2ee/OC4J_DBConsole_trac2_rac to remote nodes ... May 13, 2019 12:03:48 PM oracle.sysman.emcp.EMAgentConfig deployStateDirs INFO: Propagating /u01/app/oracle/product/11.2.0/db_1/trac1_rac to remote nodes ... May 13, 2019 12:03:53 PM oracle.sysman.emcp.EMAgentConfig deployStateDirs INFO: Propagating /u01/app/oracle/product/11.2.0/db_1/trac2_rac to remote nodes ... May 13, 2019 12:04:02 PM oracle.sysman.emcp.util.DBControlUtil secureDBConsole INFO: Securing Database Control (this may take a while) ... May 13, 2019 12:04:42 PM oracle.sysman.emcp.util.DBControlUtil startOMS INFO: Starting Database Control (this may take a while) ... May 13, 2019 12:06:25 PM oracle.sysman.emcp.EMDBPostConfig performConfiguration INFO: Database Control started successfully May 13, 2019 12:06:25 PM oracle.sysman.emcp.EMDBPostConfig performConfiguration INFO: >>>>>>>>>> The Database Control URL is https://trac1:1158/em >>>>>>>>>>>>>>>>>>> May 13, 2019 12:06:33 PM oracle.sysman.emcp.EMDBPostConfig showClusterDBCAgentMessage INFO: **************** Current Configuration **************** INSTANCE NODE DBCONTROL_UPLOAD_HOST ---------- ---------- --------------------- rac trac1 trac1 rac trac2 trac1 May 13, 2019 12:06:33 PM oracle.sysman.emcp.EMDBPostConfig invoke WARNING: *********************** WARNING ************************ Management Repository has been placed in secure mode wherein Enterprise Manager data will be encrypted. The encryption key has been placed in the file: /u01/app/oracle/product/11.2.0/db_1/trac1_rac/sysman/config/emkey.ora. Ensure this file is backed up as the encrypted data will become unusable if this file is lost. *********************************************************** Enterprise Manager configuration completed successfully FINISHED EMCA at May 13, 2019 12:06:33 PM