When starting the haddo database using srvctl i received the following error:
$ srvctl start instance -i haddo1 -d haddo PRCR-1013 : Failed to start resource ora.haddo.db PRCR-1064 : Failed to start resource ora.haddo.db on node dbrac1 CRS-5017: The resource action "ora.haddo.db start" encountered the following error: ORA-01078: failure in processing system parameters ORA-01565: error in identifying file '+DG_DATA1/haddo/spfilehaddo.ora' ORA-17503: ksfdopn:10 Failed to open file +DG_DATA1/haddo/spfilehaddo.ora ORA-12547: TNS:lost contact . For details refer to "(:CLSN00107:)" in "/u01/app/grid/diag/crs/dbrac1/crs/trace/crsd_oraagent_oracle.trc". CRS-2674: Start of 'ora.haddo.db' on 'dbrac1' failed
The DB is active only on node2 (dbrac2).
$ crsctl stat res ora.haddo.db -t --------------------------------------------------------------------- Name Target State Server State details --------------------------------------------------------------------- Cluster Resources --------------------------------------------------------------------- ora.haddo.db 1 ONLINE OFFLINE Instance Shutdown,STABLE 2 ONLINE ONLINE dbrac2 Open,STABLE ---------------------------------------------------------------------
The issue is caused by incorrect permission of $GRID_HOME/bin/oracle which lead to connection failure to ASM instance:
$ ls -l $GRID_HOME/bin/oracle -rwxr-x--x. 1 grid oinstall 291544753 Mar 2 2017 /u01/app/12.1.0.2/grid/bin/oracle
Oracle binary should have permission of 6751:
$ cd $GRID_HOME/bin $ chmod 6751 oracle $ ls -l $GRID_HOME/bin/oracle -rwsr-s--x. 1 grid oinstall 291544753 Mar 2 2017 /u01/app/12.1.0.2/grid/bin/oracle
After correcting permissions on oracle executable my issue resolved:
$ srvctl start instance -i haddo1 -d haddo $ crsctl stat res ora.haddo.db -t -------------------------------------------------------------------------------- Name Target State Server State details -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.haddo.db 1 ONLINE ONLINE dbrac1 Open,STABLE 2 ONLINE ONLINE dbrac2 Open,STABLE --------------------------------------------------------------------------------