首页 > 代码库 > ORA-00445 实例挂起
ORA-00445 实例挂起
现象:
节点2 运行:select sum(bytes) from dba_segments 长期等待: waiting for ‘gc cr request‘
节点1 僵死:select sid,serial# from gv$session where username=‘****‘
分析:
数据库内部等待关系:
blocking blocker event
388 773,397 enq: PV - syncstart
773 397 enq: PR - contention
397 enq: PR - contention os thread startup:ora-445
--1
节点1 运行语句select sid,serial# from gv$session where username=‘WANQIONG‘:
失败的原因:PROCESS 53/57 两个会话运行该语句,PROCESS 20: CKPT block这两个会话
PROCESS 20 由于waiting for ‘enq: PR - contention‘ name|mode=0x50520006
0x50520006 所有者:PROCESS 45: CJQ0
根据:awk -f ass109.awk calldb1_ora_15639.trc 分析:
Resource Holder State
Enqueue PR-00000000-00000000 45: 0: waiting for ‘os thread startup‘
Enqueue PV-00000000-00000000 38: 38: is waiting for 45:
Latch 380030bc0 ??? Blocker
PROCESS 45: CJQ0 进程是罪魁祸首!该进程无法启动。
--2 节点2
节点2运行语句:select sum(bytes) from dba_segments,等待时间0: waiting for ‘gc cr request‘
cssd,crsd没有发现明显节点通讯报错,怀疑由于节点1死机引起
--3 节点1 警告日志分析
Mon Aug 11 09:45:49 2014
Archived Log entry 58557 added for thread 1 sequence 27907 ID 0x4bcd9425 dest 1:
Mon Aug 11 16:08:44 2014
Errors in file /oracle/diag/rdbms/calldb/calldb1/trace/calldb1_cjq0_8914.trc (incident=56362):
ORA-00445: background process "J000" did not start after 120 seconds
Incident details in: /oracle/diag/rdbms/calldb/calldb1/incident/incdir_56362/calldb1_cjq0_8914_i56362.trc
kkjcre1p: unable to spawn jobq slave process
Errors in file /oracle/diag/rdbms/calldb/calldb1/trace/calldb1_cjq0_8914.trc:
Mon Aug 11 16:12:25 2014
Errors in file /oracle/diag/rdbms/calldb/calldb1/trace/calldb1_mmon_8497.trc (incident=56202):
ORA-00445: background process "m000" did not start after 120 seconds
Incident details in: /oracle/diag/rdbms/calldb/calldb1/incident/incdir_56202/calldb1_mmon_8497_i56202.trc
Mon Aug 11 16:14:28 2014
awr分析内存16G
Begin End
Host Mem (MB): 15,744.0 15,744.0
SGA use (MB): 4,608.0 4,608.0
PGA use (MB): 462.5 460.1
% Host Mem used for SGA+PGA: 32.21 32.19
重启后内存12g
[oracle@calldb1 /home/oracle]$ prtconf | grep ‘Memory‘
Memory size: 12288 Megabytes
怀疑问题出在内存上:
1379200.1 根据文档可以确定:
节点2 运行:select sum(bytes) from dba_segments 长期等待: waiting for ‘gc cr request‘
节点1 僵死:select sid,serial# from gv$session where username=‘****‘
分析:
数据库内部等待关系:
blocking blocker event
388 773,397 enq: PV - syncstart
773 397 enq: PR - contention
397 enq: PR - contention os thread startup:ora-445
--1
节点1 运行语句select sid,serial# from gv$session where username=‘WANQIONG‘:
失败的原因:PROCESS 53/57 两个会话运行该语句,PROCESS 20: CKPT block这两个会话
PROCESS 20 由于waiting for ‘enq: PR - contention‘ name|mode=0x50520006
0x50520006 所有者:PROCESS 45: CJQ0
根据:awk -f ass109.awk calldb1_ora_15639.trc 分析:
Resource Holder State
Enqueue PR-00000000-00000000 45: 0: waiting for ‘os thread startup‘
Enqueue PV-00000000-00000000 38: 38: is waiting for 45:
Latch 380030bc0 ??? Blocker
PROCESS 45: CJQ0 进程是罪魁祸首!该进程无法启动。
--2 节点2
节点2运行语句:select sum(bytes) from dba_segments,等待时间0: waiting for ‘gc cr request‘
cssd,crsd没有发现明显节点通讯报错,怀疑由于节点1死机引起
--3 节点1 警告日志分析
Mon Aug 11 09:45:49 2014
Archived Log entry 58557 added for thread 1 sequence 27907 ID 0x4bcd9425 dest 1:
Mon Aug 11 16:08:44 2014
Errors in file /oracle/diag/rdbms/calldb/calldb1/trace/calldb1_cjq0_8914.trc (incident=56362):
ORA-00445: background process "J000" did not start after 120 seconds
Incident details in: /oracle/diag/rdbms/calldb/calldb1/incident/incdir_56362/calldb1_cjq0_8914_i56362.trc
kkjcre1p: unable to spawn jobq slave process
Errors in file /oracle/diag/rdbms/calldb/calldb1/trace/calldb1_cjq0_8914.trc:
Mon Aug 11 16:12:25 2014
Errors in file /oracle/diag/rdbms/calldb/calldb1/trace/calldb1_mmon_8497.trc (incident=56202):
ORA-00445: background process "m000" did not start after 120 seconds
Incident details in: /oracle/diag/rdbms/calldb/calldb1/incident/incdir_56202/calldb1_mmon_8497_i56202.trc
Mon Aug 11 16:14:28 2014
awr分析内存16G
Begin End
Host Mem (MB): 15,744.0 15,744.0
SGA use (MB): 4,608.0 4,608.0
PGA use (MB): 462.5 460.1
% Host Mem used for SGA+PGA: 32.21 32.19
重启后内存12g
[oracle@calldb1 /home/oracle]$ prtconf | grep ‘Memory‘
Memory size: 12288 Megabytes
怀疑问题出在内存上:
1379200.1 根据文档可以确定:
声明:以上内容来自用户投稿及互联网公开渠道收集整理发布,本网站不拥有所有权,未作人工编辑处理,也不承担相关法律责任,若内容有误或涉及侵权可进行投诉: 投诉/举报 工作人员会在5个工作日内联系你,一经查实,本站将立刻删除涉嫌侵权内容。