首页 > 代码库 > "crsctl check crs" command hangs at EVMD check
"crsctl check crs" command hangs at EVMD check
Pre-11gR2: "crsctl check crs" command hangs at EVMD check (文档 ID 1578875.1)APPLIES TO:
Oracle Database - Enterprise Edition - Version 10.2.0.3 to 11.1.0.7 [Release 10.2 to 11.1]
Information in this document applies to any platform.
SYMPTOMS
In a 2 node RAC environment, with 11.1.0.7 CRS, execution of the command "crsctl check crs" hangs at EVMD check only in Node 1
[oracle@srv03401 bin]$ ./crsctl check crs
Cluster Synchronization Services appears healthy
Cluster Ready Services appears healthy
From Node1, below is the output of strace for the command "crsctl check crs"
# strace -f -t -o /tmp/crschk.trc crsctl check crs
Content of the generated output file :/tmp/crschk.trc is as follows:
28268 11:47:03 execve("./crsctl", ["./crsctl", "check", "crs"], [/* 23 vars */]) = 0
28268 11:47:03 brk(0) = 0x193d2000
28268 11:47:03 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b35b9436000
28268 11:47:03 uname({sys="Linux", node="srv03401.metra.com", ...}) = 0
28268 11:47:03 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
28268 11:47:03 open("/etc/ld.so.cache", O_RDONLY) = 3
28268 11:47:03 fstat(3, {st_mode=S_IFREG|0644, st_size=92563, ...}) = 0
28268 11:47:03 mmap(NULL, 92563, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2b35b9437000
28268 11:47:03 close(3) = 0
28268 11:47:03 open("/lib64/libtermcap.so.2", O_RDONLY) = 3
28268 11:47:03 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\20\300z2\0\0\0"..., 832) = 832
28268 11:47:03 fstat(3, {st_mode=S_IFREG|0755, st_size=15840, ...}) = 0
28268 11:47:03 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b35b944e000
28268 11:47:03 mmap(0x327ac00000, 2108944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x327ac00000
28268 11:47:03 mprotect(0x327ac03000, 2093056, PROT_NONE) = 0
28268 11:47:03 mmap(0x327ae02000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x327ae02000
28268 11:47:03 close(3) = 0
28268 11:47:03 open("/lib64/libdl.so.2", O_RDONLY) = 3
..
..
28268 11:47:03 close(3) = 0
28268 11:47:03 write(1, "Cluster Ready Services appears h"..., 39) = 39
28268 11:47:03 socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 3
28268 11:47:03 setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
28268 11:47:03 bind(3, {sa_family=AF_INET6, sin6_port=htons(0), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = 0
28268 11:47:03 getsockname(3, {sa_family=AF_INET6, sin6_port=htons(42027), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [140733193388060]) = 0
28268 11:47:03 getpeername(3, 0x7fff5f19e1e0, [140733193388060]) = -1 ENOTCONN (Transport endpoint is not connected)
28268 11:47:03 getsockopt(3, SOL_SOCKET, SO_SNDBUF, [5536382933839118336], [4]) = 0
28268 11:47:03 getsockopt(3, SOL_SOCKET, SO_RCVBUF, [5536382933843050496], [4]) = 0
28268 11:47:03 fcntl(3, F_SETFD, FD_CLOEXEC) = 0
28268 11:47:03 fcntl(3, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
28268 11:47:03 geteuid() = 700
28268 11:47:03 times({tms_utime=1, tms_stime=2, tms_cutime=0, tms_cstime=0}) = 7422615891
28268 11:47:03 socket(PF_FILE, SOCK_STREAM, 0) = 4
28268 11:47:03 access("/var/tmp/.oracle/sSYSTEM.evm.acceptor.auth", F_OK) = 0
28268 11:47:03 connect(4, {sa_family=AF_FILE, path="/var/tmp/.oracle/sSYSTEM.evm.acceptor.auth"...}, 110
CAUSE
Analysing the strace output, looks like it was trying to write to a socket.
========
28268 11:47:03 socket(PF_FILE, SOCK_STREAM, 0) = 4
28268 11:47:03 access("/var/tmp/.oracle/sSYSTEM.evm.acceptor.auth", F_OK) = 0
28268 11:47:03 connect(4, {sa_family=AF_FILE, path="/var/tmp/.oracle/sSYSTEM.evm.acceptor.auth"...}, 110 <<<<<<<
========
This, indicates a problem with the network socket file.
SOLUTION
Get the PID of evmd.bin process and kill it
$ ps -ef | grep ‘d.bin‘
oracle 21046 21045 0 2012 ? 00:07:46 /u01/app/ract/crs/bin/evmd.bin
root 21054 15845 0 2012 ? 11:34:47 /u01/app/ract/crs/bin/crsd.bin reboot
oracle 22072 21453 0 2012 ? 05:44:50 /u01/app/ract/crs/bin/ocssd.bin
root 22135 1 0 2012 ? 00:00:00 /u01/app/ract/crs/bin/oclskd.bin
oracle 22410 1 0 2012 ? 00:00:00 /u01/app/ract/crs/bin/oclskd.bin
oracle 29834 27854 0 13:22 pts/8 00:00:00 egrep d.bin
$ kill -9 21046
After killing evmd.bin process, the command "crsctl check crs" returns the complete output without any hangs.
[oracle@srv03401 bin]$ ./crsctl check crs
CSS appears healthy
CRS appears healthy
EVM appears healthy
声明:以上内容来自用户投稿及互联网公开渠道收集整理发布,本网站不拥有所有权,未作人工编辑处理,也不承担相关法律责任,若内容有误或涉及侵权可进行投诉: 投诉/举报 工作人员会在5个工作日内联系你,一经查实,本站将立刻删除涉嫌侵权内容。