10G ASM 只能在一个节点启动 奇怪了

[复制链接]
查看11 | 回复8 | 2008-7-26 09:30:48 | 显示全部楼层 |阅读模式
dbca 创建ASM实例【以前做11G RAC ASM实例都很轻松】
rac1
Sat Jul 26 09:12:53 2008
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
WARNING: No cluster interconnect has been specified. Depending on
the communication driver configured Oracle cluster traffic
may be directed to the public interface of this machine.
Oracle recommends that RAC clustered databases be configured
with a private interconnect for enhanced security and
performance.
Picked latch-free SCN scheme 2
Using LOG_ARCHIVE_DEST_1 parameter default value as /oracle/app/product/10.0.0.2/dbs/arch
Autotune of undo retention is turned off.
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.1.0.
System parameters with non-default values:
large_pool_size
= 12582912
instance_type
= asm
cluster_database = TRUE
instance_number
= 1
remote_login_passwordfile= EXCLUSIVE
background_dump_dest = /oracle/app/admin/+ASM/bdump
user_dump_dest = /oracle/app/admin/+ASM/udump
core_dump_dest = /oracle/app/admin/+ASM/cdump
asm_diskgroups =
Cluster communication is configured to use the following interface(s) for this instance
127.0.0.1
Sat Jul 26 09:12:53 2008
cluster interconnect IPC version:Oracle UDP/IP
IPC Vendor 1 proto 2
PMON started with pid=2, OS id=20804
DIAG started with pid=3, OS id=20806
PSP0 started with pid=4, OS id=20808
LMON started with pid=5, OS id=20810
LMD0 started with pid=6, OS id=20812
LMS0 started with pid=7, OS id=20814
MMAN started with pid=8, OS id=20824
DBW0 started with pid=9, OS id=20826
LGWR started with pid=10, OS id=20828
CKPT started with pid=11, OS id=20830
SMON started with pid=12, OS id=20832
RBAL started with pid=13, OS id=20834
GMON started with pid=14, OS id=20846
Sat Jul 26 09:12:53 2008
lmon registered with NM - instance id 1 (internal mem no 0)
Sat Jul 26 09:12:53 2008
Reconfiguration started (old inc 0, new inc 1)
ASM instance
List of nodes:
0
Global Resource Directory frozen
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Sat Jul 26 09:12:54 2008
LMS 0: 0 GCS shadows cancelled, 0 closed
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Post SMON to start 1st pass IR
Sat Jul 26 09:12:54 2008
LMS 0: 0 GCS shadows traversed, 0 replayed
Sat Jul 26 09:12:54 2008
Submitted all GCS remote-cache requests
Post SMON to start 1st pass IR
Fix write in gcs resources
Reconfiguration complete
LCK0 started with pid=15, OS id=20863
Sat Jul 26 09:12:54 2008
SQL> ALTER DISKGROUP ALL MOUNT
Sat Jul 26 09:12:54 2008
ALTER SYSTEM SET asm_diskgroups='' SCOPE=MEMORY;-- 到这里应该是启动起来了,但是当启动rac2的+ASM2实例之后 ,当前rac1的实例就宕掉
Sat Jul 26 09:15:55 2008
Error: KGXGN polling error (15)
Sat Jul 26 09:15:55 2008
Errors in file /oracle/app/admin/+ASM/bdump/+asm1_lmon_20810.trc:
ORA-29702: error occurred in Cluster Group Service operation
LMON: terminating instance due to error 29702
Sat Jul 26 09:15:55 2008
Dump system state for local instance only
System State dumped to trace file /oracle/app/admin/+ASM/bdump/+asm1_diag_20806.trc
Sat Jul 26 09:15:55 2008
Trace dumping is performing id=[cdmp_20080726091555]
Sat Jul 26 09:15:55 2008
Shutting down instance (abort)
License high water mark = 2
Sat Jul 26 09:15:59 2008
Instance terminated by LMON, pid = 20810
Sat Jul 26 09:16:00 2008
Instance terminated by USER, pid = 25542
Sat Jul 26 09:16:02 2008
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
WARNING: No cluster interconnect has been specified. Depending on
the communication driver configured Oracle cluster traffic
may be directed to the public interface of this machine.
Oracle recommends that RAC clustered databases be configured
with a private interconnect for enhanced security and
performance.
Picked latch-free SCN scheme 2
Using LOG_ARCHIVE_DEST_1 parameter default value as /oracle/app/product/10.0.0.2/dbs/arch
Autotune of undo retention is turned off.
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.1.0.
System parameters with non-default values:
large_pool_size
= 12582912
instance_type
= asm
cluster_database = TRUE
instance_number
= 1
remote_login_passwordfile= EXCLUSIVE
background_dump_dest = /oracle/app/admin/+ASM/bdump
user_dump_dest = /oracle/app/admin/+ASM/udump
core_dump_dest = /oracle/app/admin/+ASM/cdump
asm_diskgroups =
Cluster communication is configured to use the following interface(s) for this instance
127.0.0.1
Sat Jul 26 09:16:02 2008
cluster interconnect IPC version:Oracle UDP/IP
IPC Vendor 1 proto 2
PMON started with pid=2, OS id=25750
DIAG started with pid=3, OS id=25752
PSP0 started with pid=4, OS id=25754
LMON started with pid=5, OS id=25756
LMD0 started with pid=6, OS id=25758
LMS0 started with pid=7, OS id=25760
MMAN started with pid=8, OS id=25770
DBW0 started with pid=9, OS id=25781
LGWR started with pid=10, OS id=25793
CKPT started with pid=11, OS id=25801
SMON started with pid=12, OS id=25804
RBAL started with pid=13, OS id=25806
GMON started with pid=14, OS id=25808
Sat Jul 26 09:16:02 2008
lmon registered with NM - instance id 1 (internal mem no 0)
Sat Jul 26 09:17:42 2008
sculkget: failed to lock /oracle/app/product/10.0.0.2/dbs/lkinst+ASM1 exclusive
sculkget: lock held by PID: 25748
Oracle Instance Startup operation failed. Another process may be attempting to startup or shutdown this Instance.
Failed to acquire instance startup/shutdown serialization primitive
Sat Jul 26 09:17:53 2008
sculkget: failed to lock /oracle/app/product/10.0.0.2/dbs/lkinst+ASM1 exclusive
sculkget: lock held by PID: 25748
Oracle Instance Startup operation failed. Another process may be attempting to startup or shutdown this Instance.
Failed to acquire instance startup/shutdown serialization primitive
Sat Jul 26 09:19:13 2008
Error: KGXGN polling error (15)
Sat Jul 26 09:19:13 2008
Errors in file /oracle/app/admin/+ASM/bdump/+asm1_lmon_25756.trc:
ORA-29702: error occurred in Cluster Group Service operation
LMON: terminating instance due to error 29702
Sat Jul 26 09:19:14 2008
Starting ORACLE instance (normal)
Sat Jul 26 09:19:14 2008
Shutting down instance (abort)
License high water mark = 1
Sat Jul 26 09:19:17 2008
Instance terminated by LMON, pid = 25756
Sat Jul 26 09:19:19 2008
Instance terminated by USER, pid = 31339
Sat Jul 26 09:19:14 2008
Starting ORACLE instance (normal)
Sat Jul 26 09:19:14 2008
Shutting down instance (abort)
License high water mark = 1
Sat Jul 26 09:19:17 2008
Instance terminated by LMON, pid = 25756
Sat Jul 26 09:19:19 2008
Instance terminated by USER, pid = 31339
Sat Jul 26 09:19:51 2008
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
WARNING: No cluster interconnect has been specified. Depending on
the communication driver configured Oracle cluster traffic
may be directed to the public interface of this machine.
Oracle recommends that RAC clustered databases be configured
with a private interconnect for enhanced security and
performance.
Sat Jul 26 09:19:14 2008
Shutting down instance (abort)
License high water mark = 1
Sat Jul 26 09:19:17 2008
Instance terminated by LMON, pid = 25756
Sat Jul 26 09:19:19 2008
Instance terminated by USER, pid = 31339
Sat Jul 26 09:19:51 2008
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
WARNING: No cluster interconnect has been specified. Depending on
the communication driver configured Oracle cluster traffic
may be directed to the public interface of this machine.
Oracle recommends that RAC clustered databases be configured
with a private interconnect for enhanced security and
performance.
Picked latch-free SCN scheme 2
Using LOG_ARCHIVE_DEST_1 parameter default value as /oracle/app/product/10.0.0.2/dbs/arch
Autotune of undo retention is turned off.
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.1.0.
System parameters with non-default values:
large_pool_size
= 12582912
instance_type
= asm
cluster_database = TRUE
instance_number
= 1
remote_login_passwordfile= EXCLUSIVE
background_dump_dest = /oracle/app/admin/+ASM/bdump
user_dump_dest = /oracle/app/admin/+ASM/udump
core_dump_dest = /oracle/app/admin/+ASM/cdump
asm_diskgroups =
Cluster communication is configured to use the following interface(s) for this instance
127.0.0.1
Sat Jul 26 09:19:51 2008
cluster interconnect IPC version:Oracle UDP/IP
IPC Vendor 1 proto 2
PMON started with pid=2, OS id=32536
DIAG started with pid=3, OS id=32538
PSP0 started with pid=4, OS id=32540
LMON started with pid=5, OS id=32542
LMD0 started with pid=6, OS id=32544
LMS0 started with pid=7, OS id=32546
MMAN started with pid=8, OS id=32556
DBW0 started with pid=9, OS id=32558
LGWR started with pid=10, OS id=32560
CKPT started with pid=11, OS id=32562
SMON started with pid=12, OS id=32564
RBAL started with pid=13, OS id=32576
GMON started with pid=14, OS id=32588
Sat Jul 26 09:19:52 2008
lmon registered with NM - instance id 1 (internal mem no 0)
Sat Jul 26 09:19:52 2008
Reconfiguration started (old inc 0, new inc 1)
ASM instance
List of nodes:
0
Global Resource Directory frozen
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Sat Jul 26 09:19:52 2008
LMS 0: 0 GCS shadows cancelled, 0 closed
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Post SMON to start 1st pass IR
Sat Jul 26 09:19:52 2008
LMS 0: 0 GCS shadows traversed, 0 replayed
Sat Jul 26 09:19:52 2008
Submitted all GCS remote-cache requests
Post SMON to start 1st pass IR
Fix write in gcs resources
Reconfiguration complete
LCK0 started with pid=15, OS id=32595
Sat Jul 26 09:19:53 2008
SQL> ALTER DISKGROUP ALL MOUNT
Sat Jul 26 09:19:53 2008
ALTER SYSTEM SET asm_diskgroups='' SCOPE=MEMORY;
Sat Jul 26 09:22:53 2008
Error: KGXGN polling error (15)
Sat Jul 26 09:22:53 2008
Errors in file /oracle/app/admin/+ASM/bdump/+asm1_lmon_32542.trc:
ORA-29702: error occurred in Cluster Group Service operation
LMON: terminating instance due to error 29702
Sat Jul 26 09:22:53 2008
Dump system state for local instance only
System State dumped to trace file /oracle/app/admin/+ASM/bdump/+asm1_diag_32538.trc
Sat Jul 26 09:22:53 2008
Trace dumping is performing id=[cdmp_20080726092253]
Sat Jul 26 09:22:54 2008
Shutting down instance (abort)
License high water mark = 2
Sat Jul 26 09:22:58 2008
Instance terminated by LMON, pid = 32542
Sat Jul 26 09:22:59 2008
Instance terminated by USER, pid = 4871
Sat Jul 26 09:23:00 2008
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
WARNING: No cluster interconnect has been specified. Depending on
the communication driver configured Oracle cluster traffic
may be directed to the public interface of this machine.
Oracle recommends that RAC clustered databases be configured
with a private interconnect for enhanced security and
performance.
Picked latch-free SCN scheme 2
Using LOG_ARCHIVE_DEST_1 parameter default value as /oracle/app/product/10.0.0.2/dbs/arch
Autotune of undo retention is turned off.
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.1.0.
System parameters with non-default values:
large_pool_size
= 12582912
instance_type
= asm
cluster_database = TRUE
instance_number
= 1
remote_login_passwordfile= EXCLUSIVE
background_dump_dest = /oracle/app/admin/+ASM/bdump
user_dump_dest = /oracle/app/admin/+ASM/udump
core_dump_dest = /oracle/app/admin/+ASM/cdump
asm_diskgroups =
Cluster communication is configured to use the following interface(s) for this instance
127.0.0.1
Sat Jul 26 09:23:01 2008
cluster interconnect IPC version:Oracle UDP/IP
IPC Vendor 1 proto 2
PMON started with pid=2, OS id=5044
DIAG started with pid=3, OS id=5046
PSP0 started with pid=4, OS id=5048
LMON started with pid=5, OS id=5050
LMD0 started with pid=6, OS id=5052
LMS0 started with pid=7, OS id=5054
MMAN started with pid=8, OS id=5064
DBW0 started with pid=9, OS id=5066
LGWR started with pid=10, OS id=5078
CKPT started with pid=11, OS id=5090
SMON started with pid=12, OS id=5097
RBAL started with pid=13, OS id=5099
GMON started with pid=14, OS id=5101
Sat Jul 26 09:23:01 2008
lmon registered with NM - instance id 1 (internal mem no 0)
Sat Jul 26 09:23:57 2008
sculkget: failed to lock /oracle/app/product/10.0.0.2/dbs/lkinst+ASM1 exclusive
sculkget: lock held by PID: 5042
Oracle Instance Startup operation failed. Another process may be attempting to startup or shutdown this Instance.
Failed to acquire instance startup/shutdown serialization primitive
Sat Jul 26 09:24:49 2008
sculkget: failed to lock /oracle/app/product/10.0.0.2/dbs/lkinst+ASM1 exclusive
sculkget: lock held by PID: 5042
Oracle Instance Startup operation failed. Another process may be attempting to startup or shutdown this Instance.
Failed to acquire instance startup/shutdown serialization primitive
Sat Jul 26 09:25:59 2008
Error: KGXGN aborts the instance (6)
Sat Jul 26 09:25:59 2008
Errors in file /oracle/app/admin/+ASM/bdump/+asm1_lmon_5050.trc:
ORA-29702: error occurred in Cluster Group Service operation
LMON: terminating instance due to error 29702
Sat Jul 26 09:25:59 2008
Errors in file /oracle/app/admin/+ASM/bdump/+asm1_lmd0_5052.trc:
ORA-29702: error occurred in Cluster Group Service operation
rac2 的实例
GMON started with pid=14, OS id=11941
Sat Jul 26 09:12:56 2008
lmon registered with NM - instance id 2 (internal mem no 1)
Sat Jul 26 09:16:01 2008
Reconfiguration started (old inc 0, new inc 1)
ASM instance
List of nodes:
1
Global Resource Directory frozen
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Sat Jul 26 09:16:01 2008
LMS 0: 0 GCS shadows cancelled, 0 closed
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Post SMON to start 1st pass IR
Sat Jul 26 09:16:01 2008
LMS 0: 0 GCS shadows traversed, 0 replayed
Sat Jul 26 09:16:01 2008
Submitted all GCS remote-cache requests
Post SMON to start 1st pass IR
Fix write in gcs resources
Reconfiguration complete
LCK0 started with pid=15, OS id=16544
Sat Jul 26 09:16:02 2008
SQL> ALTER DISKGROUP ALL MOUNT
Sat Jul 26 09:19:13 2008
Error: KGXGN polling error (15)
Sat Jul 26 09:19:13 2008
Errors in file /oracle/app/admin/+ASM/bdump/+asm2_lmon_11915.trc:
ORA-29702: error occurred in Cluster Group Service operation
LMON: terminating instance due to error 29702
Sat Jul 26 09:19:13 2008
Dump system state for local instance only
System State dumped to trace file /oracle/app/admin/+ASM/bdump/+asm2_diag_11911.trc
Sat Jul 26 09:19:13 2008
Trace dumping is performing id=[cdmp_20080726091913]
Sat Jul 26 09:19:13 2008
Shutting down instance (abort)
License high water mark = 1
Sat Jul 26 09:19:17 2008
Instance terminated by LMON, pid = 11915
Sat Jul 26 09:19:18 2008
Instance terminated by USER, pid = 22171
Sat Jul 26 09:19:19 2008
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
WARNING: No cluster interconnect has been specified. Depending on
the communication driver configured Oracle cluster traffic
may be directed to the public interface of this machine.
Oracle recommends that RAC clustered databases be configured
with a private interconnect for enhanced security and
performance.
Picked latch-free SCN scheme 2
Using LOG_ARCHIVE_DEST_1 parameter default value as /oracle/app/product/10.0.0.2/dbs/arch
Autotune of undo retention is turned off.
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.1.0.
System parameters with non-default values:
large_pool_size
= 12582912
instance_type
= asm
cluster_database = TRUE
instance_number
= 2
remote_login_passwordfile= EXCLUSIVE
background_dump_dest = /oracle/app/admin/+ASM/bdump
user_dump_dest = /oracle/app/admin/+ASM/udump
core_dump_dest = /oracle/app/admin/+ASM/cdump
asm_diskgroups =
Cluster communication is configured to use the following interface(s) for this instance
127.0.0.1
Sat Jul 26 09:19:20 2008
cluster interconnect IPC version:Oracle UDP/IP
IPC Vendor 1 proto 2
PMON started with pid=2, OS id=22440
DIAG started with pid=3, OS id=22442
PSP0 started with pid=4, OS id=22444
LMON started with pid=5, OS id=22446
LMD0 started with pid=6, OS id=22448
LMS0 started with pid=7, OS id=22450
MMAN started with pid=8, OS id=22460
DBW0 started with pid=9, OS id=22462
LGWR started with pid=10, OS id=22464
CKPT started with pid=11, OS id=22466
SMON started with pid=12, OS id=22468
RBAL started with pid=13, OS id=22470
GMON started with pid=14, OS id=22472
Sat Jul 26 09:19:20 2008
lmon registered with NM - instance id 2 (internal mem no 1)
Sat Jul 26 09:19:20 2008
Reconfiguration started (old inc 0, new inc 1)
ASM instance
List of nodes:
1
Global Resource Directory frozen
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Sat Jul 26 09:19:20 2008
LMS 0: 0 GCS shadows cancelled, 0 closed
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Post SMON to start 1st pass IR
Sat Jul 26 09:19:20 2008
LMS 0: 0 GCS shadows traversed, 0 replayed
Sat Jul 26 09:19:20 2008
Submitted all GCS remote-cache requests
Post SMON to start 1st pass IR
Fix write in gcs resources
Reconfiguration complete
LCK0 started with pid=15, OS id=22499
Sat Jul 26 09:19:21 2008
SQL> ALTER DISKGROUP ALL MOUNT
Sat Jul 26 09:19:39 2008
Shutting down instance: further logons disabled
Sat Jul 26 09:19:39 2008
Shutting down instance (immediate)
License high water mark = 1
Sat Jul 26 09:19:39 2008
SQL> ALTER DISKGROUP ALL DISMOUNT
Sat Jul 26 09:19:54 2008
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
WARNING: No cluster interconnect has been specified. Depending on
the communication driver configured Oracle cluster traffic
may be directed to the public interface of this machine.
Oracle recommends that RAC clustered databases be configured
with a private interconnect for enhanced security and
performance.
Picked latch-free SCN scheme 2
Using LOG_ARCHIVE_DEST_1 parameter default value as /oracle/app/product/10.0.0.2/dbs/arch
Autotune of undo retention is turned off.
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.1.0.
System parameters with non-default values:
large_pool_size
= 12582912
instance_type
= asm
cluster_database = TRUE
instance_number
= 2
remote_login_passwordfile= EXCLUSIVE
background_dump_dest = /oracle/app/admin/+ASM/bdump
user_dump_dest = /oracle/app/admin/+ASM/udump
core_dump_dest = /oracle/app/admin/+ASM/cdump
asm_diskgroups =
Cluster communication is configured to use the following interface(s) for this instance
127.0.0.1
Sat Jul 26 09:19:54 2008
cluster interconnect IPC version:Oracle UDP/IP
IPC Vendor 1 proto 2
PMON started with pid=2, OS id=23830
DIAG started with pid=3, OS id=23832
PSP0 started with pid=4, OS id=23834
LMON started with pid=5, OS id=23836
LMD0 started with pid=6, OS id=23838
LMS0 started with pid=7, OS id=23840
MMAN started with pid=8, OS id=23850
DBW0 started with pid=9, OS id=23852
LGWR started with pid=10, OS id=23854
CKPT started with pid=11, OS id=23856
SMON started with pid=12, OS id=23858
RBAL started with pid=13, OS id=23860
GMON started with pid=14, OS id=23862
Sat Jul 26 09:19:55 2008
lmon registered with NM - instance id 2 (internal mem no 1)
Sat Jul 26 09:22:59 2008
Reconfiguration started (old inc 0, new inc 1)
ASM instance
List of nodes:
1
Global Resource Directory frozen
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Sat Jul 26 09:23:00 2008
LMS 0: 0 GCS shadows cancelled, 0 closed
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Post SMON to start 1st pass IR
Sat Jul 26 09:23:00 2008
LMS 0: 0 GCS shadows traversed, 0 replayed
Sat Jul 26 09:23:00 2008
Submitted all GCS remote-cache requests
Post SMON to start 1st pass IR
Fix write in gcs resources
Reconfiguration complete
LCK0 started with pid=15, OS id=28449
Sat Jul 26 09:23:00 2008
SQL> ALTER DISKGROUP ALL MOUNT
Sat Jul 26 09:26:08 2008
Error: KGXGN aborts the instance (6)
Sat Jul 26 09:26:08 2008
Errors in file /oracle/app/admin/+ASM/bdump/+asm2_lmon_23836.trc:
ORA-29702: error occurred in Cluster Group Service operation
LMON: terminating instance due to error 29702
Sat Jul 26 09:26:08 2008
Dump system state for local instance only
System State dumped to trace file /oracle/app/admin/+ASM/bdump/+asm2_diag_23832.trc
Sat Jul 26 09:26:08 2008
Trace dumping is performing id=[cdmp_20080726092608]
Sat Jul 26 09:30:47 2008
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
WARNING: No cluster interconnect has been specified. Depending on
the communication driver configured Oracle cluster traffic
may be directed to the public interface of this machine.
Oracle recommends that RAC clustered databases be configured
LMON started with pid=5, OS id=23836
LMD0 started with pid=6, OS id=23838
LMS0 started with pid=7, OS id=23840
MMAN started with pid=8, OS id=23850
DBW0 started with pid=9, OS id=23852
LGWR started with pid=10, OS id=23854
CKPT started with pid=11, OS id=23856
SMON started with pid=12, OS id=23858
RBAL started with pid=13, OS id=23860
GMON started with pid=14, OS id=23862
Sat Jul 26 09:19:55 2008
lmon registered with NM - instance id 2 (internal mem no 1)
Sat Jul 26 09:22:59 2008
Reconfiguration started (old inc 0, new inc 1)
ASM instance
List of nodes:
1
Global Resource Directory frozen
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Sat Jul 26 09:23:00 2008
LMS 0: 0 GCS shadows cancelled, 0 closed
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Post SMON to start 1st pass IR
Sat Jul 26 09:23:00 2008
LMS 0: 0 GCS shadows traversed, 0 replayed
Sat Jul 26 09:23:00 2008
Submitted all GCS remote-cache requests
Post SMON to start 1st pass IR
Fix write in gcs resources
Reconfiguration complete
LCK0 started with pid=15, OS id=28449
Sat Jul 26 09:23:00 2008
SQL> ALTER DISKGROUP ALL MOUNT
Sat Jul 26 09:26:08 2008
Error: KGXGN aborts the instance (6)
Sat Jul 26 09:26:08 2008
Errors in file /oracle/app/admin/+ASM/bdump/+asm2_lmon_23836.trc:
ORA-29702: error occurred in Cluster Group Service operation
LMON: terminating instance due to error 29702
Sat Jul 26 09:26:08 2008
Dump system state for local instance only
System State dumped to trace file /oracle/app/admin/+ASM/bdump/+asm2_diag_23832.trc
Sat Jul 26 09:26:08 2008
Trace dumping is performing id=[cdmp_20080726092608]
[ 本帖最后由 heming96 于 2008-7-26 10:22 编辑 ]
回复

使用道具 举报

千问 | 2008-7-26 09:30:48 | 显示全部楼层
两台机器都重启之后会自动尝试重启ASM实例,rac1能启动,rac2启动不了 ,日志分别是 :
rac1 :
Sat Jul 26 09:30:19 2008
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
WARNING: No cluster interconnect has been specified. Depending on
the communication driver configured Oracle cluster traffic
may be directed to the public interface of this machine.
Oracle recommends that RAC clustered databases be configured
with a private interconnect for enhanced security and
performance.
Picked latch-free SCN scheme 2
Using LOG_ARCHIVE_DEST_1 parameter default value as /oracle/app/product/10.0.0.2/dbs/arch
Autotune of undo retention is turned off.
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.1.0.
System parameters with non-default values:
large_pool_size
= 12582912
instance_type
= asm
cluster_database = TRUE
instance_number
= 1
remote_login_passwordfile= EXCLUSIVE
background_dump_dest = /oracle/app/admin/+ASM/bdump
user_dump_dest = /oracle/app/admin/+ASM/udump
core_dump_dest = /oracle/app/admin/+ASM/cdump
asm_diskgroups =
Cluster communication is configured to use the following interface(s) for this instance
127.0.0.1
Sat Jul 26 09:30:20 2008
cluster interconnect IPC version:Oracle UDP/IP
IPC Vendor 1 proto 2
PSP0 started with pid=4, OS id=7058
PMON started with pid=2, OS id=7054
DIAG started with pid=3, OS id=7056
LMON started with pid=5, OS id=7060
LMD0 started with pid=6, OS id=7062
LMS0 started with pid=7, OS id=7064
MMAN started with pid=8, OS id=7074
DBW0 started with pid=9, OS id=7076
LGWR started with pid=10, OS id=7078
CKPT started with pid=11, OS id=7080
SMON started with pid=12, OS id=7082
RBAL started with pid=13, OS id=7084
GMON started with pid=14, OS id=7086
Sat Jul 26 09:30:20 2008
lmon registered with NM - instance id 1 (internal mem no 0)
Sat Jul 26 09:30:21 2008
Reconfiguration started (old inc 0, new inc 1)
ASM instance
List of nodes:
0
Global Resource Directory frozen
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Sat Jul 26 09:30:21 2008
LMS 0: 0 GCS shadows cancelled, 0 closed
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Post SMON to start 1st pass IR
Sat Jul 26 09:30:21 2008
LMS 0: 0 GCS shadows traversed, 0 replayed
Sat Jul 26 09:30:21 2008
Submitted all GCS remote-cache requests
Post SMON to start 1st pass IR
Fix write in gcs resources
Reconfiguration complete
LCK0 started with pid=15, OS id=7113
Sat Jul 26 09:30:22 2008
SQL> ALTER DISKGROUP ALL MOUNT
Sat Jul 26 09:34:04 2008
Reconfiguration started (old inc 1, new inc 2)
List of nodes:
0
Global Resource Directory frozen
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Sat Jul 26 09:34:04 2008
LMS 0: 0 GCS shadows cancelled, 0 closed
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Sat Jul 26 09:34:04 2008
LMS 0: 0 GCS shadows traversed, 0 replayed
Sat Jul 26 09:34:04 2008
Submitted all GCS remote-cache requests
Post SMON to start 1st pass IR
Fix write in gcs resources
Reconfiguration complete
---------------------------------------
rac2:
Sat Jul 26 09:30:47 2008
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
WARNING: No cluster interconnect has been specified. Depending on
the communication driver configured Oracle cluster traffic
may be directed to the public interface of this machine.
Oracle recommends that RAC clustered databases be configured
with a private interconnect for enhanced security and
performance.
Picked latch-free SCN scheme 2
Using LOG_ARCHIVE_DEST_1 parameter default value as /oracle/app/product/10.0.0.2/dbs/arch
Autotune of undo retention is turned off.
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.1.0.
System parameters with non-default values:
large_pool_size
= 12582912
instance_type
= asm
cluster_database = TRUE
instance_number
= 2
remote_login_passwordfile= EXCLUSIVE
background_dump_dest = /oracle/app/admin/+ASM/bdump
user_dump_dest = /oracle/app/admin/+ASM/udump
core_dump_dest = /oracle/app/admin/+ASM/cdump
asm_diskgroups =
Cluster communication is configured to use the following interface(s) for this instance
127.0.0.1
Sat Jul 26 09:30:48 2008
cluster interconnect IPC version:Oracle UDP/IP
IPC Vendor 1 proto 2
PSP0 started with pid=4, OS id=7375
PMON started with pid=2, OS id=7371
DIAG started with pid=3, OS id=7373
LMON started with pid=5, OS id=7377
LMD0 started with pid=6, OS id=7379
LMS0 started with pid=7, OS id=7381
MMAN started with pid=8, OS id=7391
DBW0 started with pid=9, OS id=7393
LGWR started with pid=10, OS id=7395
CKPT started with pid=11, OS id=7397
SMON started with pid=12, OS id=7399
RBAL started with pid=13, OS id=7401
GMON started with pid=14, OS id=7403
Sat Jul 26 09:30:48 2008
lmon registered with NM - instance id 2 (internal mem no 1)
Sat Jul 26 09:33:59 2008
Error: KGXGN polling error (15)
Sat Jul 26 09:33:59 2008
Errors in file /oracle/app/admin/+ASM/bdump/+asm2_lmon_7377.trc:
ORA-29702: error occurred in Cluster Group Service operation
LMON: terminating instance due to error 29702
Instance terminated by LMON, pid = 7377
------------------------------------------------跟踪文件
[oracle@rac2 bdump]$ more /oracle/app/admin/+ASM/bdump/+asm2_lmon_7377.trc
/oracle/app/admin/+ASM/bdump/+asm2_lmon_7377.trc
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, Real Application Clusters, OLAP and Data Mining options
ORACLE_HOME = /oracle/app/product/10.0.0.2
System name:Linux
Node name:rac2
Release:2.6.9-42.ELsmp
Version:#1 SMP Wed Jul 12 23:27:17 EDT 2006
Machine:i686
Instance name: +ASM2
Redo thread mounted by this instance: 0
Oracle process number: 5
Unix process pid: 7377, image: oracle@rac2 (LMON)
*** SERVICE NAME

) 2008-07-26 09:30:48.733
*** SESSION ID

46.1) 2008-07-26 09:30:48.733
GES resources 1890 pool 8
GES enqueues 2510
GES IPC: Receivers 2Senders 2
GES IPC: BuffersReceive 1000Send (i:830 b:830) Reserve 81
GES IPC: Msg SizeRegular 388Batch 4096
Batching factor: enqueue replay 99, ack 110
Batching factor: cache replay 71 size per lock 56
kjxggin: receive buffer size = 32768
*** 2008-07-26 09:30:48.968
kjxgmrcfg: Reconfiguration started, reason 1
kjxgmcs: Setting state to 0 0.
*** 2008-07-26 09:30:48.968
Name Service frozen
kjxgmcs: Setting state to 0 1.
kjfcpiora: published my fusion master weight 5622
kjfcpiora: publish my flogb 9
*** 2008-07-26 09:33:59.474
=====================================================
kjxgmpoll: CGS state (0 1) start 0xea042bd4 cur 0xf51aed71 diff 0xb16c19d
=====================================================
Group name: +ASM
Member id: 1
Cached KGXGN event: 0
Group State:
State: 0 1
Reconfig startedstart-tm 0xea042ba9tmout period 0x7bfa480
Commited Map: 0 1
New Map: 0 1
KGXGN Map: 0 1
Master node: 0
Memcnt 0Rcvcnt 0
Substate Proposal: false
Inc Proposal:
incarn 1memcnt 2master 0
proposal truematched false
map: 0 1
Master Inc State:
incarn 0memcnt 0agrees 0flag 0x1
wmap:
nmap:
ubmap:
Substate Handler Execution State
substate 0status done
substate 1status done
substate 2status done
substate 3status done
substate 4status done
substate 5status done
substate 6status done
kjxgmpoll: terminate the CGS reconfig.
Error: KGXGN polling error (15)
error 29702 detected in background process
ORA-29702: error occurred in Cluster Group Service operation
ksuitm: waiting for [5] seconds before killing DIAG
[oracle@rac2 bdump]$
回复

使用道具 举报

千问 | 2008-7-26 09:30:48 | 显示全部楼层
“WARNING: No cluster interconnect has been specified”
cluster设置有问题?网卡?配置?
回复

使用道具 举报

千问 | 2008-7-26 09:30:48 | 显示全部楼层
ClusterWare 都装好了, 数据库软件也都装完了,建库的时候也都是可以选双节点,ASM实例在两个节点启动,只是感觉这两个节点总是冲突
其中一个启动另一个就启动不了。
回复

使用道具 举报

千问 | 2008-7-26 09:30:48 | 显示全部楼层
换成11G ,很轻松就装好了 。
可能是当时装10g的时候用户的gid是dba而不是 oinstall ,当然还有少了个软件包 unixODBC-devel 的原因。
回复

使用道具 举报

千问 | 2008-7-26 09:30:48 | 显示全部楼层
Errors in file /oracle/app/admin/+ASM/bdump/+asm1_lmon_20810.trc:
ORA-29702: error occurred in Cluster Group Service operation
LMON: terminating instance due to error 29702
从这里看,是CSS的问题导致ASM的问题,看看CSS的log
另外,11g,oinstall是oracle user的primray group, for 10.2 and below, dba is the primary group for oracle user.
回复

使用道具 举报

千问 | 2008-7-26 09:30:48 | 显示全部楼层
分别ping一下2个节点的Priate地址,看是否2个节点的Private地址是否能够ping通
回复

使用道具 举报

千问 | 2008-7-26 09:30:48 | 显示全部楼层
关于此问题本人遇到与你一样。
google,baidu,所有贴子看了一遍。没有解决答案。
看了rac trouble shooting:得出结果:
现象:
[oracle@db1 css]$ crs_stat -t
Name Type TargetState Host
------------------------------------------------------------
ora....SM1.asm applicationONLINEONLINEdb1
ora....B1.lsnr applicationONLINEONLINEdb1
ora.db1.gsdapplicationONLINEONLINEdb1
ora.db1.onsapplicationONLINEONLINEdb1
ora.db1.vipapplicationONLINEONLINEdb1
ora....SM2.asm applicationONLINEOFFLINE

ora....B2.lsnr applicationONLINEONLINEdb2
ora.db2.gsdapplicationONLINEONLINEdb2
ora.db2.onsapplicationONLINEONLINEdb2
ora.db2.vipapplicationONLINEONLINEdb2
[oracle@db1 css]$ srvctl start asm -n db2
PRKS-1009 : Failed to start ASM instance "+ASM2" on node "db2", [PRKS-1009 : Failed to start ASM instance "+ASM2" on node "db2", [CRS-0215: Could not start resource 'ora.db2.ASM2.asm'.]]
[PRKS-1009 : Failed to start ASM instance "+ASM2" on node "db2", [CRS-0215: Could not start resource 'ora.db2.ASM2.asm'.]]
日志报错也看了,如你一样:
Tue Apr 21 21:45:29 2009
Error: KGXGN polling error (15)
Tue Apr 21 21:45:29 2009
Errors in file /oracle/oracle/admin/+ASM/bdump/+asm2_lmon_9354.trc:
ORA-29702: error occurred in Cluster Group Service operation
LMON: terminating instance due to error 29702

[oracle@db2 bdump]$ cat /oracle/oracle/admin/+ASM/bdump/+asm2_lmon_9354.trc:
cat: /oracle/oracle/admin/+ASM/bdump/+asm2_lmon_9354.trc:: No such file or directory
[oracle@db2 bdump]$ cat /oracle/oracle/admin/+ASM/bdump/+asm2_lmon_9354.trc
/oracle/oracle/admin/+ASM/bdump/+asm2_lmon_9354.trc
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, Real Application Clusters, OLAP and Data Mining options
ORACLE_HOME = /oracle/oracle/oracle/product/10.2.0/db_1
System name:Linux
Node name:db2.zctt.com
Release:2.6.9-78.ELsmp
Version:#1 SMP Wed Jul 9 15:39:47 EDT 2008
Machine:i686
Instance name: +ASM2
Redo thread mounted by this instance: 0
Oracle process number: 5
Unix process pid: 9354, image: [email protected] (LMON)
*** SERVICE NAME

) 2009-04-21 21:42:17.996
*** SESSION ID

46.1) 2009-04-21 21:42:17.996
GES resources 1890 pool 3
GES enqueues 2510
GES IPC: Receivers 2Senders 2
GES IPC: BuffersReceive 1000Send (i:830 b:830) Reserve 81
GES IPC: Msg SizeRegular 388Batch 4096
Batching factor: enqueue replay 99, ack 110
Batching factor: cache replay 71 size per lock 56
kjxggin: receive buffer size = 32768
*** 2009-04-21 21:42:19.185
kjxgmrcfg: Reconfiguration started, reason 1
kjxgmcs: Setting state to 0 0.
*** 2009-04-21 21:42:19.185
Name Service frozen
kjxgmcs: Setting state to 0 1.
kjfcpiora: published my fusion master weight 5622
kjfcpiora: publish my flogb 9
*** 2009-04-21 21:45:29.601
=====================================================
kjxgmpoll: CGS state (0 1) start 0x6bef5dfb cur 0x7704cc4e diff 0xb156e53
=====================================================
Group name: +ASM
Member id: 1
Cached KGXGN event: 0
Group State:
State: 0 1
Reconfig startedstart-tm 0x6bef5dadtmout period 0x7bfa480
Commited Map: 0 1
New Map: 0 1
KGXGN Map: 0 1
Master node: 0
Memcnt 0Rcvcnt 0
Substate Proposal: false
Inc Proposal:
incarn 1memcnt 2master 0
proposal truematched false
map: 0 1
Master Inc State:
incarn 0memcnt 0agrees 0flag 0x1
wmap:
nmap:
ubmap:
Substate Handler Execution State
substate 0status done
substate 1status done
substate 2status done
substate 3status done
substate 4status done
substate 5status done
substate 6status done
kjxgmpoll: terminate the CGS reconfig.
Error: KGXGN polling error (15)
error 29702 detected in background process
ORA-29702: error occurred in Cluster Group Service operation
ksuitm: waiting for [5] seconds before killing DIAG
解决方法:
1. oifcfg getif 没有结果。
2. 进行set:
[oracle@db2 bin]$ oifcfg setif -global eth1/10.10.100.0:cluster_interconnect
[oracle@db2 bin]$ oifcfg setif -global eth0/211.110.100.0

ublic
[oracle@db2 bin]$ oifcfg getif
eth110.10.100.0globalcluster_interconnect
eth0211.110.100.0globalpublic
3. $crs_stop -all;
$crs_start -all;
结果:
[oracle@db1 bdump]$ crs_stat -t
Name Type TargetState Host
------------------------------------------------------------
ora....SM1.asm applicationONLINEONLINEdb1
ora....B1.lsnr applicationONLINEONLINEdb1
ora.db1.gsdapplicationONLINEONLINEdb1
ora.db1.onsapplicationONLINEONLINEdb1
ora.db1.vipapplicationONLINEONLINEdb1
ora....SM2.asm applicationONLINEONLINEdb2
ora....B2.lsnr applicationONLINEONLINEdb2
ora.db2.gsdapplicationONLINEONLINEdb2
ora.db2.onsapplicationONLINEONLINEdb2
ora.db2.vipapplicationONLINEONLINEdb2
[oracle@db1 bdump]$ crs_stop -all
Attempting to stop `ora.db1.gsd` on member `db1`
Attempting to stop `ora.db1.ons` on member `db1`
Attempting to stop `ora.db2.gsd` on member `db2`
Attempting to stop `ora.db2.ons` on member `db2`
Stop of `ora.db1.gsd` on member `db1` succeeded.
Stop of `ora.db1.ons` on member `db1` succeeded.
Stop of `ora.db2.gsd` on member `db2` succeeded.
Stop of `ora.db2.ons` on member `db2` succeeded.
Attempting to stop `ora.db1.LISTENER_DB1.lsnr` on member `db1`
Attempting to stop `ora.db2.LISTENER_DB2.lsnr` on member `db2`
Stop of `ora.db1.LISTENER_DB1.lsnr` on member `db1` succeeded.
Attempting to stop `ora.db1.ASM1.asm` on member `db1`
Stop of `ora.db2.LISTENER_DB2.lsnr` on member `db2` succeeded.
Attempting to stop `ora.db2.ASM2.asm` on member `db2`
Stop of `ora.db1.ASM1.asm` on member `db1` succeeded.
Attempting to stop `ora.db1.vip` on member `db1`
Stop of `ora.db2.ASM2.asm` on member `db2` succeeded.
Stop of `ora.db1.vip` on member `db1` succeeded.
Attempting to stop `ora.db2.vip` on member `db2`
Stop of `ora.db2.vip` on member `db2` succeeded.
[oracle@db1 bdump]$ crs_start -all
Attempting to start `ora.db1.vip` on member `db1`
Attempting to start `ora.db2.vip` on member `db2`
Start of `ora.db1.vip` on member `db1` succeeded.
Attempting to start `ora.db1.ASM1.asm` on member `db1`
Start of `ora.db2.vip` on member `db2` succeeded.
Attempting to start `ora.db2.ASM2.asm` on member `db2`
Start of `ora.db1.ASM1.asm` on member `db1` succeeded.
Attempting to start `ora.db1.LISTENER_DB1.lsnr` on member `db1`
Start of `ora.db1.LISTENER_DB1.lsnr` on member `db1` succeeded.
Start of `ora.db2.ASM2.asm` on member `db2` succeeded.
Attempting to start `ora.db2.LISTENER_DB2.lsnr` on member `db2`
Start of `ora.db2.LISTENER_DB2.lsnr` on member `db2` succeeded.
CRS-1002: Resource 'ora.db1.ons' is already running on member 'db1'
CRS-1002: Resource 'ora.db2.ons' is already running on member 'db2'
Attempting to start `ora.db1.gsd` on member `db1`
Attempting to start `ora.db2.gsd` on member `db2`
Start of `ora.db1.gsd` on member `db1` succeeded.
Start of `ora.db2.gsd` on member `db2` succeeded.
CRS-0223: Resource 'ora.db1.ons' has placement error.
CRS-0223: Resource 'ora.db2.ons' has placement error.
[oracle@db1 bdump]$ crs_stat -t
Name Type TargetState Host
------------------------------------------------------------
ora....SM1.asm applicationONLINEONLINEdb1
ora....B1.lsnr applicationONLINEONLINEdb1
ora.db1.gsdapplicationONLINEONLINEdb1
ora.db1.onsapplicationONLINEONLINEdb1
ora.db1.vipapplicationONLINEONLINEdb1
ora....SM2.asm applicationONLINEONLINEdb2
ora....B2.lsnr applicationONLINEONLINEdb2
ora.db2.gsdapplicationONLINEONLINEdb2
ora.db2.onsapplicationONLINEONLINEdb2
ora.db2.vipapplicationONLINEONLINEdb2
[oracle@db1 bdump]$
[ 本帖最后由 sudaba 于 2009-4-21 22:49 编辑 ]
回复

使用道具 举报

千问 | 2008-7-26 09:30:48 | 显示全部楼层
OK,Mark
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

主题

0

回帖

4882万

积分

论坛元老

Rank: 8Rank: 8

积分
48824836
热门排行