我这里有告警日志请大家帮忙诊断一下是什么情况？

显示全部楼层 · 2009-11-6 23:12:33

我取出的log片段如下：
库的存储磁盘阵列配置和性能没有检查
库I/O问题异常突出

Wed Jan 12 14:36:29 2011
Thread 1 advanced to log sequence 33564 (LGWR switch)
Current log# 15 seq# 33564 mem# 0: /oracle/product/oradata/xxxxx/redo15.log
Wed Jan 12 14:41:14 2011
Error 2068 trapped in 2PC on transaction 8.4.126440. Cleaning up.
Error stack returned to user:
ORA-02068: 以下严重错误源于 ZJ_72
ORA-03113: 通信通道的文件结束
Wed Jan 12 14:45:25 2011
WARNING: inbound connection timed out (ORA-3136)
Wed Jan 12 14:58:50 2011
Thread 1 advanced to log sequence 33565 (LGWR switch)
Current log# 16 seq# 33565 mem# 0: /oracle/product/oradata/xxxxx/redo16.log
Wed Jan 12 15:17:08 2011
Thread 1 advanced to log sequence 33566 (LGWR switch)
Current log# 14 seq# 33566 mem# 0: /oracle/product/oradata/xxxxx/redo14.log
Wed Jan 12 15:21:39 2011
Thread 1 cannot allocate new log, sequence 33567
Private strand flush not complete
Current log# 14 seq# 33566 mem# 0: /oracle/product/oradata/xxxxx/redo14.log
Wed Jan 12 15:22:29 2011
Thread 1 advanced to log sequence 33567 (LGWR switch)
Current log# 13 seq# 33567 mem# 0: /oracle/product/oradata/xxxxx/redo13.log
Wed Jan 12 15:37:04 2011
Thread 1 advanced to log sequence 33568 (LGWR switch)
Current log# 12 seq# 33568 mem# 0: /oracle/product/oradata/xxxxx/redo12.log

请帮忙看看！
切换有时候很久：
Wed Jan 12 10:01:52 2011
Thread 1 advanced to log sequence 33541 (LGWR switch)
Current log# 16 seq# 33541 mem# 0: /oracle/product/oradata/xxxxx/redo16.log
Wed Jan 12 10:25:12 2011

truncate很慢对应的等待事件：
local write wait
帖子里说说一般是存储有问题了？
[ 本帖最后由 yyp2009 于 2011-1-12 16:27 编辑 ]

千问 · 2009-11-6 23:12:33

http://www.itpub.net/viewthread. ... RA-02068&page=1
同问

千问 · 2009-11-6 23:12:33

http://www.itpub.net/viewthread. ... highlight=ORA-02068

千问 · 2009-11-6 23:12:33

iostat 看看？

千问 · 2009-11-6 23:12:33

这是今天的，可惜昨天的没有记录下来，昨天比今天严重多了
：
Linux 2.6.18-164.el5xen 01/12/2011
09:00:24 AM CPU %user %nice %system %iowait%steal %idle
09:00:34 AM all6.590.000.95 23.530.01 68.91
09:00:44 AM all6.680.000.50 22.000.02 70.80
09:00:54 AM all6.500.000.71 20.790.00 72.01
09:01:04 AM all7.520.001.20 27.430.02 63.83
09:01:14 AM all6.330.000.50 28.130.01 65.03
09:01:24 AM all9.560.000.90 32.320.00 57.21
09:01:34 AM all8.700.000.61 31.300.02 59.36
09:01:44 AM all8.620.000.95 34.760.01 55.66
Average:
all7.560.000.79 27.520.01 64.11
[oracle@ ~]$ sar -u 5 4
Linux 2.6.18-164.el5xen (sjckjs)01/12/2011
09:04:04 AM CPU %user %nice %system %iowait%steal %idle
09:04:09 AM all 10.370.001.05 33.620.00 54.96
09:04:14 AM all9.260.000.79 27.670.02 62.25
09:04:19 AM all8.650.000.98 24.910.03 65.43
09:04:24 AM all 11.470.00 14.04 29.770.00 44.72
Average:
all9.940.004.19 28.990.01 56.87
[oracle@ ~]$ vmstat 5
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
rb swpd free buffcache si sobibo in cs us sy id wa st
26 1628402791235612 19572824213175 5371161 81 120
24 1628402513235596 1957393600 10487 23318 2065 221881 66 250
15 1628402639635564 1957370000 12211 12856 2220 2285 101 57 320
15 1628402813235580 1957148400 139799454 2381 2411 111 59 290
35 1628402680435608 1957393200 12544 14305 2222 2306 101 61 280
25 1628402613235612 1957476400 122139982 2170 2261 101 57 320
15 1628402667635668 19574636007878 22761 1688 184681 63 290
25 1628402763635600 1957446400 12275 13494 2278 238491 63 270
15 1628402903235596 1957412400 135408970 2432 2428 101 56 320
15 1628402935235640 1957394800 12544 13466 2242 230191 60 300
15 1628402906035656 1957399600 140429805 2416 2414 111 56 330

其中昨天的 b值都到22了今天才是5！！！

这个也是今天的：
[oracle@sjckjs ~]$ iostat -x
Linux 2.6.18-164.el5xen (sjckjs)01/12/2011
avg-cpu:%user %nice %system %iowait%steal %idle
6.450.000.80 11.970.02 80.76
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz awaitsvctm%util
cciss/c0d03.06 340.26 10.40 35.53 250.193006.5270.91 0.194.14 3.5616.37
cciss/c0d0p1

0.00 0.000.000.00 0.00 0.00 6.92 0.004.87 3.99 0.00
cciss/c0d0p2

3.06 340.26 10.40 35.53 250.193006.5270.91 0.194.14 3.5616.37
sda
17.54 432.63 85.22 35.139399.233742.20 109.19 0.726.01 1.9923.96
sda1
17.54 432.63 85.22 35.139399.233742.20 109.19 0.726.01 1.9923.96
sdb
79.88 209.16 253.61 24.41 41169.651868.68 154.80 0.311.12 2.4768.79
sdb1
79.88 209.16 253.61 24.41 41169.651868.68 154.80 0.311.12 2.4768.79
sdc
0.00 0.000.000.00 0.00 0.0014.36 0.001.62 1.25 0.00
sdc1
0.00 0.000.000.00 0.00 0.0012.46 0.002.14 1.43 0.00
dm-0
0.00 0.009.02 372.96 214.642983.65 8.37 0.471.22 0.4115.83
dm-1
0.00 0.004.442.8635.5622.88 8.00 0.52 70.72 1.70 1.24
[oracle@sjckjs ~]$ iostat -d
Linux 2.6.18-164.el5xen (sjckjs)01/12/2011
Device:
tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
cciss/c0d0 45.93 250.193006.52 1238680203 14884921556
cciss/c0d0p10.00 0.00 0.00 2486
4
cciss/c0d0p2 45.93 250.193006.52 1238677525 14884921552
sda
120.369399.203742.19 46534292587 18527125952
sda1
120.369399.203742.19 46534291827 18527125952
sdb
278.02 41169.521868.68 203825154291 9251593240
sdb1
278.02 41169.521868.68 203825153539 9251593240
sdc
0.00 0.00 0.00 1522
0
sdc1
0.00 0.00 0.00698
0
dm-0
381.98 214.642983.64 1062648594 14771654464
dm-1
7.3035.5622.88176028640113267088

千问 · 2009-11-6 23:12:33

Linux 2.6.18-164.el5xen (sjckjs)01/12/2011
avg-cpu:%user %nice %system %iowait%steal %idle
6.450.000.80 11.970.02 80.76
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz awaitsvctm%util
cciss/c0d03.06 340.26 10.40 35.52 250.193006.5070.91 0.194.14 3.5616.37
cciss/c0d0p1

0.00 0.000.000.00 0.00 0.00 6.92 0.004.87 3.99 0.00
cciss/c0d0p2

3.06 340.26 10.40 35.52 250.193006.5070.91 0.194.14 3.5616.37
sda
17.54 432.61 85.22 35.139398.743742.01 109.19 0.726.01 1.9923.96
sda1
17.54 432.61 85.22 35.139398.743742.01 109.19 0.726.01 1.9923.96
sdb
79.87 209.16 253.60 24.41 41167.481868.68 154.80 0.311.12 2.4768.79
sdb1
79.87 209.16 253.60 24.41 41167.481868.68 154.80 0.311.12 2.4768.79
sdc
0.00 0.000.000.00 0.00 0.0014.36 0.001.62 1.25 0.00
sdc1
0.00 0.000.000.00 0.00 0.0012.46 0.002.14 1.43 0.00
dm-0
0.00 0.009.02 372.95 214.642983.63 8.37 0.471.22 0.4115.83
dm-1
0.00 0.004.442.8635.5522.88 8.00 0.52 70.72 1.70 1.24
[oracle@sjckjs ~]$ iostat -d
Linux 2.6.18-164.el5xen (sjckjs)01/12/2011
Device:
tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
cciss/c0d0 45.93 250.193006.50 1238723107 14885576404
cciss/c0d0p10.00 0.00 0.00 2486
4
cciss/c0d0p2 45.93 250.193006.50 1238720429 14885576400
sda
120.359398.743742.01 46534314547 18527150856
sda1
120.359398.743742.01 46534313787 18527150856
sdb
278.01 41167.451868.68 203825163211 9252086968
sdb1
278.01 41167.451868.68 203825162459 9252086968
sdc
0.00 0.00 0.00 1522
0
sdc1
0.00 0.00 0.00698
0
dm-0
381.98 214.642983.63 1062690642 14772309312
dm-1
7.3035.5522.88176029496113267088

晕格式不对齐了。

千问 · 2009-11-6 23:12:33

原帖由 iori809 于 2011-1-12 16:44 发表
iostat 看看？
请教一下你有什么结论啊！？

千问 · 2009-11-6 23:12:33

cciss/c0d0
这个是你的存储吗？
你现在的速度正常吗？

千问 · 2009-11-6 23:12:33

昨天hen不正常了，今天好一点了，怎么说，请教一下？!

千问 · 2009-11-6 23:12:33

下回你在感觉慢的时候
执行一下
iostat -m
看看速度