DBA日记 第二部 (33) 外来的和尚好念经 5月8日 危机再现(1)
原来计划5月6号去肇庆鼎湖山玩,6号下了一场大雨,就把行程推迟到7号了。原本计划7号当天回家的,鼎湖山新鲜的空气让大家都决定在鼎湖住一夜,回到深圳的时候已经是8号的下午了。8号早上和老万通了一个电话,问问那边的情况,老万说早上CPU使用率有点吓人,已经冲到100%了,不过从数据库的状态来看,倒是还可以,活跃会话的数量一直在100-120之间,按照老万他们的经验,活跃会话数量小于200的时候,系统是比较稳定的,超过200的时候,一般来说系统的性能就有问题了。
既然老万那边没什么问题,我也就不着急回家了,到七星岩转了一圈,买了一包肇庆裹蒸粽才打道回府。到家后洗了个澡,感觉浑身还在冒汗,深圳比肇庆热多了,真有点怀念鼎湖山的清凉。打开电脑收了一下邮件,首先看到马工发过来的statspack报告,最近一年以来,每天采集两份statspack报告并发给我是马工每天雷打不动的工作。打开今天的STATSPACK报告,看到今天的平均事务响应时间是1600毫秒,响应时间和30号优化实施前差不太多,但是8号的平均每秒事务数小于29,30号,说明8号的业务量还没有恢复到节前的水平,这估计和长假后部分休假的人员还没有回来上班有关。从STATSPACK报告上看:
Load Profile
~~~~~~~~~~~~ Per Second Per Transaction
--------------- ---------------
Redo size: 239,525.76 6,734.11
Logical reads: 86,551.62 2,433.34
Block changes: 1,409.14 39.62
Physical reads: 5,325.29 149.72
Physical writes: 467.88 13.15
User calls: 4,843.94 136.18
Parses: 661.67 18.60
Hard parses: 56.70 1.59
Sorts: 32.95 0.93
Logons: 0.38 0.01
Executes: 1,004.14 28.23
Transactions: 35.57
% Blocks changed per Read: 1.63 Recursive Call %: 22.12
Rollback per transaction %: 1.13 Rows per Sort: 3406.02
Instance Efficiency Percentages (Target 100%)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Buffer Nowait %: 99.97 Redo NoWait %: 100.00
Buffer Hit %: 94.12 In-memory Sort %: 99.95
Library Hit %: 98.24 Soft Parse %: 91.43
Execute to Parse %: 34.11 Latch Hit %: 97.38
Parse CPU to Parse Elapsd %: 4.64 % Non-Parse CPU: 93.84
Shared Pool Statistics Begin End
------ ------
Memory Usage %: 74.54 74.51
% SQL with executions>1: 90.66 92.64
% Memory for SQL w/exec>1: 98.48 98.76
Top 5 Timed Events
~~~~~~~~~~~~~~~~~~ % Total
Event Waits Time (s) Ela Time
-------------------------------------------- ------------ ----------- --------
db file sequential read 24,318,031 342,222 75.24
db file scattered read 711,781 39,184 8.62
latch free 2,132,500 35,154 7.73
CPU time 28,834 6.34
buffer busy waits 108,299 3,717 .82
-------------------------------------------------------------
Avg
Total Wait wait Waits
Event Waits Timeouts Time (s) (ms) /txn
---------------------------- ------------ ---------- ---------- ------ --------
db file sequential read 24,318,031 0 342,222 14 90.5
db file scattered read 711,781 0 39,184 55 2.8
latch free 2,132,500 0 35,154 16 16.7
buffer busy waits 108,299 1 3,717 34 0.8
io done 352,620 0 1,625 5 2.8
log file sync 135,975 31 1,045 8 1.1