Thanks
Riyaj

LGWR process parallel write to both members at same time? (Example: Say Group1 having two members (multiplexing))

Once writing to both members of Group 1 then only LGWR signal / reported back successful to Users process?

Please suggest..

Really you are amazing…..

ADDM Report for Task ‘TASK_51215′
———————————

Analysis Period
—————
AWR snapshot range from 41947 to 41959.
Time period starts at 12-MAR-13 06.00.45 AM
Time period ends at 12-MAR-13 06.00.02 PM

Analysis Target
—————
Database ‘PR1′ with DB ID 759962060.
Database version 11.2.0.2.0.
ADDM performed an analysis of instance PR1, numbered 1 and hosted at cnuspp01.

Activity During the Analysis Period
———————————–
Total database time was 1931342 seconds.
The average number of active sessions was 44.75.

Summary of Findings
——————-
Description Active Sessions Recommendation
s
Percent of Activity
—————————————- ——————- ————–
-
1 Commits and Rollbacks 29.38 | 65.66 1
2 I/O Throughput 12.78 | 28.55 2
3 Undersized Buffer Cache 3.23 | 7.22 1
4 Top Segments by “User I/O” and “Cluster” .99 | 2.2 1

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Findings and Recommendations
—————————-

Finding 1: Commits and Rollbacks
Impact is 29.38 active sessions, 65.66% of total activity.
———————————————————-
Waits on event “log file sync” while performing COMMIT and ROLLBACK operations
were consuming significant database time.

Recommendation 1: Host Configuration
Estimated benefit is 29.38 active sessions, 65.66% of total activity.
———————————————————————
Action
Investigate the possibility of improving the performance of I/O to the
online redo log files.
Rationale
The average size of writes to the online redo log files was 176 K and
the average time per write was 255 milliseconds.
Rationale
The total I/O throughput on redo log files was 636 K per second for
reads and 1.2 M per second for writes.
Rationale
The redo log I/O throughput was divided as follows: 0% by RMAN and
recovery, 66% by Log Writer, 0% by Archiver, 0% by Streams AQ and 33% by
all other activity.

Symptoms That Led to the Finding:
———————————
Wait class “Commit” was consuming significant database time.
Impact is 29.38 active sessions, 65.66% of total activity.

Finding 2: I/O Throughput
Impact is 12.78 active sessions, 28.55% of total activity.
———————————————————-
The throughput of the I/O subsystem was significantly lower than expected.

Recommendation 1: Host Configuration
Estimated benefit is 12.78 active sessions, 28.55% of total activity.
———————————————————————
Action
Consider increasing the throughput of the I/O subsystem. Oracle’s
recommended solution is to stripe all data files using the SAME
methodology. You might also need to increase the number of disks for
better performance.
Rationale
During the analysis period, the average data files’ I/O throughput was
1.3 M per second for reads and 945 K per second for writes. The average
response time for single block reads was 89 milliseconds.

Recommendation 2: Host Configuration
Estimated benefit is 2.4 active sessions, 5.36% of total activity.
——————————————————————
Action
The performance of some data and temp files was significantly worse than
others. If striping all files using the SAME methodology is not
possible, consider striping these file over multiple disks.
Rationale
For file /oracle/PR1/sapdata2/dat_6/dat.data6, the average response time
for single block reads was 265 milliseconds, and the total excess I/O
wait was 22709 seconds.
Related Object
Database file
“/oracle/PR1/sapdata2/dat_6/dat.data6″
Rationale
For file /oracle/PR1/sapdata2/dat_10/dat.data10, the average response
time for single block reads was 279 milliseconds, and the total excess
I/O wait was 20281 seconds.
Related Object
Database file
“/oracle/PR1/sapdata2/dat_10/dat.data10″
Rationale
For file /oracle/PR1/sapdata1/dat_5/dat.data5, the average response time
for single block reads was 249 milliseconds, and the total excess I/O
wait was 20268 seconds.
Related Object
Database file
“/oracle/PR1/sapdata1/dat_5/dat.data5″
Rationale
For file /oracle/PR1/sapdata4/dat_21/dat.data21, the average response
time for single block reads was 124 milliseconds, and the total excess
I/O wait was 20230 seconds.
Related Object
Database file
“/oracle/PR1/sapdata4/dat_21/dat.data21″
Rationale
For file /oracle/PR1/sapdata3/dat_12/dat.data12, the average response
time for single block reads was 251 milliseconds, and the total excess
I/O wait was 20001 seconds.
Related Object
Database file
“/oracle/PR1/sapdata3/dat_12/dat.data12″

Symptoms That Led to the Finding:
———————————
Wait class “User I/O” was consuming significant database time.
Impact is 13.61 active sessions, 30.41% of total activity.

Finding 3: Undersized Buffer Cache
Impact is 3.23 active sessions, 7.22% of total activity.
——————————————————–
The buffer cache was undersized causing significant additional read I/O.
The value of parameter “db_cache_size” was “13568 M” during the analysis
period.

Recommendation 1: Database Configuration
Estimated benefit is .63 active sessions, 1.41% of total activity.
——————————————————————
Action
Increase the buffer cache size by setting the value of parameter
“db_cache_size” to 14784 M.

Symptoms That Led to the Finding:
———————————
Wait class “User I/O” was consuming significant database time.
Impact is 13.61 active sessions, 30.41% of total activity.

Finding 4: Top Segments by “User I/O” and “Cluster”
Impact is .99 active sessions, 2.2% of total activity.
——————————————————
Individual database segments responsible for significant “User I/O” and
“Cluster” waits were found.

Recommendation 1: Segment Tuning
Estimated benefit is .99 active sessions, 2.2% of total activity.
—————————————————————–
Action
Investigate application logic involving I/O on TABLE “SAPDAT.INDX” with
object ID 355222.
Related Object
Database object with ID 355222.
Rationale
The I/O usage statistics for the object are: 0 full object scans,
2687152 physical reads, 2462181 physical writes and 0 direct reads.

Symptoms That Led to the Finding:
———————————
Wait class “User I/O” was consuming significant database time.
Impact is 13.61 active sessions, 30.41% of total activity.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Additional Information
———————-

Miscellaneous Information
————————-
Wait class “Application” was not consuming significant database time.
Wait class “Concurrency” was not consuming significant database time.
Wait class “Configuration” was not consuming significant database time.
CPU was not a bottleneck for the instance.
Wait class “Network” was not consuming significant database time.
Session connect and disconnect calls were not consuming significant database
time.
Hard parsing of SQL statements was not consuming significant database time.

Truss of LGWR shows that LGWR is submitting Asynch I/O :
kaio(AIOWRITE, 261, 0x602A4600, 512, 0xFC73DDD00D684A00) = 0

file_id 261 of LGWR process through pfiles command is:

261: S_IFCHR mode:0755 dev:291,0 ino:15729286 uid:601 gid:503 rdev:30,321
O_RDWR|O_NONBLOCK|O_DSYNC|O_LARGEFILE FD_CLOEXEC
/devices/iscsi/disk@0000iqm.demo.volumes-san0001,4:b,raw

As you see, LGWR has opened the file with DSYNC flag.

I need to probe Linux to see how to do this, but I would imagine lsof should do the trick.

Cheers
Riyaj

Great article, thanks. I’ve been playing around with strace and lsof +fG on my linux server tracing the lgwr process but cannot find any evidence that redo data is opened/written in DSYNC mode as you mention above:

“…redo log files are opened with DSYNC flag”.

How can we confirm DSYNC is indeed being used?