Re: O_DIRECT performance impact on 2.4.18 (was: Re: [PATCH] 2.5.14 IDE 56)

[Gerrit Huizenga]

In message <[e-mail]>, > : Jens Axboe writes:
> On Sat, May 11 2002, Gerrit Huizenga wrote:
> > In message <[e-mail]>, > : Jens Axboe writes:
> > > On Sat, May 11 2002, Roy Sigurd Karlsbakk wrote:
> > > > On Friday 10 May 2002 17:55, Linus Torvalds wrote:
> > > > > On Fri, 10 May 2002, Lincoln Dale wrote:
> > > > > > so O_DIRECT in 2.4.18 still shows up as a 55% performance hit versus no
> > > > > > O_DIRECT. anyone have any clues?
> > > > >
> > > > > Yes.
> > > > >
> > > > > O_DIRECT isn't doing any read-ahead.
> > > > >
> > > > > For O_DIRECT to be a win, you need to make it asynchronous.
> > > > 
> > > > Will the use of O_DIRECT affect disk elevatoring?
> > > 
> > > No, the I/O scheduler can't even tell whether it's being handed
> > > O_DIRECT buffers or not.
> > 
> > We tried disabling the elevator while doing Raw IO with DB2
> > a couple of weeks ago.  The database performance degraded much
> 
> I'm curious how you did this -- did you disable sorting and merging, or
> just sorting? Merging is pretty essential to getting decent I/O speeds
> in current kernels.
 
I believe sorting AND merging were turned off.  BTW, this was 2.4 only,
our primary focus is getting product into people's hands this year.  We
are hoping to play with the 2.5 IO scheduler, possibly in a few months.

> > more than expected.  Disks were FC connected Tritons or SCSI
> > connected ServerRaid (or both?).  Oracle often asks for a patch
> > to disable the elevator since they believe they can schedule IO
> > better.  We didn't try with Oracle in this case, but DB2 and RAW
> > IO without and elevator was not a good choice.
> 
> Due to excessive queue scan times, lock contention, or just slight waste
> of cycles?
 
A lot more interrupts on the RAID device, indicating a lot more
IOs, probably a direct result of disabling merging.  Overall IO throughput
dropped pretty dramatically, reducing database throughput.

A good indication to gen a patch with just sorting turned off and
see where that gets us...

gerrit