Re: performance of O_DIRECT on md/lvm

[Andrea Arcangeli]

On Fri, Jan 18, 2002 at 05:50:53PM +0000, Arjan van de Ven wrote:
> Justin Cormack wrote:
> > 
> > Reading files with O_DIRECT works very nicely for me off a single drive
> > (for video streaming, so I dont want cacheing), but is extremely slow on
> > software raid0 devices, and striped lvm volumes. Basically a striped
> > raid device reads at much the same speed as a single device with O_DIRECT,
> > while reading the same file without O_DIRECT gives the expected performance
> > (but with unwanted cacheing).
> > 
> > raw devices behave similarly (though if you are using them you can probably
> > do your own raid0).
> > 
> > My guess is this is because of the md blocksizes being 1024, rather than
> > 4096: is this the case and is there a fix (my quick hack at md.c to try
> > to make this happen didnt work).
> 
> well not exactly. Raid0 is faster due to readahead (eg you read one
> block and the kernel 
> sets the OTHER disk also working in parallel in anticipation of you
> using that). O_DIRECT
> is of course directly in conflict with this as you tell the kernel that
> you DON'T want
> any optimisations....

if you read in chunks of a few mbytes per read syscall, the lack of
readahead shouldn't make much difference (this is true for both raid and
standalone device). If there's a relevant difference it's more liekly an
issue with the blocksize.

Andrea