Re: 2.[45] fixes for design locking bug in

[Andrew Morton]

Andrea Arcangeli wrote:
> 
> 
> no, you forgot readahead, that's buggy and needs fixing in readpage too
> (same in 2.4 and 2.5).

readahead in 2.5 unplugs.  Which is quite wrong for AIO, but is unavoidable
given the current situation...
 
> ...
> 
> > I dunno.  I bet there are still more holes, and I for one am heartily sick
> > of unplug bugs.  Why not make the damn queue unplug itself after ten
> > milliseconds or 16 requests?  I bet that would actually increase throughput,
> > especially in the presence of kernel preemption and scheduling points.
> 
> This doesn't remove the need to avoid the mean 5 msec delay before the
> queue unplug (I know the race triggers rarely though, but if we left all the
> places broken the number of 5msec mean waits will increase bug by bug
> over time). I'm not very excited by the idea (in particular for 2.4), if
> it performs so much better I would say it's a broken length of the queue
> that is way too oversized and that leads to other problems with fariness
> of task against task etc... We need the big number of requests with
> small requests only, with big requests we should allow only very few
> requests to be returned.

No, no, no.  Step back, miles back.  Forget the current implementation
and ask yourself "what is a good design for plugging?".  Because I can
assure you that if someone came out and proposed the current design for
inclusion in today's kernel, they would be pulverised.

Plugging is an optimisation which is specific to, and internal to the
request layer.  It should not require that every client of that layer
know about it, and be careful to unplug (with bizarrely complex locking
and ordering rules) simply because the request layer is too lame to look
after its own stuff.

Particularly because the clients of the request layer do not have enough
information to make a good decision about when to unplug.  Only the
request layer knows that.

And particularly because we've been hitting the same darn bug for years.
And we've been blaming the poor old callers!

The request layer should perform its own unplug "when the time is right",
and the external unplug should be viewed as an optimisation hint.

It all becomes rather obvious when you try to answer the question "when
should the kernel unplug for AIO reads"?

And yes, the fact that the time-based component of the unplug decision
would obscure failure by callers to make use of the optimisation hint
is unfortunate.  But it's only a few callsites, and 99.9999% coverage
there is good enough.