Re: aio

[Suparna Bhattacharya]

On Wed, Dec 19, 2001 at 10:09:40PM -0800, Linus Torvalds wrote:
> Could we get back on track, and possibly discuss the patches themselves,
> ok? We want _constructive_ criticism of the interfaces.
> 
> I think it's clear that many people do want to have aio support. At least

Yes, to add to the lot, even though you probably don't need any more proof :)
we have at least 3 different products requiring this. Both for scalable 
communications aio (large no. of connections) as well as file/disk aio.  
In fact, one of the things that I've been worried about once 2.5 opened up
and the bio changes started flowing in (much as I was delighted to see
Jens' stuff finally getting integrated), was whether this would mean a 
longer timeframe before we can hope to see aio in a distribution (which is 
the question which I have to respond to our product groups about). 

> as far as I'm concerned, that's not the reason I want to have public
> discussion. I want to make sure that the interfaces are good for aio
> users, and that the design isn't stupid.
> 

My feeling about this is that we shouldn't necessarily need to
have the entire aio code in perfect shape and bring it in all in one shot.
The thing I like about the design is that it is quite possible to split
this up into smaller core patches and bring them in slowly. And I agree
with Ingo that we should be able to start stabilizing the basic internal
mechanisms or foundations on which aio is built before we freeze the 
external interfaces. In fact, these two things could happen parallely.
So I was hoping that an evolutionary approach would work. In the current 
design, the aio path is handled separately from the normal i/o paths, so the 
impact on existing interfaces is less. It mainly affects aio users. So it 
ought to be possible to integrate it in a way that doesn't hurt regular
operations, and it should be easier to change it once it is in without
breaking too many things.

Existing aio users on other platforms that I have come across seem to use 
either POSIX aio (for file /disk aio) or completion port style interfaces 
(mainly for communications aio), both of which seem to be possible with 
Ben's implementation. One has to explicitly associate each i/o with the 
completion queue (ctx), rather than associate an fd as a whole with it so 
that all completion events on that fd come to the ctx. That should be OK. 
Besides with async poll support we can have per-file readiness notification 
as well. I was hoping for the async poll piece being available early to 
exercise the top half or event handling side of aio, so we have scalable 
select/poll support, so was focussing on that part to start with.

Your point about some critical discussion of the interfaces and the design 
is well-taken. We have had a few discussions on the aio mailing and more on 
irc about some aspects, but not quite a thorough out and out analysis of 
pros and cons of the whole design. I just started writing some points here, 
but then realized that it is going to take much longer, so decided to do
that while working with Ben on the documentation, and discuss more after 
that.

Regards
Suparna