Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

[Andrea Arcangeli]

On Tue, Jan 09, 2001 at 09:12:04PM +0100, Jens Axboe wrote:
> I haven't heard anything beyond the raised QUEUE_NR_REQUEST, so I'd like to
> see what you have pending so we can merge :-). The tiotest seek increase was
> mainly due to the elevator having 3000 requests to juggle and thus being able
> to eliminate a lot of seeks right?

Raising QUEUE_NR_REQUEST is possible because of the rework of other parts of
ll_rw_block meant to fix the lowmem boxes.

> > write numbers, streaming I/O doesn't change on highmem boxes but it doesn't
> > hurt lowmem boxes anymore). Current blk-13B isn't ok for integration yet
> > because it hurts with lowmem (try with mem=32m with your scsi array that gets
> > 512K*512 requests in flight :) and it's not able to exploit the elevator as
> 
> I don't see any lowmem problems -- if under pressure, the queue should be
> fired and thus it won't get as long as if you have lots of memory free.`

A write(2) shouldn't cause the allocator to wait I/O completion. It's the write
that should block when it's only polluting the cache or you'll hurt the
innocent rest of the system that isn't writing.

At least with my original implementation of the 512K large scsi command
support that you merged, before a write could block you first had to generate
at least 128Mbyte of memory _locked_ all queued in the I/O request list waiting
the driver to process the requests (only locked, without considering
the dirty part of memory).

Since you raised from 256 requests per queue to 512 with your patch you
may have to generate 256Mbyte of locked memory before a write can block.

This is great on the 8G boxes that runs specweb but this isn't that great on a
32Mbyte box connected incidentally to a decent SCSI adapter.

I say "may" because I didn't checked closely if you introduced any kind of
logic to avoid this. It seems not though because such a logic needs to touch at
least blkdev_release_request and that's what I developed in my tree and then I
could raise the number of I/O request in the queue up to 10000 if I wanted
without any problem, the max-I/O in flight was controlled properly. (this
allowed me to optimize away not 256 or in your case 512 seeks but 10000 seeks)
This is what I meant with exploiting the elevator. No panic, there's no buffer
overflow there ;)

Andrea