Re: Bio pool & scsi scatter gather pool usage

[Andrew Morton]

Mark Peloquin wrote:
> 
> ...
> > Tell me why this won't work?
> 
> This would require the BIO assembly code to make at least one
> call to find the current permissible BIO size at offset xyzzy.
> Depending on the actual IO size many foo_max_bytes calls may
> be required. Envision the LVM or RAID case where physical
> extents or chunks sizes can be as small as 8Kb I believe. For
> a 64Kb IO, its conceivable that 9 calls to foo_max_bytes may
> be required to package that IO into permissibly sized BIOs.

True.  But probably the common case is not as bad as that, and
these repeated calls are probably still cheaper than allocating
and populating the redundant top-level BIO.

Also, the top-level code can be cache-friendly.  The bad way
to write it would be to do:

	while (more to send) {
		maxbytes = bio_max_bytes(block);
		build_and_send_a_bio(block, maxbytes);
		block += maxbytes / whatever;
	}

That creates long code paths and L1 cache thrashing.  Kernel
tends to do that rather a lot in the IO paths.

The good way is:

	int maxbytes[something];
	int i = 0;

	while (more_to_send) {
		maxbytes[i] = bio_max_bytes(block);
		block += maxbytes[i++] / whatever;
	}
	i = 0;
	while (more_to_send) {
		build_and_send_a_bio(block, maxbytes[i]);
		block += maxbytes[i++] / whatever;
	}

if you get my drift.  This way the computational costs of
the second and succeeding bio_max_bytes() calls are very
small.

One thing which concerns me about the whole scheme at
present is that the uncommon case (volume managers, RAID,
etc) will end up penalising the common case - boring
old ext2 on boring old IDE/SCSI.

Right now, BIO_MAX_SECTORS is only 64k, and IDE can
take twice that.  I'm not sure what the largest
request size is for SCSI - certainly 128k.

Let's not create any designed-in limitations at this
stage of the game.

-