Re: [PATCH] rq-dyn-alloc, dynamic request allocation

[Andrew Morton]

Jens Axboe <[e-mail]> wrote:
>
> Hi,
> 
> This patch adds dynamic request allocation to the block io path. On
> systems with lots of disks (and thus queues) it saves a non-significant
> amount of low memory. It also allows for much better experimentation
> with larger queue lengths, this experimental patch tops the queue depth
> off at 16384 (vs 128 before).

heh, 16k requests per queue?  Last time I played with 1024 certain popular
benchmarks ran like a bullet.

> Please play with it. Andrew, want a version for -mm?

Would be much appreciated, thanks.

>   */
>  static struct request *get_request_wait(request_queue_t *q, int rw)
>  {
> -	DEFINE_WAIT(wait);
> -	struct request_list *rl = &q->rq[rw];
>  	struct request *rq;
>  
> -	spin_lock_prefetch(q->queue_lock);
> -
>  	generic_unplug_device(q);
>  	do {
> -		int block = 0;
> +		rq = get_request(q, rw, GFP_NOIO);
>  
> -		prepare_to_wait_exclusive(&rl->wait, &wait,
> -					TASK_UNINTERRUPTIBLE);
> -		spin_lock_irq(q->queue_lock);
> -		if (!rl->count)
> -			block = 1;
> -		spin_unlock_irq(q->queue_lock);
> -
> -		if (block)
> +		if (!rq)
>  			io_schedule();

hmm.  I fear that if a SCHED_FIFO/SCHED_RR task hits this, it will just pick
itself to run again in the schedule() and the box locks up.

A blk_congestion_wait(WRITE, HZ/50) may be better here.  It will send the
caller to sleep until someone puts a write request back, which seems
appropriate.