Re: Linux networking and disk IO issues

[Andi Kleen]

[this time with l-k cc]

Mark Hayden <[e-mail]> writes:

> * The Linux networking stack requires all skbuff buffers to be
>   contiguous.  As far as I can tell, this makes it impossible to
>   write high-bandwidth UDP applications on Linux.  For instance, the
>   kernel will drop a fragmented 8KB message if it cannot allocate 8KB
>   of contiguous memory to reassemble it into.  I have found that it
>   is relatively easy to enter regimes where this can cause massive
>   packet loss.

2.4.4+ supports fragmented packets and packet lists.

You're probably seeing the 8K allocation problem for incoming packets which need to be
allocated by the driver on interrupt time with GFP_ATOMIC. GFP_ATOMIC memory is limited.
The 2.4 VM unfortunately has no way to keep more GFP_ATOMIC free ATM and tune for heavy
interrupt load (2.2 allowed this by increasing the freepages sysctl). Hopefully this VM bug 
will be fixed in the not too far future.

A workaround in the driver would be to use the 2.4.4 fragmented buffers 
(of course you'll still run into GFP_ATOMIC limits without manual tuning)
or allocate RX memory from a thread with GFP_KERNEL.

-Andi