Re: [PATCH][IO_APIC] 2.5.63bk7 irq_balance improvments / bug-fixes

[Jeff Garzik]

Kamble, Nitin A wrote:
> There are few issues we found with the user level daemon approach.

Thanks much for the response!

>    Static binding compatibility: With the user level daemon, users can
> not  
> use the /proc/irq/i/smp_affinity interface for the static binding of
> interrupts.

Not terribly accurate:  in "one-shot" mode, where the daemon balances 
irqs once at startup, users can change smp_affinity all they want.

In the normal continuous-balance mode, it is quite easy to have the 
daemon either (a) notice changes users make or (b) configure the daemon. 
  The daemon does not do (a) or (b) currently, but it is a simple change.

>   There is some information which is only available in the kernel today,
> Also the future implementation might need more kernel data. This is
> important for interfaces such as NAPI, where interrupts handling changes
> on the fly.

This depends on the information :)  Some information that is useful for 
balancing is only [easily] available from userspace.   In-kernel 
information may be easily exported through "sysfs", which is designed to 
export in-kernel information.

Further, for NAPI and networking in general, it is recommended to bind 
each NIC to a single interrupt, and never change that binding. 
Delivering a single NIC's interrupts to multiple CPUs leads to a 
noticeable performance loss.  This is why some people complain that 
their specific network setups are faster on a uniprocessor kernel than 
an SMP kernel.

I have not examined interrupt delivery for other peripherals, such at 
ATA or SCSI hosts, but for networking you definitely want to statically 
bind each NIC's irqs to a separate CPU, and then not touch that binding.

Best regards, and thanks again for your valuable feedback,

	Jeff