Re: [Lse-tech] Node affine NUMA scheduler extension

[Andi Kleen]

On Tue, May 27, 2003 at 10:31:55AM +0200, Erich Focht wrote:
> This patch is an adaptation of the earlier work on the node affine
> NUMA scheduler to the NUMA features meanwhile integrated into
> 2.5. Compared to the patch posted for 2.5.39 this one is much simpler
> and easier to understand.

Yes, it is also much simpler than my implementation (I did a similar homenode
scheduler for an 2.4 kernel). The basic principles are the same.

But the main problems I have is that the tuning for threads is very 
difficult. On AMD64 where Node equals CPU it is important
to home node balance threads too. After some experiments I settled on 
homenode assignment on the first load balance (called "lazy homenode")  
When a thread clones it initially executes on the CPU of the parent, but 
there is a window until the first load balance tick where it can allocate
memory on the wrong node.  I found a lot of code runs very badly until the 
cache decay parameter is set to 0 (no special cache affinity) to allow
quick initial migration. Migration directly on fork/clone requires a lot 
of changes and also breaks down on some benchmarks.

-Andi