Re: Quick question about hyper-threading (also some NUMA stuff)

[Timothy Miller]

From: Tony 'Nicoya' Mantler ([e-mail])

> Perhaps the same effect could be obtained by preferentially scheduling
processes
> to execute on the "node" (a node being a single cpu in an SMP system, or
an HT
> virtual CPU pair, or a NUMA node) that they were last running on.

> I think the ideal semantics would probably be something along the lines
of:

>  - a newly fork()ed thread executes on the same node as the creating
thread
>  - calling exec() sets a "feel free to shuffle me elsewhere" flag
>  - threads are otherwise only shuffled to other nodes when a certain load
ratio
> is exceeded (current-node:idle-node)

This sounds like the most sensible approach.  I like considering the
extremes of performance, but sometimes, the time for math required for some
optimization can be worse than any benefit you get out of it.  Your
suggestion is simple.  It increases the likelihood (10% better for little
extra effort is better than 10% worse) of related processes being run on the
same node, while not impacting the system's ability to balance load.  This,
as you say, is also very important for NUMA.

Does the NUMA support migrate pages to the node which is running a process?
Or would processes jump nodes often enough to make that not worth the
effort?

In order for page migration to be worth it, node affinity would have to be
fairly strong.  It's particularly important when a process maps pages which
belong to another node.  Is there any logic there to duplicate pages in
cases where there is enough free memory for it?  We'd have to tag the pages
as duplicates so the VM could reclaim them.