Re: [patch 3/4] slab reclaim balancing

[Manfred Spraul]

Andrew Morton wrote:
> 
> Was the microbenchmark actually touching the memory which it was
> allocating from slab?  If so then yes, we'd expect to see cache
> misses against those cold pages coming out of the buddy.
>  

No, it was just measuring the cost of the kmem_cache_grow/shrink.

Btw, 140 cycles for kmem_cache_alloc+free is inflated - someone enabled 
kmem_cache_alloc_head() even in the no-debugging version.
As expected, done by Andrea, who neither bothered to cc me, nor actually 
understood the code.

> 
>>For SMP and slabs that are per-cpu cached, the change could be right,
>>because the arrays should absorb bursts. But I do not think that the
>>change is the right approach for UP.
> 
> 
> I'd suggest that we wait until we have slab freeing its pages into
> the hotlists, and allocating from them.  That should pull things back.
 >
You are asking a interesting question:

The slab is by design far from LIFO - it tries to find pages with no 
allocated objects, that are possible to return to the page allocator. It 
doesn't try to optimize for cache hit rates.

Is that actually the right approach? For large objects, it would be 
possible to cripple the freeable slabs list, and to perform the cache 
hit optimization (i.e. per-cpu LIFO) in page_alloc.c, but that doesn't 
work with small objects.

On SMP, the per-cpu arrays are the LIFO and should give good cache hit 
rates. On UP, I haven't enabled them, because they could increase the 
internal fragmentation of the slabs.

Perhaps we should enable the arrays on UP, too, and thus improve the 
cache hit rates? If there is no increase in fragmentation, we could 
ignore it. Otherwise we could replace the 3-list Bonwick slab with 
another backend, something that's stronger at reducing the internal 
fragmentation.

--
	Manfred