Re: [reiserfs-dev] Re: Note describing poor dcache utilization under high memory pressure

[Daniel Phillips]

On January 29, 2002 01:16 am, Hans Reiser wrote:
> Daniel Phillips wrote:
> 
> >On January 28, 2002 08:25 pm, Hans Reiser wrote:
> >
> >>If I understand you right, your scheme has the fundamental flaw that one 
> >>dcache entry on a page can keep an entire page full of "slackers" in 
> >>memory, and since there is little correlation in usage between dcache 
> >>entries that happen to get stored on a page, the result is that the 
> >>effectiveness per megabyte of the dcache is decreased by an order of 
> >>magnitude.  It would be worse to have one dcache entry per page, but 
> >>maybe not by as much as you might expect.
> >>
> >>When objects smaller than a page are stored on a page but not correlated 
> >>in their usage, they need to be aged individually not as a page, and 
> >>then garbage collected as needed.
> >>
> >
> >I had the identical thought - i.e., that this is a job for object aging and 
> >not lru, then I realized that a slight modification to lru can do the job, 
> >that is:
> >
> >  - An access to any object on the page promotes the page to the hot end
> >    of the lru list.
> >
> >  - When it's time to recover a page (or pages) scan from the cold end
> >    towards the hot end, and recover the first page(s) on which all
> >    objects are free.
> >
> >>Neither the current model nor your 
> >>proposed scheme solve the fundamental problem Josh's measurements prove 
> >>exists.  
> >>
> >
> >My suggestion might.
> >
> This fails to recover an object (e.g. dcache entry) which is used once, 
> and then spends a year in cache on the same page as an object which is 
> hot all the time.

You don't worry about that case.  If there's so much pressure that you
scan all the way to the hot end of the lru list then you will recover
that hot/cold page[1] and all will be well.  Not that the hot/cold page
will tend to migrate further away from the hot end of the list than a
hot/hot page.

[1] Assuming all the objects are freeable, otherwise you have a very
different problem: fragmentation.

> This means that the hot set of objects becomes 
> diffused over an order of magnitude more pages than if garbage 
> collection squeezes them all together.

I don't see that.  How does garbage collection squeeze things together?
I'm assuming these objects are pinned, at least from the VM's point of
view.

-- 
Daniel