Re: Caches, page coloring, virtual indexed caches, and more

[Eric W. Biederman]

Anton Blanchard <[e-mail]> writes:

> Hi,
>  
> > Where do you do this?  And how do you handle the case of aliases with kseg,
> > the giant kernel mapping.
> 
> Aliases between user and kernel mappings of a page are handled by
> flush_page_to_ram the old interface) or {copy,clear}_user_page,
> flush_dcache_page and update_mmu_cache (new interface). Sparc64 already
> uses the new interface and there are patches for ppc and ia64 to use it.
> 
> The new interface allows flushes to be avoided, leading to rather nice
> performance increases.
> 
> See Documentation/cachetlb.txt for more info.

Thanks,

Well they are a step in the right direction....
But they are still racy, especially on SMP.

The bad case is:
Process A in kernel space calls flush_dcache_page.
Then process B in a separate thread writes to the first word in a
cache line. The Process A writes to the last word in the cache line. 

Assuming the virtual addresses from Process A and Process B are of a
different color this gives two non overlapping writes with a well
defined meaning, which the kernel gets wrong.  In particular the ram
will only see one write or the other not both.

What it looks like to me is that SHMLBA needs to be extended to normal
mmapings, making all pages in user space
(page->index << PAGE_SHIFT) % SHMLBA 
virtually aligned.

And whenever we access a page in the page cache that is not
appropriately virtually aligned in the fixed kernel mapping, 
we can use the kmap infrastructure to map it to a better kernel
location.  If we reuse the same optimizations from flush_dcache_page
it shouldn't be any worse, and in the pathological cases it will be
faster.   While removing the races seen above.

Any thoughts?

Eric