trying to understand mmap, msync, O_SYNC, and devices

[Chris Friesen]

With the discussions over the past while, it is now clear that shared mappings 
of normal files will not be written out until forced to, and that this is the 
correct thing to do in almost all cases.

In this case, changing something in the mapped area will make the changes 
visible to all other processes mapping the same area, but the changes are not 
guaranteed to be flushed to the backing store.

What I am not clear on is how this interacts with mapping *devices*.  Does it 
still go through the buffer cache?

Details follow for those who are interested:

I have hardware which does not wipe ram on reset as long as it is not powered 
off.  On boot I force a limit on the amount of memory used by the kernel, which 
leaves a chunk of unused memory beyond normally-accessable memory.  I have a 
char driver which then implements mmap by mapping this memory to the calling 
process.  This mapping is shared between many processes and is used for logging, 
with appropriate locking mechanisms.

My key issue is this--how do I ensure that the contents of this memory are 
consistant and up to date when the board may be rebooted at any time?

I have three levels of enforcement that I have considered:

1) Opening the device with O_SYNC.  This also tells the mmap code in the driver 
to set the memory to uncached, the same way that mem.c does in pgprot_noncached().

2) Calls to wmb() in the code to enforce order when setting critical fields.

3) Explicit calls to msync() in place of the wmb() calls. (Which slows it down 
by a factor of 2.5 as compared to #2.)

I assume that #1 is required to guard against resetting of the board.  Beyond 
that is #2 sufficient, or is #3 required?

Thanks,

Chris

-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: [phone]
3500 Carling Avenue              | fax:  [phone]
Nepean, ON K2H 8E9 Canada        | email: [e-mail]