Re: Deadlock on the mm->mmap_sem

[David Howells]

> David, coredump is the only difficult recursive user of mmap_sem.  ptrace &
> /proc/pid/mem double buffer into kernel buffers, fork just doesn't lock the
> new mm_struct - it's new, noone can get a pointer to it before it's linked
> into the various lists.

Yes, you're right. So what you and Andrea are proposing is to have a field in
the task struct that counts the number of active readlocks you hold on your
own mm_struct. If this is >0, then you can add another readlock to it. If this
is the case, then you can add an extra asm-rwsem operation that simply
increments the semaphore counter. BUT you can only use this operation if you
_know_ you already have a readlock. And as you know that some function higher
up the stack holds the lock, you can guarantee that the lock isn't going to go
away.

Give me a few minutes, and I can handle this:-)

David