Re: [PATCH] Re: Shared memory shmat/dt not working well in

[Hugh Dickins]

On Tue, 1 Oct 2002, Petr Vandrovec wrote:
> 
> You are my hero!

Aww, shucks!  I guess I'd better have a go at your next problem...

> Unfortunately it did not fixed another problem I have with sys_mprotect.
> If I start X, system stops to do anything useful. After SAK I could
> do remote connect and running 'w' and 'ps axf' moved them to 'D' state.

> Oct  1 17:47:55 vana kernel:  [<c0107ec9>]error_code+0x2d/0x38
> Oct  1 17:47:55 vana kernel:  [<c011216f>]flush_tlb_mm+0x1b/0x70
> Oct  1 17:47:55 vana kernel:  [<c0133ed1>]change_protection+0x1a1/0x1dc
> Oct  1 17:47:55 vana kernel:  [<c01341ed>]mprotect_fixup+0x16d/0x188

Looks to me like flush_tlb_mm is faulting on the vma->vm_mm given it.
And looks to me like mprotect_fixup, in the merge case, may be passing
an already freed vma to change_protection.  I'm not as confident about
this patch as the earlier one, but I believe it's correct: please
give it a try, and maybe Christoph will confirm or deny it.

Hugh

--- 2.5.40/mm/mprotect.c	Fri Sep 27 23:56:45 2002
+++ linux/mm/mprotect.c	Tue Oct  1 18:00:31 2002
@@ -186,8 +186,10 @@
 		/*
 		 * Try to merge with the previous vma.
 		 */
-		if (mprotect_attempt_merge(vma, *pprev, end, newflags))
+		if (mprotect_attempt_merge(vma, *pprev, end, newflags)) {
+			vma = *pprev;
 			goto success;
+		}
 	} else {
 		error = split_vma(mm, vma, start, 1);
 		if (error)