[PATCH] Re: Shared memory shmat/dt not working well in 2.5.x

[Hugh Dickins]

On Tue, 1 Oct 2002, Zlatko Calusic wrote:
> 
> Still having problems with Oracle on 2.5.x (it can't even be started),
> I devoted some time trying to pinpoint where the problem is. Reading
> many traces of Oracle, and rebooting a dozen times, I finally found
> that the culprit is weird behaviour of shmat/shmdt functions in 2.5,
> when combined with mprotect() calls. I wrote a simple test app
> (attached) and I'm also appending output of it below (running on
> 2.4.19 & 2.5.39 kernels, see the difference).

Exemplary bug report!  Many thanks for taking so much trouble to
reproduce the problem.  Patch below (against 2.5.39) should fix it:
I'll send Linus and Andrew when I can get hold of a 2.5.40 tree.

Hugh

--- 2.5.39/mm/mmap.c	Fri Sep 20 17:57:49 2002
+++ linux/mm/mmap.c	Tue Oct  1 13:59:54 2002
@@ -1055,7 +1055,7 @@ int split_vma(struct mm_struct * mm, str
 	if (new_below) {
 		new->vm_end = addr;
 		vma->vm_start = addr;
-		vma->vm_pgoff += ((addr - vma->vm_start) >> PAGE_SHIFT);
+		vma->vm_pgoff += ((addr - new->vm_start) >> PAGE_SHIFT);
 	} else {
 		vma->vm_end = addr;
 		new->vm_start = addr;