2.4.10pre10aa1 (first spin to the vm rewrite included)

Andrea Arcangeli (andrea@suse.de)
Mon, 17 Sep 2001 18:09:14 +0200

Messages sorted by: [ date ][ thread ][ subject ][ author ]
Next message: Eric W. Biederman: "Re: broken VM in 2.4.10-pre9"
Previous message: Eric W. Biederman: "Re: broken VM in 2.4.10-pre9"
Next in thread: Dipankar Sarma: "Re: 2.4.10pre10aa1 (first spin to the vm rewrite included)"
Reply: Dipankar Sarma: "Re: 2.4.10pre10aa1 (first spin to the vm rewrite included)"
Reply: Olaf Zaplinski: "Re: 2.4.10pre10aa1 (first spin to the vm rewrite included)"

Some of the the main features of 2.4.10pre10aa1 are:

o vm rewrite (see below a description)
o do all the possible vma merging during mmap/mremap/mprotect and
rewrite of the vma lookup engine using rb trees checkpointing for
avoiding various browse of the tree during vma merging, mmap etc...
o blkdev in pagecache (you can also open a blkdev and mmap it,
or you can open it with O_DIRECT and do rawio without using /dev/raw)
o rawio improvement with preallocation of the kiobuf in the f_iobuf
provided by the O_DIRECT support (next step is to split the kiobuf
for the contention case that doesn't happen during the DB workload
and for the non I/O users like bttv)
o O_DIRECT support
o latest TUX 2 release developed by Ingo Molnar (RedHat Inc) (you need
the tux 2.1.0 userspace package from redhat.com to use it)
o latest UML release developed by Jeff Dike
o rcu core included (thought it will be rewritten, I agreed with
Dipankar that the per-cpu scheduling sequence number seems
the best approch, such number can serve also as a statistic
to usespace infact, it is very similar to Rusty's patch but it
doesn't add any branch to the schedule fast path, he's rewriting
the patch at the moment and I'll include it in the next release)

Only in 2.4.10pre8aa1: 00_gcc-30-extern-static-3
Only in 2.4.10pre8aa1: 00_nfs-delete-lock-1
Only in 2.4.10pre8aa1: 00_tgid-1

Merged into mainline.

Only in 2.4.10pre10aa1: 80_vm-aa-1

VM rewrite, should fix:

- kswapd looping forever on DMA or NORMAL classzones
- swap+ram will be almost all available address space
(modulo when the swap cache serves to avoid swapins
of shares anonymous memory after a fork)
- slowdowns after multiple runs of the workloads
- swapout storms
- dbench runs twice faster with this patch applied

I've several ideas on how to change the vm algorithms to
be even better, this first one may be a little too agrressive, however
I tried a few of them and I did a check of the dbench numbers after
that and they were always slower than the algorithm in the 80_vm-aa-1
patch. (OTOH I don't want a VM that works wonderfully on dbench
but not with other normal workloads)

This patch may still swap a little too much (but it shouldnt'
generate the storms during heavy I/O), I need feedback from differnt
workloads to be sure it's ok, at this point it is trival for me to
change the behaviour radically after I get the feedback and numbers
from users and developers.

So please test it so I will know if I must change something.

About stability it should be rock solid, but of course there's
not a very good statistic yet (my most critical desktop is running
2.4.10pre10aa1 since last night under high load without problems yet).
I probably have a bug in swapoff but let's ignore it for now, just
try to run swapoff only before shutting down the machine. The fact
is that the 2.4 VM is broken freeing physically dirty pages.
The last owner of the page (usually the VM except in swapoff) has to
clear the dirty flag before freeing the page, in swapoff it may
be a little more complicate (we may need to grab the pagecache_lock
to ensure nobody start using the page while we clear it). And swapoff
is probably racy anyways as usual (swapoff in 2.2 is racy too). In
short I didn't focused on swapoff yet, I just made an hack to make it
to work while shutting down the machine so far.

I know this patch could hurt some developer since I backed out
lots of (broken) stuff that probably they liked, I'm sorry for that,
but I will mainly listen to numbers, not to "voice/email" vapourware
arguments (like the argument numa is broken, that's just a red herring,
this patch flyes on the numa wildfire simulation with all the numa
heuristics enabled as well as on uml with mem=4m, origin 2k or whatever
x86/ia64 numa hardware will never ever have problems either with it and
I will make sure it will work correctly). So in short I will love to
be proved wrong on the _paper_ with real life issues.

Only in 2.4.10pre10aa1: 00_alpha-compile-1

Fix a few alpha compile troubles.

Only in 2.4.10pre8aa1: 00_alpha-fp-disabled-1
Only in 2.4.10pre10aa1: 00_alpha-fp-disabled-2

Rediffed due trivial rejects.

Only in 2.4.10pre8aa1: 00_copy-user-lat-2
Only in 2.4.10pre10aa1: 00_copy-user-lat-3

Rediffed due trivial rejects.

Only in 2.4.10pre8aa1: 00_module-gfp-2
Only in 2.4.10pre10aa1: 00_module-gfp-3

Rediffed due trivial rejects.

Only in 2.4.10pre8aa1: 00_silent-stack-overflow-8
Only in 2.4.10pre10aa1: 00_silent-stack-overflow-9

Rediffed due trivial rejects.

Only in 2.4.10pre8aa1: 10_prefetch-6
Only in 2.4.10pre10aa1: 10_prefetch-8

Part of it merged into mainline, rediffed the missing part.

Only in 2.4.10pre8aa1: 40_blkdev-pagecache-16
Only in 2.4.10pre10aa1: 40_blkdev-pagecache-17

Rediffed due trivial rejects.

Only in 2.4.10pre8aa1: 50_uml-patch-2.4.9-4.bz2
Only in 2.4.10pre10aa1: 50_uml-patch-2.4.9-6.bz2

Picked last update with the move of the kernel address space from
sourceforge.

Only in 2.4.10pre8aa1: 60_tux-2.4.9-ac10-H2
Only in 2.4.10pre10aa1: 60_tux-2.4.9-ac10-H7

Picked last update from www.redhat.com/~mingo/ .

Only in 2.4.10pre8aa1: 70_mmap-rb-8
Only in 2.4.10pre10aa1: 70_mmap-rb-9

Rediffed due trivial rejects.

URL:

ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.10pre10aa1.bz2
ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.10pre10aa1/
ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.10pre10aa1/80_vm-aa-1

Some diffstats:

diffstat of the whole patch:

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Next message: Eric W. Biederman: "Re: broken VM in 2.4.10-pre9"
Previous message: Eric W. Biederman: "Re: broken VM in 2.4.10-pre9"
Next in thread: Dipankar Sarma: "Re: 2.4.10pre10aa1 (first spin to the vm rewrite included)"
Reply: Dipankar Sarma: "Re: 2.4.10pre10aa1 (first spin to the vm rewrite included)"
Reply: Olaf Zaplinski: "Re: 2.4.10pre10aa1 (first spin to the vm rewrite included)"