Re: 15000+ processes -- poor performance ?!

[Denis Vlasenko]

On 18 December 2002 22:46, Till Immanuel Patzschke wrote:
> Dear List(s),
>
> as part of my project I need to run a very high number of
> processes/threads on a linux machine.  Right now I have a Dual-PIII
> 1.4G w/ 8GB RAM -- I am running 4000 processes w/ 2-3 threads each
> totaling in a process count of 15000+ processes (since Linux doesn't
> really distinguish between threads and processes...).

BTW, can you say _what_ are you trying to do?

> Once I pass the 10000 (+/-) pocesses load increases drastically (on
> startup, although it returns to normal), however the system time (on
> one processor) reaches for 54% (12061 procs) while the only non
> sleeping process is top -- the system is basically doing nothing
> (except scheduling the "nothing" which consumes significant system
> time).
> Is there anything I can do to reduce that system load/time?  (I
> haven't been able to exactly define the "line" but it definitly gets
> worse the more processes need to be handled.)
> Does any of the patchsets address this particular problem?
> BTW: The processes are all alike...

You need to collect memory info (especially lowmem and highmem situation)
and maybe profile your kernel to find out where does it spend that time
doing "nothing".

BTW, your .config?
--
vda