Re: Memory management in Kernel 2.4.x

[Peter Wächtler]

Andrea Arcangeli wrote:
> On Mon, May 27, 2002 at 03:45:55PM +0200, Peter Wächtler wrote:
> 
>>Andreas Hartmann wrote:
>>
>>>Zwane Mwaikambo wrote:
>>>
>>>
>>>
>>>>On Mon, 27 May 2002, Andreas Hartmann wrote:
>>>>
>>>>
>>>>
>>>>>rsync allocates all of the memory the machine has (256 MB RAM, 128 MB
>>>>>swap). When this occures, processes get killed like described in the
>>>>>posting before. The machine doesn't respond as long as the rsync -
>>>>>process isn't killed, because it fetches all the memory which gets free
>>>>>after a process has been killed.
>>>>>
>>>>>
>>>>And the rsync process never gets singled out? nice!
>>>>
>>>>
>>>Until it's killed by the kernel (if overcommitment isn't deactivated). If 
>>>overcommitment is deactivated, the services of the machine are dead 
>>>forever. There will be nothing, which kills such a process. Or am I wrong?
>>>
>>>
>>There is still the oom killer (Out Of Memory).
>>But it doesn't trigger and the machine pages "forever".
>>Usually kswapd eats the CPU then, discarding and reloading pages,
>>searching lists for pages to evict and so on.
>>
> 
> can you reproduce with 2.4.19pre9aa2? I expect at least the deadlock
> (if it's a deadlock and not a livelock) should go away.
> 
> Also I read in another email that somebody grown the per-socket buffer
> to hundred mbytes, if you do that on a 32bit arch with highmem you'll
> run into troubles, too much ZONE_NORMAL ram will be constantly pinned
> for the tcp pipeline and the machine can enter livelocks.
> 

Sorry for the confusion.
I was just trying to explain that without overcommitment there would be
the "normal" OOM handling. But I don't know this feature from the -ac
kernels. Am I wrong?

It's Andreas who has the problem with rsync being killed and the machine
seems to hang.
But I still think that the buffer cache has to be better restricted.
The vm is caching far too aggressively (but I never tried with -aa)