"User does not understand how *NIX systems utilize memory" is
not the same thing as "bug". What you are seeing is the kernel
using memory for file system cache.
On 03/22/2001 12:58 -0800, Craig Cummings wrote:
>> Hi there,
>>=09
>> I think this qualifies as a bug but let me know if this could be a
>> configuration or hardware issue.
>>=09
>> I've been having problems with memory leaks when I run programs on
>> large--up to 250MB text files. (I know this is huge, but that's the 3
>> billion base human genome for you.) At first I though it was a Perl
>> problem but I later found that a completely unrelated C program also
>> caused memory leaks. I recently upgraded to the 2.4 kernel, hoping to
>> solve these problems (see below). However, the memory leaks are still
>> happening and this time I know the problem is at a deeper level than my
>> programs. Some standard UNIX programs are leaking a lot of memory. I
>> would appreciate some advice and ultimately, a fix. Unfortunately, My
>> programming skills are not sufficient for tinkering with the kernel
>> source. Thank you, in advance for your help. Details follow.
>>=09
>> Regards,
>>=09
>> Craig Cummings
>>=09
>>=09
>> Here are the specs for my system:
>> Dell Precision XPSt700, Pentium III, 512 MB RAM
>> I've recently upgraded from Red Hat 6.2 with the 2.2.14 kernel to
>> Red Hat 7, then built the 2.4.2 kernel on my own.
>>=09
>> Here's what happens with grep:
>>=09
>> Output of free, freshly booted system:
>>=09
>> total used free shared buffers cached
>> Mem: 513616 47516 466100 0 2476 27048
>> -/+ buffers/cache: 17992 495624
>> Swap: 128480 0 128480
>>=09
>> Output of free after grep 'NT_005289' Data/hs_chr12.fa:
>>=09
>> total used free shared buffers cached
>> Mem: 513616 183548 330068 0 2624 159616
>> -/+ buffers/cache: 21308 492308
>> Swap: 128480 0 128480
>>=09
>> Output of grep 'NT_005289' Data/hs_chr2.fa:
>>=09
>> >gi|12728771|ref|NT_005289.2|Hs2_5446 Homo sapiens chromosome 2 working =
draft sequence segment
>>=09
>> Output of free after this grep:
>>=09
>> total used free shared buffers cached
>> Mem: 513616 424272 89344 0 2860 394232
>> -/+ buffers/cache: 27180 486436
>> Swap: 128480 0 128480
>>=09
>> Output of grep 'NT_005289' Data/hs_chr2.fa:
>>=09
>> >gi|12728771|ref|NT_005289.2|Hs2_5446 Homo sapiens chromosome 2 working =
draft sequence segment
>>=09
>> Output of free after this grep:
>>=09
>> total used free shared buffers cached
>> Mem: 513616 424272 89344 0 2860 394232
>> -/+ buffers/cache: 27180 486436
>> Swap: 128480 0 128480
>>=09
>> File sizes of the two files grep'ed:
>>=09
>> -rw-rw-r-- 1 cummings genomics 135744469 Mar 12 22:09 Data/hs_chr12.fa
>> -rw-rw-r-- 1 cummings genomics 240244039 Mar 12 22:24 Data/hs_chr2.fa
>>=09
>> Note that these file sizes are equivalent to the amount of memory leaked
>> when grep is called on that file.
>>=09
>> When I grep the same file a second time, very little additional memory is
>> leaked.
>>=09
>>=09
>> This same phenomenon occurs when I run a different UNIX program, e.g. wc:
>>=09
>> Output of wc -l Data/hs_chr3.fa:
>>=09
>> 2915465 Data/hs_chr3.fa
>>=09
>> Output of free:
>>=09
>> total used free shared buffers cached
>> Mem: 513616 511520 2096 0 1252 481020
>> -/+ buffers/cache: 29248 484368
>> Swap: 128480 0 128480
>>=09
>> Interestingly, after running wc a second time on the same file, it goes
>> very fast and very little additional memory is leaked:
>>=09
>> total used free shared buffers cached
>> Mem: 513616 510732 2884 0 1204 480948
>> -/+ buffers/cache: 28580 485036
>> Swap: 128480 40 128440
>>=09
>>=09
>> -------------------------------------------
>> Craig Cummings, Ph.D.
>>=09
>> Relman Laboratory
>> Stanford University School of Medicine
>> Department of Microbiology and Immunology
>>=09
>> e-mail: cummings@cmgm.stanford.edu
>> phone: 650-498-5998
>> fax: 650-852-3291
>>=09
>> -
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" =
in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at http://www.tux.org/lkml/
End of included message
--=20
+--------------------------+------------------------------+
| Tim Walberg | tewalberg@mediaone.net |
| 828 Marshall Ct. | www.concentric.net/~twalberg |
| Palatine, IL 60074 | |
+--------------------------+------------------------------+
--azLHFNyN32YCQGCU
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: PGP 6.5.1i
iQA/AwUBOrpyDsPlnI9tqyVmEQLRtQCfVYZ7E773WmqDv5t/e+DlDBwBivoAmgPp
wEmZUwBXNWhnmreNOkY5cprp
=/cm/
-----END PGP SIGNATURE-----
--azLHFNyN32YCQGCU--
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/