Re: [PATCH RFC] 1/2 central workspace for zlib

[Jörn Engel]

On Sat, 31 May 2003 01:29:42 +1000, James Morris wrote:
> 
> In 2.5 (and soon 2.4), the crypto/deflate.c module makes use of zlib as
> well.  This is typically used in ipsec for ipcomp.  Each ipcomp security
> association has a zlib context, and access to the workspace is serialized 
> by the security association's bh lock.
> 
> (Something needs to be done for this particular case in general: a box
> with 1000 tunnels could use use 450MB of atomic kernel memory just for
> zlib workspaces alone.  A solution I've been looking at for this is to
> allow workspaces to be dynamically sized based on the compression
> parameters instead of using worst-case figures.  The memory savings may be
> up to 90% in this case).

Or 99.9%, if you map all those workspaces to one, similar to my patch.
The softirq context requires another set of workspaces, but in
principle the same could be done.  Downside is that the Init- and End-
functions have to be called more often, which will surely cost cpu
time.  I haven't done any benchmarks yet, so it remains to be shown if
this is relevant.

Your approach basically reintroduces the zmalloc() and zfree()
functions and makes behaviour under memory pressure interesting.  It
is not impossible to get it right, but quite hard for sure.

> > This patch creates an extra workspace of 400k per cpu, that is used for
> > both inflate and deflate.  One of the central workspaces is used for
> > users that don't provide their own.  Semaphore protection is done in
> > zlib_(in|de)flateInit() and zlib_(in|de)flateEnd, so the user has to
> > call those functions more often to release the semaphores before
> > returning to userspace.
> 
> This won't work for the bh lock protected case outlined above, and will
> cause contention between different users of zlib.

Image a 2-cpu machine that does reads and writes to jffs2 on two
devices simultaneously.  When one process is reading and one is
writing, everything is fine.  When both perform the same operation,
the current design makes one wait at the semaphore, while one cpu is
idle.  In my design, you have one workspace per cpu, so both can work
simultaneously.

What contention were you talking about? :)

The bh lock is another issue.  Here we need another set of workspaces,
independent of the existing one (with or without my patch).  But in
principly, the same should be possible.

Jörn

-- 
The grand essentials of happiness are: something to do, something to
love, and something to hope for.
-- Allan K. Chalmers