Scheduling problem with 2.4?

[David Kastrup]

I have a problem with Emacs which crawls when used as a shell.  If I
call M-x shell RET and then do something like
hexdump -v /dev/null|dd count=100k bs=1
then characters appear rather slowly at first, most of them in single
character packets, with an occasional 1024 packet in between.  More
of them at the end.  The rate of the packets does not exceed 100/sec.

Other operating systems do not exhibit this extreme case of
throttling.  Emacs is using select calls AFAICS for dealing with
packets arriving, and it would appear that the delivering process
causes a context switch immediately when writing its single byte to
the pipe or is taken from the run queue while the select call is
pending.  Whatever.  In the ensuing time slice, Emacs processes the
single arriving byte and then enters the select call again.

Now that the pipe is not allowed to fill up in most cases (most: in
some, increasingly more at the later time, the pipe gets filled
alternately complete and with single bytes) before a context switch
occurs is already unfortunate with regard to the efficiency of
processing.  What is quite more of a nuisance is the apparent
throttling to a single byte per clock tick: the CPU is idling most of
the time.  It would be unfortunate enough if it were going full blast
at processing 1-byte packets, but most of the time it does nothing
but wait for the next tick.

Now I would want to

a) get sufficient information about what might go wrong inside of
Linux as compared to other operating systems.

b) find a good workaround for not triggering this kind of behavior.

I don't have sufficient kernel experience to debug this one, and it
definitely impacts the speed of operations severely, and in close
cooperation with the operating system.  So any pointers would be
welcome.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum