Re: Breaking system configuration in stable kernels

[Bill Davidsen]

On Mon, 8 Oct 2001, Chris Siebenmann wrote:

> You write:
> |   I've beaten this dead horse before, but Linux will not look to
> | management like a viable candidate for default o/s until whoever releases
> | new versions of *stable* kernel series with cosmetic changes which break
> | existing systems running earlier releases of the same stable kernel
> | series.
> [...]
> |   I love getting problems like this on my vacation, [...]
> 
>  Although I sympathise, I have to ask: why are you rolling new kernels
> (or new anythings) onto production servers without testing and a
> reversion plan? In years of experience with all sorts of vendors as well
> as Linux, I have yet to see *anyone* be reliable about this (the worst
> offender I've ever had to deal with was SGI, who would release 'urgent'
> kernel patches with crash bugs).

I certainly had a reversion plan and backups, but I don't have in-house
hardware duplicating every client configuration. But there's only so much
testing you can do before you put user load on the machine. Also, existing
machines doesn't mean irreplacable mission critical machines, I don't
upgrade unless I have a reason, and piss-poor VM performance certainly is
a reason.
 
>  I strive to never put anything on our servers that I have not tested
> on an expendable machine that is configured as closely as possible to
> the server. And I also try not to be anywhere near the leading edge
> on production servers, unless there is some strong benefit to it.
> 
>  The only time I roll something out 'right now stat!' is when it is
> an urgent security fix.

I don't have that luxury. I can't afford to have a copy of most machines,
and after load and stability testing I have to try on production machines.
None of which addresses the issue that parameter name changes buy nothing
functional, and can cause serious problems. If the module in question was
needed for boot and in the initrd file the system wouldn't boot at all and
there would be little to tell why.

I'd like Linux to be more widely used, and things like this are not
invisible to the people who choose standards.

-- 
bill davidsen <[e-mail]>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.