Re: Two rapid VT_ACTIVATE's

[Peter Benie]

In article <[e-mail]> you write:
>
>> I wonder what happens when somebody calls VT-ACTIVATE in close succession.
>> So that the second one overwrites the first one. Let's assume the first one is
>> generated by program #1 and the second one by program #2. And some of the programs
>> may be using graphics and some of them text. What will happen? Will screen corruption
>> and/or kernel crash induced by botched videocard registers result?
>
>In theory the vt_dont_switch lock should protect you from this. 

vt_dont_switch has nothing to do with it, and is fundamentally broken
for other reasons. (You need a call to atomically activate and lock a
particular VT, not a call to lock whichever VT happens to be selected
at the moment.)

With the VT_ACTIVATE race, what will typically happen is:

#1 calls VT_ACTIVATE to request the VT change
#2 calls VT_ACTIVATE does likewise

#1 calls VT_WAITACTIVE to wait for the change to complete
#2 calls VT_WAITACTIVE does likewise (and overwrites want_console)

#2 returns from VT_WAITACTIVE (there is now no VT change pending)
#1 does not return until the user explicitly changes to the right
console, or the ioctl is interrupted by a signal

You can see this in practice if you start two X servers from xdm.
After a while, xdm sends a SIGTERM to the X server that hasn't started
yet, interrupting the ioctl, but the X server doesn't check the return
value from the ioctl, assumes the VT change completed, and fiddles
with the video card parameters. Sometimes you get a peculiar mixture of
two X servers, sometimes the machine just crashes.

Clearly, the crash is due to a bug in the X server (ignoring the error
from the ioctl), but the VT API is also faulty since the X server
cannot find out that its VT_ACTIVATE was 'lost' by the kernel.

Another API problem in this area is with VT_OPENQRY. An application
calls this to get the next free VT number, then VT_ACTIVATE to swtich
to it, allocating it if necessary.  If two applications do this, they
will both get the same VT number from VT_OPENQRY, and both will then
call VT_ACTIVATE and VT_SETMODE. Both applications believe they
have a VT which has been allocated and locked, despite only having one
VT between them!

Peter