[RFC] Scheduler issue 1, RT tasks ...

[Davide Libenzi]

I'd like to have some comments about RT tasks implementation in a SMP
system because POSIX it's not clear about how the priority rules apply to
multiprocessor systems.
The Balanced Multi Queue Scheduler ( BMQS, http://www.xmailserver.org/linux-patches/mss-2.html )
i'm working on tries to keep CPU schedulers the more independent as
possible and currently implements two kind of RT tasks, local one and
global ones.
Local RT tasks apply POSIX priority rules inside the local CPU, that means
that an RT task running on CPU0 cannot preempt another task ( being it
normal or RT ) on CPU1. This keeps schedulers interlocking very low
because of the very fast path in reschedule_idle() ( no multi lock
acquisition, CPU queue loops, etc...).
Global RT tasks, that live in a separate run queue, have the ability to
preempt remote CPU and this can lead ( in the unfortunate case that the
last CPU running the RT task is running another RT task ) to an higher
cost in reschedule_idle().
The check for a global RT task selection is done in a very fast way before
checking the local queue :

    if (!list_empty(&runqueue_head(RT_QID)))
        goto rt_queue_select;
rt_queue_select_back:

and this does not affect the scheduler latency at all.
On the contrary, by having a separate queue for global RT tasks, can
improve it in high run queue load cases.
The local/global RT task selection is done with setscheduler() with a new
( or'ed ) flag SCHED_RTGLOBAL, and this means that the default is RT task
local.
I'd like to have comments on this before jumping to the next Scheduler
issue ( balancing mode ).

- Davide