Truly wild, truly crazy. OK, this is getting interesting. I'll go
read the dbench source now, I really want to understand how the IO and
thread sheduling are interrelated. I'm not even going to try to
advance a theory just yet ;-)
I'd mentioned that dbench seems to run fastest when threads run and
complete all at different times instead of all together. It's easy to
see why this might be so: if the sum of all working sets is bigger than
memory then the system will thrash and do its work much more slowly.
If the threads *can* all run independently (which I think is true of
dbench because it simulates SMB accesses from a number of unrelated
sources) then the optimal strategy is to suspend enough processes so
that all the working sets do fit in memory. Linux has no mechanism for
detecting or responding to such situations (whereas FreeBSD - our
arch-rival in the mm sweepstakes - does) so we sometimes see what are
essentially random variations in scheduling causing very measurable
differences in throughput. (The "butterfly effect" where the beating
wings of a butterfly in Alberta set in motion a chain of events that
culminates with a hurricane in Florida.)
I am not saying this is the effect we're seeing here (the working set
effect, not the butterfly:-) but it is something to keep in mind when
investigating this. There is such a thing as being too fair, and maybe
that's what we're running into here.
-- Daniel - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/