Re: Kexec, DMA, and SMP

[Corey Minyard]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Eric W. Biederman wrote:

|Corey Minyard <[e-mail]> writes:
|
|>Eric W. Biederman wrote:
|>
|>|Corey Minyard <[e-mail]> writes:
|>|
|>|>
|>|>You don't understand.  You don't *want* to set aside a block of 
memory that's
|>|>reserved for DMA.  You want to be able to DMA directly into any user 
address.
|>|>Consider demand paging.  The performance would suck if you DMA into some
|>|>fixed region then copied to the user address.  Plus you then have 
another
|>|>resource you have to manage in the kernel.  And you still have to 
change all
|>|>the drivers, buffer management, etc. to add a flag that says "I'm 
going to use
|>
|>|>this for DMA" to allocations.  You might as well add the quiesce 
function,
|>it's
|>
|>|>probably easier to do.  And it doesn't help if you DMA to static memory
|>|>addresses.
|>|>
|>|>I, too, would like a simpler solution.  I just don't think this is it.
|>|
|>|
|>|You have it backwards.  It is not about reserving a block of memory
|>|for DMA.  It is about reserving a block of memory to not do DMA in.
|>|Something like 4MB or so.  |
|>|The idea is not to let the original kernel touch the reserved block 
at all.
|>|We just put the kernel that kexec will start in that block of memory.
|>|
|>|Eric
|>|
|>Ah, it makes much more sense now.  Thank you.  I still don't think 
it's as easy
|>as you think, though.
|>Because there's no designation on most memory allocations to give you this
|>information.  There's
|>GFP_DMA, but according to the docs that's just for x86 ISA DMA 
devices.  You
|>would
|>have to hunt down all the memory allocations, figure out of they are 
DMA targets
|>
|>or not, and add a
|>flag for that.  I still say it's easier to just add the function to 
the drivers.
|
|
|It is trivial if you don't let alloc_pages give the memory to anyone for
|any purpose.

Ok, agreed, if you reserve a section of physical memory just for kexec 
to copy it's kernel into, it will
prevent DMA from clobbering something from the time kexec copies the 
kernel there to the time
decompressing starts.

Another thought.  If you add a delay with all other processors and 
interrupts off, the disk devices
will run out of things to do.

Once you add all the necessary quiesce functions, these can go away.

I do doubt these will make a big difference, though.  The problem we 
were seeing was with the
shared control structures in memory.  The new kernel laid memory out a 
little differently and
things like buffer pointers were overwritten with new data.  This, in 
turn, cause the device to
do random things.  I would guess this is the most likely scenario, since 
the time you are protecting
against with the memory layout is small compared to the time spent booting.

- -Corey
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQE+SnDpmUvlb4BhfF4RAruzAJ9Wts5ovfYf4Ncl+trn755L6JCc6QCfZffG
xGMlv58qX1v3ue0iLwxMRaw=
=aH5T
-----END PGP SIGNATURE-----