Re: Is sendfile all that sexy?

[Rogier Wolff]

Linus Torvalds wrote:
> In article <[e-mail]>,
> Ben Mansell  <[e-mail]> wrote:
> >On 14 Jan 2001, Linus Torvalds wrote:
> >
> >> And no, I don't actually hink that sendfile() is all that hot. It was
> >> _very_ easy to implement, and can be considered a 5-minute hack to give
> >> a feature that fit very well in the MM architecture, and that the Apache
> >> folks had already been using on other architectures.
> >
> >The current sendfile() has the limitation that it can't read data from
> >a socket. Would it be another 5-minute hack to remove this limitation, so
> >you could sendfile between sockets? Now _that_ would be sexy :)
> 
> I don't think that would be all that sexy at all.
> 
> You have to realize, that sendfile() is meant as an optimization, by
> being able to re-use the same buffers that act as the in-kernel page
> cache as buffers for sending data. So you avoid one copy.
> 
> However, for socket->socket, we would not have such an advantage.  A
> socket->socket sendfile() would not avoid any copies the way the
> networking is done today.  That _may_ change, of course.  But it might
> not.  And I'd rather tell people using sendfile() that you get EINVAL if
> it isn't able to optimize the transfer.. 

Linus, 

I admire your good taste in designing interface, but here is one where
we disagree.

I'd prefer an interface that says "copy this fd to that one, and
optimize that if you can".

All cases that can't be optimized would end up doing an in-kernel read
/ write loop. Sure, there is no advantage above doing that same loop
in userspace, but this way the kernel can "grow" and optimize more
stuff later on.

For example, copying a file from one disk to another. I'm pretty sure
that some efficiency can be gained if you don't need to handle the
possibility of the userspace program accessing the data in between the
read and the write. Sure this may not qualify as a "trivial
optimization, that can be done with the existing infrastructure" right
now, but programs that want to indicate "kernel, please optimize this
if you can" can say so.

Currently, once the optimization happens to become possible (*), we'll
have to upgrade all apps that happen to be able to use it. If now we
start advertizing the interface (at a cost of a read/write loop in the
kernel: five lines of code) we will be able to upgrade the kernel, and
automatically improve the performance of every app that happens to use
the interface.

			Roger. 

(*) Either because the infrastructure makes it "trivial", or because
someone convinces you that it is a valid optimization that makes a
huge difference in an important case.

-- 
** [e-mail] ** http://www.BitWizard.nl/ ** [phone] **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* There are old pilots, and there are bold pilots. 
* There are also old, bald pilots.