Re: Editing-in-place of a large file

[Ingo Oeser]

On Sun, Sep 02, 2001 at 05:59:38PM -0700, Larry McVoy wrote:
> > What's needed is a generalisation of sparse files and truncate().
> > They both handle similar problems.
> 
> how about 
> 
> 	fzero(int fd, off_t off, size_t len)
> 	fdelete(int fd, off_t off, size_t len)
 
and 

   finsert(int fd, off_t off, size_t len, void *buf, size_t buflen)

> The main problem with this is if the off/len are not block aligned.  If they
> are, then this is just block twiddling, if they aren't, then this is a file
> rewrite anyway.

Yes, that's why I solved this in user space by implementing a C++
stream consisting of multiple mmaps() of files and anonymous
memory. I needed this for someone editing audio streams.

It's basically creating a binary diff ;-)

Another solution for the original problem is to rewrite the file
in-place by coping from the end of the gap to the beginning of
the gap until the gap is shifted to the end of the file and thus
can be left to ftruncate().

This will at least not require more space on disk, but will take
quite a while and risk data corruption for this file in case of
abortion.

But fzero, fdelete and finsert might be worth considering, since
some file systems, which pack tails could also pack these kind of
partial used blocks and handle them properly. 

We already handle partial pages, so why not handle them with
offset/size pairs and enable this mechanisms? Multi media streams
would love these kind of APIs ;-)

Regards

Ingo Oeser