Re: Q: sparse file creation in existing data?

[Ph. Marek]

>For your specific problem I'd suggest the following approach:
>
>write a new filter prog, kind of a 'destructive cat' command.
>
>Open the file for read-modify (non destructive)
>Let it read some blocks (number controllable on commandline) from the
>beginning and pipe them to stdout. Then.. read in same amount (well;
>block aligned) from the end of file and copy it over the beginning,
>truncate the file accordingly. After some time reading and truncating
>will meet in the middle of the file, then you read the blocks back in
>on reverse (and truncate them after reading)
>
>It shouldn't be too difficult to write and result in a multipurpose
>commandline utility.
>
>It does some disk i/o but I don't think it will too bad due to disk
>buffers of the kernel and read-ahead. Theoretically, the kernel, buffers
>etc. could detect you are just moving file blocks to different places and
>does this in a 'zero-copy' fashion by just moving some inode entries
>around, but I doubt that it is soo smart.
>
>Drawback: If you have choosen the read block size too large (hmm.. you
>might want to read some data from beginning, some from the end, copy over
>beginning, then truncate, and only then pipe to stdout: this should
>eliminate this 'buffer/diskspace overrun')  of 'destructive cat' too big,
>or the archive is compressed and doesn't fit uncompressed, the backup was
>destructed. This is by far not as safe as any backup tool should be.
>
>A major advantage, however, is that this tool will run w/o a kernel patch,
>on any Unix on any filesystem, even those w/o holes (I think truncate
>works in general, maybe not samba?) 
I didn't even get this idea - it sound's very io-intensive. 
Of course, if you look at it from the right angle - you'd
- read the first halve,
- read the 2nd halve,
- and write the 2nd halve to the 1st,
- and then read the 1st again.

That's two times the amount of io.
If I assume using tar it's because of writing the data only one third more
io than necessary.

Hmmm, on second thought ... But I'd like it better to have a fcntl for
hole-making :-)
Maybe I'll implement this myself.

Thanks,

Phil