[clug] Scrubbing non-allocated blocks on ext2

[dave]

Hi Paul,
If you are after a user-space tool which will tell you which blocks are
occupied by which files, sleuthkit is a good option.

I'm a developer of a forensics tool called pyflag. I built a tool
against sleuthkit which determines block allocation for all files in a
filesystem and places this data in a database for use by the rest of
pyflag. Pyflag does something simillar to what you describe, it locates
all unallocated blocks and slack space at the end of allocated blocks
and performs special analysis of this area of the disk. One could
convievably use this data to seek to all these areas and wipe them out.
However I certainly wouldnt recommend it without thorough testing as
sleuthkit/pyflag may not return all filesystem metadata blocks as being
allocated to real files, and it would suck to (say) accidentally zero
out a bunch of important inodes.

A much easier solution is (as you suggested), copy (cp -a) all your
files to another unix filesystem (to preserve unix file semantics),
trash the device, create a new filesystem, the copy them all back again.
This method should not retain any unallocated data of the original
partition.

Dave

On Wed, Jul 20, 2005 at 04:24:12PM +1000, Paul Wayper wrote:
> Hi all,
> 
> So far my fairly extensive researches have shown that
> there's no program that will go through an ext2 filesystem
> and basically remove evidence of deleted files.  AFAICS,
> this would require three stages:
> 
> 1) Go through the directory structure and reorganise
> directories.
> 2) Construct a list of all blocks, and partial blocks, not
> in use.
> 3) Scrub those blocks.
> 
> Step 1 is important because, from what I read, when you
> delete a file its space is reallocated to the previous file
> as room to grow into.  Or maybe not - I haven't absorbed the
> entire "Design and Implementation of the Second Extended
> File System" yet.  Either way, standard file system design
> means that you don't immediately try to reallocate those
> blocks to the next file to be written, you try and localise
> reference and/or choose a chunk of free space that suits
> your new file.  Deleted files still remain in the directory
> but are marked.
> 
> Yes, I know about the 's' attribute for file systems, that's
> supposed to force the kernel to overwrite the space with
> random data, or zeroes, or smiley faces.  It doesn't work;
> it hasn't since about kernel 2.2.something, when Linus found
> that it caused a bit of a slowdown and didn't work in all
> cases.  No-one shows any sign of trying to reimplement it.
> 
> There used to be a defragmenter for ext2, but it's out of
> date as it assumes that blocks are 1K in size.  Yes, ext2
> does actually suffer from fragmentation, but magic pixies
> somewhere in the OS make it less of an issue.  I'll
> understand how those magic pixies do their work when I've
> finished reading the above document.  A defragmenter would
> do a third of what I want; it would be a good starting
> point.
> 
> I know about wipe, shred and similar programs that will
> erase a file and overwrite it.  If I'd thought about using
> them, then I would have.  It's too late now.  I know about
> The Defiler's Toolkit, and it seems to sort of work except
> for the fact that even the most elementary probing with
> debugfs shows the files that necrofile is supposed to have
> removed all trace of, and klismafile doesn't seem to work in
> the way the example shows.  Maybe it's the fact that the
> disk I'm testing it on is in a USB case.  It certainly seems
> to be synced, anyway.
> 
> So I thought I'd call upon the wisdom of the CLUG.  Is there
> something that will defragment a disk?  Is it just going to
> be down to doing a dump backup, wiping the disk, and then
> restoring it?  Is this, indeed, enough?  Is there a better
> way?
> 
> I'm assuming that actually trying to learn how write a
> program to do this will result in extreme concussion from
> hitting my head against the brick wall too much.
> 
> Have fun,
> 
> Paul
> -- 
> linux mailing list
> linux at lists.samba.org
> https://lists.samba.org/mailman/listinfo/linux
>