writing to tapes

[Marr]

Jason,

Like Don Flynn, I'd recommend a 'find'-based solution.

Via 'cron', I run an hourly incremental backup via a 'bash' shell script.  
The backup goes to a 2nd hard drive, not tape, but the concept should be 
easily adaptable to tape if that's what you intended.  The script works as 
follows....

First, run 'find' to detect all of the files that need backing up, based on 
their timestamp:

find /important-dir-1 /important-dir-2 /important-dir-3 \
   -path '/important-dir-2/junk-stuff' -prune -o \
   -type f -newer /tmp/increm-bu-tag-file -print \
   >/tmp/increm-bu-list 2>/tmp/increm-bu-errors

The first line lists the 'important' directories to backup.  The second 
(optional) line is a good example of excluding some subdirectory that lives 
below one of the 'important' directories.  For example, I exclude my web 
browser's cache subdirectory this way.  The third and fourth lines cause the 
file '/tmp/increm-bu-tag-file' (explained below) to be used as the 
'compare-to' file to decide which files have been altered (and therefore get 
backed up).  A list of files that need backing up ultimately gets written to 
the '/tmp/increm-bu-list' file (more on that below too).  The error messages 
(if any) go to '/tmp/increm-bu-errors'.

Second, create a big TGZ file of all the files that need backing up with this 
command:

tar cvfz /path-to-backup-device-or-dir/increm-`date +%Y%m%d_%H%M%S`.tgz \
   -T /tmp/increm-bu-list

The 'date' command gives the 'tar' file a name like 
'increm-20020103_173000.tgz' for a Jan 3, 2002 5:30pm creation.  The '-T' 
option gives the filename with the list of files for 'tar' to process.

Third, update the timestamp of the (permanent) file whose timestamp indicates 
the last time our incremental backup was performed:

touch /tmp/increm-bu-tag-file

Finally, delete the now-unneeded file which contains the list of files backed 
up:

rm /tmp/increm-bu-list

Now admittedly, this script could be improved.  One example is that the 
date/time should be saved at the start of the script and the 
'increm-bu-tag-file' should take on that timestamp (once the 'find' is 
completed).  The way it is now, a file that is altered in the short time 
between the time the 'find' is performed and the time the 'touch' is done 
(but not altered afterward) would get forever missed for backup.  On my 
system, this window was too small to worry about, but it's a design flaw 
nonetheless.  Actually, moving the 'touch' command to just after the 'find' 
command would shorten this 'window of failure'.  (However, I do full nightly 
backups to CD-RW anyway, so it's really moot.)

Another caveat is that I have not backed up any SMB shares this way, but I do 
back up parts of a FAT32 partition, so I suspect you'd be OK.

Hope some of this info is helpful to you.

Bill Marr

------------------------------------------------------------------

On Thursday 03 January 2002 08:45am, Jason D. Justman wrote:
> Yes, that would work fine under normal conditions which dump was designed
> for (ext2).  The only problem is the filesystem I am trying to back up  is
> (right now) a smb share on another machine mounted locally.  tar cf
> /dev/st0 /home/raid works just fine, but will result in me wasting about 6
> gigs a day, when only about 5 meg changes (if that).
>
> jason
>
> Mark Szidik wrote:
> > On Thu, 3 Jan 2002, Jason D. Justman wrote:
> > > speaking of which, does anyone know of a good application which will
> > > support incremental backups?  taper says it supports it, but for some
> > > reason says it limits the archives to 2gig.
> >
> > I use dump/restore. (tar too) It can handle 9 levels of
> > incremental backups.  I only use two levels, a level 0 (full
> > backup) and then a higher level (2) for an incremental.
> >
> > 0 full backup
> > 1 all files changes since the last lower level
> > 2 all files changes since the last lower level
> > 3 all files changes since the last lower level
> > etc...
> >
> > I believe that dump works best on idle file systems.  If a
> > filesystem is actively writing then dump will miss this stuff.
> > Ben has mentioned in the past that Linus does not like dump's
> > method of backup, so be careful.
> >
> > I really like the restore programs interactive function.  It is
> > very easy to retrive single files/directories from a dumped file
> > system.
> >
> > -Mark