Backup Server

IRIDIA Backup System

Personal Backups

Everyone has 20GB available space on fallopius. You are responsible for backup up your own data into the space you have allocated on fallopius.

Push Backup

Backups are initiated from the client machine ("pushed" onto the server).

You can use you personal favourite to push your backups on the server. Please make sure that your data does not exceed the limit (i.e. purge old data regularly).

Examples of good backup programs are rdiff-backup, rsync, hdup, unison and many others. See below for a list .

Pull Backup

Backups are initiated from the backup server ("pulled").

Halva's old system

There is an old automated backup system that is still in place, but no longer supported. It was written by Halva and uses rdiff-backup. A script pulls the contents of the user's home directory onto the backup server. Instructions here: Old Backup System (Halva's rdiff system). Good luck.

Clean up some of your used space

Imagining that you don't need the entire history of your backup, you can use rdiff to delete files that are older than a specified amount of time.

 /usr/bin/rdiff-backup -v0 --force --remove-older-than 3W <fallopiususername>@fallopius::/home/<fallopiususername>/backup/<localusername>@<localmachine>.ulb.ac.be/

Please use once in a while

Server Backup

iridia.ulb.ac.be

All data on the iridia server is backed up by a separate (supported) nightly script that uses dar (see below). This is an incremental backup program that is like a network version of tar. The scripts are run by the crontab of the user servicebackup on iridia using the script /home/servicebackup/iridia_service_backup_system/backup.sh. The shell functions that do the meat of the work are included from the file include.sh in the same directory.

The backed up data is stored in the directory are stored on fallopius in the directory /home/iridiaserver/service_backup_dar. To restore data you will need to familiarise yourself with the dar program, or use the script /home/servicebackup/iridia_service_backup_system/restore.sh on iridia.

The data that is backed up is divided into 'services'. A list of these services can be found in the directory /home/servicebackup/iridia_service_backup_system/services. Each service is a directory that must at a minimum contain scripts called <SERVICE_NAME>_backup.sh and <SERVICE_NAME>_restore.sh. These two scripts should make use of the functions provided by the file /home/servicebackup/iridia_service_backup_system/include.sh which provide wrappers around the dar commands. Optionally, inside each service directory there can be scripts called <SERVICE_NAME>_suspend.sh and <SERVICE_NAME>_resume.sh which will be called before and after the backup, and can be used to stop and start any processes that might interfere with the integrity of the backed up data if they are left running.

A mail gets sent every day to a list of addresses defined in the backup.sh script with the success / failure of the backup in the subject line.

Troubleshooting

Logfiles for each backup are stored in /home/servicebackup/iridia_service_backup_system/log on iridia.ulb.ac.be

A quick way of debugging the system is to run the script /home/servicebackup/iridia_service_backup_system/just_test_backup.sh . This script does a test backup of a small directory and outputs to the logfile test.log.

There is an ugly soft link from /usr/lib/libdar.so.3.0.1 -> /home/iridiaserver/dar/lib/libdar.so.3.0.1 This link has been known to disappear after an upgrade of the system.

svn

The scripting system is maintained and versioned in the svn repository: file:///usr/local/share/svn_repositories/iridia_service_backup_system

Backup Software

dar

DAR (http://dar.sourceforge.net/) is an incremental backup program that is like a network version of tar. It is used by the iridia server.

rdiff-backup

The current supported version is 1.1.17. Follow the steps below for Debian etch:

 $ wget http://iridia.ulb.ac.be/~abrutschy/rdiff-backup_1.1.17-1_etch_i386.deb
 $ sudo dpkg -i rdiff-backup_1.1.17-1_etch_i386.deb

and for Ubuntu Intrepid/Debian Lenny:

 $ wget http://iridia.ulb.ac.be/~abrutschy/rdiff-backup_1.1.17-1_lenny_intrepid_i386.deb
 $ sudo dpkg -i rdiff-backup_1.1.17-1_lenny_intrepid_i386.deb

and for Ubuntu Jaunty:

 $ wget http://iridia.ulb.ac.be/~abrutschy/rdiff-backup_1.1.17-1_jaunty_i386.deb
 $ sudo dpkg -i rdiff-backup_1.1.17-1_jaunty_i386.deb

In case you use another system, download and install from the source ( http://iridia.ulb.ac.be/wiki/images/1/18/Rdiff-backup_1.1.17.orig.tar.gz or http://download.savannah.gnu.org/releases/rdiff-backup/rdiff-backup-1.1.17.tar.gz). Note that it could not work if your system has a version that is different).

Add the following lines to "/etc/apt/preferences" in order to prevent apt to install another version of rdiff-backup:

 Package: rdiff-backup
 Pin: version 1.1.17-1
 Pin-Priority: 1001

Unison

If you know unison, note that you can also use that for your personal backups. Installed version is 2.27.57. For documentation look here http://www.cis.upenn.edu/~bcpierce/unison/.