Difference between revisions of "Backup Server"
| Line 1: | Line 1: | ||
| − | + | = IRIDIA Backup System = |
|
| + | == Personal Backups == |
||
| − | Everyone has 20GB available space on fallopius. You are responsible for backup up your own data into the space you have allocated on fallopius. There is an old automated backup system that is still in place, but no longer supported. It was written by Halva and uses rdiff-backup. (Current supported version is 1.1.17. Download from here the source http://iridia.ulb.ac.be/wiki/images/1/18/Rdiff-backup_1.1.17.orig.tar.gz ). Note that it could not work if your system has a version that is different). Instructions here: [[Old Backup System (Halva's rdiff system)]]. Good luck. |
||
| + | Everyone has 20GB available space on fallopius. You are responsible for backup up your own data into the space you have allocated on fallopius. |
||
| ⚫ | |||
| + | === Push Backup === |
||
| ⚫ | All data on the iridia server is backed up by a separate (supported) nightly script that uses dar. This is an incremental backup program that is like a network version of tar. The scripts are run by the crontab of the user servicebackup on iridia using the script /home/servicebackup/iridia_service_backup_system/backup.sh. The shell functions that do the meat of the work are included from the file include.sh in the same directory. |
||
| + | |||
| + | Backups are initiated from the client machine ("pushed" onto the server). |
||
| + | |||
| + | You can use you personal favourite to push your backups on the server. Please make sure that your data does not exceed the limit (i.e. purge old data regularly). |
||
| + | |||
| + | Examples of good backup programs are rdiff-backup, rsync, hdup, unison and many others. See below for a list . |
||
| + | |||
| + | === Pull Backup === |
||
| + | |||
| + | Backups are initiated from the backup server ("pulled"). |
||
| + | |||
| + | ==== Halva's old system ==== |
||
| + | |||
| + | There is an old automated backup system that is still in place, but no longer supported. It was written by Halva and uses rdiff-backup. A script pulls the contents of the user's home directory onto the backup server. Instructions here: [[Old Backup System (Halva's rdiff system)]]. Good luck. |
||
| + | |||
| + | |||
| + | == Server Backup == |
||
| + | |||
| + | === iridia.ulb.ac.be === |
||
| + | |||
| ⚫ | All data on the iridia server is backed up by a separate (supported) nightly script that uses dar (see below). This is an incremental backup program that is like a network version of tar. The scripts are run by the crontab of the user servicebackup on iridia using the script /home/servicebackup/iridia_service_backup_system/backup.sh. The shell functions that do the meat of the work are included from the file include.sh in the same directory. |
||
The backed up data is stored in the directory are stored on fallopius in the directory /home/iridiaserver/service_backup_dar. |
The backed up data is stored in the directory are stored on fallopius in the directory /home/iridiaserver/service_backup_dar. |
||
| Line 14: | Line 35: | ||
A mail gets sent every day to a list of addresses defined in the backup.sh script with the success / failure of the backup in the subject line. |
A mail gets sent every day to a list of addresses defined in the backup.sh script with the success / failure of the backup in the subject line. |
||
| − | + | ===== Troubleshooting ===== |
|
Logfiles for each backup are stored in /home/servicebackup/iridia_service_backup_system/log on iridia.ulb.ac.be |
Logfiles for each backup are stored in /home/servicebackup/iridia_service_backup_system/log on iridia.ulb.ac.be |
||
| Line 22: | Line 43: | ||
There is an ugly soft link from /usr/lib/libdar.so.3.0.1 -> /home/iridiaserver/dar/lib/libdar.so.3.0.1 This link has been known to disappear after an upgrade of the system. |
There is an ugly soft link from /usr/lib/libdar.so.3.0.1 -> /home/iridiaserver/dar/lib/libdar.so.3.0.1 This link has been known to disappear after an upgrade of the system. |
||
| + | ===== svn ===== |
||
| − | ;svn |
||
The scripting system is maintained and versioned in the svn repository: |
The scripting system is maintained and versioned in the svn repository: |
||
file:///usr/local/share/svn_repositories/iridia_service_backup_system |
file:///usr/local/share/svn_repositories/iridia_service_backup_system |
||
| + | |||
| + | |||
| + | == Backup Software == |
||
| + | |||
| + | === dar === |
||
| + | |||
| + | DAR (http://dar.sourceforge.net/) is an incremental backup program that is like a network version of tar. It is used by the iridia server. |
||
| + | |||
| + | === rdiff-backup === |
||
| + | |||
| + | The current supported version is 1.1.17. Follow the steps below for Debian etch: |
||
| + | |||
| + | $ wget http://iridia.ulb.ac.be/~abrutschy/rdiff-backup_1.1.17-1_etch_i386.deb |
||
| + | $ dpkg -i rdiff-backup_1.1.17-1_etch_i386.deb |
||
| + | |||
| + | and for Ubuntu Intrepid/Debian Lenny: |
||
| + | |||
| + | $ wget http://iridia.ulb.ac.be/~abrutschy/rdiff-backup_1.1.17-1_lenny_intrepid_i386.deb |
||
| + | $ dpkg -i rdiff-backup_1.1.17-1_lenny_intrepid_i386.deb |
||
| + | |||
| + | In case you use another system, download and install from the source ( http://iridia.ulb.ac.be/wiki/images/1/18/Rdiff-backup_1.1.17.orig.tar.gz ). Note that it could not work if your system has a version that is different). |
||
| + | |||
| + | === Unison === |
||
| + | |||
| ⚫ | |||
Revision as of 19:19, 16 February 2009
IRIDIA Backup System
Personal Backups
Everyone has 20GB available space on fallopius. You are responsible for backup up your own data into the space you have allocated on fallopius.
Push Backup
Backups are initiated from the client machine ("pushed" onto the server).
You can use you personal favourite to push your backups on the server. Please make sure that your data does not exceed the limit (i.e. purge old data regularly).
Examples of good backup programs are rdiff-backup, rsync, hdup, unison and many others. See below for a list .
Pull Backup
Backups are initiated from the backup server ("pulled").
Halva's old system
There is an old automated backup system that is still in place, but no longer supported. It was written by Halva and uses rdiff-backup. A script pulls the contents of the user's home directory onto the backup server. Instructions here: Old Backup System (Halva's rdiff system). Good luck.
Server Backup
iridia.ulb.ac.be
All data on the iridia server is backed up by a separate (supported) nightly script that uses dar (see below). This is an incremental backup program that is like a network version of tar. The scripts are run by the crontab of the user servicebackup on iridia using the script /home/servicebackup/iridia_service_backup_system/backup.sh. The shell functions that do the meat of the work are included from the file include.sh in the same directory.
The backed up data is stored in the directory are stored on fallopius in the directory /home/iridiaserver/service_backup_dar. To restore data you will need to familiarise yourself with the dar program, or use the script /home/servicebackup/iridia_service_backup_system/restore.sh on iridia.
The data that is backed up is divided into 'services'. A list of these services can be found in the directory /home/servicebackup/iridia_service_backup_system/services. Each service is a directory that must at a minimum contain scripts called <SERVICE_NAME>_backup.sh and <SERVICE_NAME>_restore.sh. These two scripts should make use of the functions provided by the file /home/servicebackup/iridia_service_backup_system/include.sh which provide wrappers around the dar commands. Optionally, inside each service directory there can be scripts called <SERVICE_NAME>_suspend.sh and <SERVICE_NAME>_resume.sh which will be called before and after the backup, and can be used to stop and start any processes that might interfere with the integrity of the backed up data if they are left running.
A mail gets sent every day to a list of addresses defined in the backup.sh script with the success / failure of the backup in the subject line.
Troubleshooting
Logfiles for each backup are stored in /home/servicebackup/iridia_service_backup_system/log on iridia.ulb.ac.be
A quick way of debugging the system is to run the script /home/servicebackup/iridia_service_backup_system/just_test_backup.sh . This script does a test backup of a small directory and outputs to the logfile test.log.
There is an ugly soft link from /usr/lib/libdar.so.3.0.1 -> /home/iridiaserver/dar/lib/libdar.so.3.0.1 This link has been known to disappear after an upgrade of the system.
svn
The scripting system is maintained and versioned in the svn repository: file:///usr/local/share/svn_repositories/iridia_service_backup_system
Backup Software
dar
DAR (http://dar.sourceforge.net/) is an incremental backup program that is like a network version of tar. It is used by the iridia server.
rdiff-backup
The current supported version is 1.1.17. Follow the steps below for Debian etch:
$ wget http://iridia.ulb.ac.be/~abrutschy/rdiff-backup_1.1.17-1_etch_i386.deb $ dpkg -i rdiff-backup_1.1.17-1_etch_i386.deb
and for Ubuntu Intrepid/Debian Lenny:
$ wget http://iridia.ulb.ac.be/~abrutschy/rdiff-backup_1.1.17-1_lenny_intrepid_i386.deb $ dpkg -i rdiff-backup_1.1.17-1_lenny_intrepid_i386.deb
In case you use another system, download and install from the source ( http://iridia.ulb.ac.be/wiki/images/1/18/Rdiff-backup_1.1.17.orig.tar.gz ). Note that it could not work if your system has a version that is different).
Unison
If you know unison, note that you can also use that for your personal backups. Installed version is 2.27.57. For documentation look here http://www.cis.upenn.edu/~bcpierce/unison/.
