Backups
From Peyton Hall Documentation
m |
Current revision (15:32, 26 July 2013) (view source) (Backup changes due to OIT's file server being /u) |
||
| Line 3: | Line 3: | ||
== What is backed up? == | == What is backed up? == | ||
| - | Unfortunately, backing up data (or keeping it on highly reliable media) can be expensive. Because of that, we cannot afford to backup everything. However, everything that is stored in /u is backed up, as well as kept on a very reliable NFS server | + | Unfortunately, backing up data (or keeping it on highly reliable media) can be expensive. Because of that, we cannot afford to backup everything. However, everything that is stored in /u is backed up, as well as kept on a very reliable NFS server maintained by OIT. We also backup everything from the email server. |
'''''We do not backup scratch disks!''''' The very nature of a scratch disk is that it's used for temporary storage of data and intermediate files; perhaps a local copy of data you have on CD elsewhere, or some outputs for in-between states of a data pipeline. Read [[Scratch disks]] for more information. | '''''We do not backup scratch disks!''''' The very nature of a scratch disk is that it's used for temporary storage of data and intermediate files; perhaps a local copy of data you have on CD elsewhere, or some outputs for in-between states of a data pipeline. Read [[Scratch disks]] for more information. | ||
| Line 9: | Line 9: | ||
== On-site backups == | == On-site backups == | ||
=== Snapshots === | === Snapshots === | ||
| - | The | + | The OIT-maintained file server (which stores your home directory in /u) has a feature available called snapshotting. Every two hours a "snapshot" of the current state of your files is taken and frozen. This is useful if you accidentally delete a file or discover that a file is corrupt, as it means you can "roll back the clock" so to speak and recover an earlier version of your file. |
Snapshots for your directory are located in the ~/.snapshot directory (you won't see it in a directory list, but it *is* there if you '<tt>cd</tt>' to it). Under that directory you'll see other directories with names like "<tt>hourly.0</tt>" or "<tt>nightly.0</tt>". The higher the number, the older the snapshot is. <tt>hourly.0</tt> was taken at the last even hour, <tt>hourly.1</tt> was from two hours before that, and so on. The nightly snapshots are taken at midnight, so <tt>nightly.0</tt> is the state of your files at midnight this morning, and <tt>nightly.1</tt> is midnight the day before. | Snapshots for your directory are located in the ~/.snapshot directory (you won't see it in a directory list, but it *is* there if you '<tt>cd</tt>' to it). Under that directory you'll see other directories with names like "<tt>hourly.0</tt>" or "<tt>nightly.0</tt>". The higher the number, the older the snapshot is. <tt>hourly.0</tt> was taken at the last even hour, <tt>hourly.1</tt> was from two hours before that, and so on. The nightly snapshots are taken at midnight, so <tt>nightly.0</tt> is the state of your files at midnight this morning, and <tt>nightly.1</tt> is midnight the day before. | ||
| + | |||
| + | {| class="wikitable" | ||
| + | |+ Snapshot Schedule | ||
| + | |- | ||
| + | ! Name | ||
| + | ! Frequency | ||
| + | ! Number | ||
| + | |- | ||
| + | | Hourly | ||
| + | | Every 2 hours (even) | ||
| + | ! 12 | ||
| + | |- | ||
| + | | Nightly | ||
| + | | Every night at midnight | ||
| + | ! 7 | ||
| + | |- | ||
| + | | Weekly | ||
| + | | Every Monday at midnight | ||
| + | ! 4 | ||
| + | |} | ||
==== How to recover via snapshots ==== | ==== How to recover via snapshots ==== | ||
To recover a snapshot, just copy the file from the snapshot directory to anywhere in your home directory. Note that you won't be able to delete, rename, or otherwise modify anything under .snapshot as it's read only. | To recover a snapshot, just copy the file from the snapshot directory to anywhere in your home directory. Note that you won't be able to delete, rename, or otherwise modify anything under .snapshot as it's read only. | ||
| - | |||
| - | |||
== Off-site backups == | == Off-site backups == | ||
| - | + | While the file server is backed up, those backups are for disaster recovery only and not file recovery. If you need a file restored, you must get it from the available snapshots - if the file is not there, it is not recoverable. OIT's backups are performed every two weeks and are rotated three times, thus making roughly 6 weeks of data available should there be a catastrophic failure of the file system. | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
Current revision
Data stored on hard drives has become very precious to many. In some cases, the ones and zeros written to disk can mean weeks, months or years worth of work to their author, and in other cases (such as data from a telescope run) they can never be recovered. For this reason, we not only have some redundant systems in place, but also some backup systems to try to make sure important data is never lost.
Contents |
What is backed up?
Unfortunately, backing up data (or keeping it on highly reliable media) can be expensive. Because of that, we cannot afford to backup everything. However, everything that is stored in /u is backed up, as well as kept on a very reliable NFS server maintained by OIT. We also backup everything from the email server.
We do not backup scratch disks! The very nature of a scratch disk is that it's used for temporary storage of data and intermediate files; perhaps a local copy of data you have on CD elsewhere, or some outputs for in-between states of a data pipeline. Read Scratch disks for more information.
On-site backups
Snapshots
The OIT-maintained file server (which stores your home directory in /u) has a feature available called snapshotting. Every two hours a "snapshot" of the current state of your files is taken and frozen. This is useful if you accidentally delete a file or discover that a file is corrupt, as it means you can "roll back the clock" so to speak and recover an earlier version of your file.
Snapshots for your directory are located in the ~/.snapshot directory (you won't see it in a directory list, but it *is* there if you 'cd' to it). Under that directory you'll see other directories with names like "hourly.0" or "nightly.0". The higher the number, the older the snapshot is. hourly.0 was taken at the last even hour, hourly.1 was from two hours before that, and so on. The nightly snapshots are taken at midnight, so nightly.0 is the state of your files at midnight this morning, and nightly.1 is midnight the day before.
| Name | Frequency | Number |
|---|---|---|
| Hourly | Every 2 hours (even) | 12 |
| Nightly | Every night at midnight | 7 |
| Weekly | Every Monday at midnight | 4 |
How to recover via snapshots
To recover a snapshot, just copy the file from the snapshot directory to anywhere in your home directory. Note that you won't be able to delete, rename, or otherwise modify anything under .snapshot as it's read only.
Off-site backups
While the file server is backed up, those backups are for disaster recovery only and not file recovery. If you need a file restored, you must get it from the available snapshots - if the file is not there, it is not recoverable. OIT's backups are performed every two weeks and are rotated three times, thus making roughly 6 weeks of data available should there be a catastrophic failure of the file system.
Personal backups
If you have a laptop and want to back it up (you should, unless you wouldn't care at all if all the data disappeared suddenly - and since laptops travel around, they're more prone to breakage and drive failure than most desktops), there's a couple options.
Personal TSM backups
Pros:
- Backups are handled by the software automatically (with few exceptions)
- Off-site storage decreases chance of laptop and backup both falling victim to a disaster
- You don't have to manage the backup storage, TSM handles it for you
Cons:
- Your laptop must be plugged in to the network to backup (wireless will work in a pinch, but be painfully slow)
- If you need to restore a file, you must be plugged in to the campus network
- If you write data while on a trip, and aren't back in the building before a failure, you won't have a backup
See the TSM link above for information on registering your laptop for the TSM backup services.
External storage device
Another solution for backing up a laptop is an external storage device, such as a USB or Firewire hard drive enclosure.
Pros:
- Your laptop can be backed up any time you like, as long as you have the drive with you
- No recurring fee for backups, and many enclosures can have their drives replaced for future expansion
- Restores can happen anytime and anywhere, as long as your backup drive is nearby
Cons:
- You are responsible for storing the backups
- If your backup drive and laptop are both in the office during a disaster, you could lose both
