Scratch disks
From Peyton Hall Documentation
Scratch disks are usually smaller (but sometimes quite large) drives where you can temporarily store data that does not need to be backed up. They may be used for intermediate stages in a data pipeline, a local copy of a dataset available elsewhere or on a non-online media (CD, DLT, DVD), or temporary storage for outputs of your work. Scratch disks are not backed up at all, though some of them are larger RAID5 disk systems which are more fault-tolerant than a single disk on a workstation. They are usually NFS mounted across all machines in the building for easy access.
Contents |
How do I access scratch disks?
Scratch disks are exported via nfs and are available on any department-supported Linux desktop or server via automounts. You can access existing scratch space via the directory structure "/scr/hostname#" where "hostname" is the name of the host the space is exported from (minus the astro.princeton.edu) and "#" is the partition number. For example, 'cd /scr/depot0' would cd into the first (counting from 0) scratch disk on the server depot. If you're using space that's directly attached to your local machine, you can also access it directly via /scr# to avoid a bit of NFS overhead.
What scratch space is available?
General
We have ~75TB available at /scratch for anyone in the department from any departmental Linux computer. To request a directory, contact us.
Private servers
There are a number of projects in the department with scratch disks which were purchased for the use of those projects. If you're working on a specific project (such as SDSS or ACT just to name a couple) and need a place to store files related to that project, contact the project's coordinator and ask them to contact us to request access to their space for your account.
Desktop scratch space
Desktops used to have a lot of space left over, however many now use most if not all of that space for the OS. With many desktops being all-in-one devices, there generally isn't any room to install additional disks. Combined with the added reliability of RAID systems and increased network bandwidth, it's generally preferable to use space on a purpose-built machine (such as where /scratch is housed) than local scratch space.
NOTE:
- We suggest you think carefully about what files you want to use on local scratch space, however. This space, like all scratch space, is not backed up, and it's on a single hard drive with no redundancy. Do not use it for data that is irreplaceable or would take a long time to regenerate.
FAQ
I can't run ls in my home directory
I can't login to my account without pressing Ctrl-C to abort something
I can't use the window manager's file browser
- Note that these are all lumped together because they usually have the same answer
Chances are, you have a symlink in there somewhere to a scratch disk which is currently unreachable. You should never put symlinks to scratch disks in your home directory - if for some reason that disk is not reachable, you run a good risk of not being able to access your account, your home directory, or much of anything else until someone deletes it for you. Make shell aliases such as "export FOO=/scratch/username" and use 'cd $FOO' if you must have a shortcut - at least then you'll be able to access your files if that disk goes down.
