Scratch disks
From Peyton Hall Documentation
Scratch disks are usually smaller (but sometimes quite large) drives where you can temporarily store data that does not need to be backed up. They may be used for intermediate stages in a data pipeline, a local copy of a dataset available elsewhere or on a non-online media (CD, DLT, DVD), or temporary storage for outputs of your work. Scratch disks are not backed up at all, though some of them are larger RAID5 disk systems which are more fault-tolerant than a single disk on a workstation. They are usually NFS mounted across all machines in the building for easy access.
Contents |
How do I access scratch disks?
Scratch disks are exported via nfs and are available on any department-supported Linux desktop or server via automounts. You can access existing scratch space via the directory structure "/scr/hostname#" where "hostname" is the name of the host the space is exported from (minus the astro.princeton.edu) and "#" is the partition number. For example, 'cd /scr/depot0' would cd into the first (counting from 0) scratch disk on the server depot. If you're using space that's directly attached to your local machine, you can also access it directly via /scr# to avoid a bit of NFS overhead.
What scratch space is available?
depot
We have ~70TB available at /scr/depot0 available to everyone in the department from any departmental Linux computer. To request a directory, contact us.
Private servers
There are a number of projects in the department with scratch disks which were purchased for the use of those projects. If you're working on a specific project (such as SDSS or ACT just to name a couple) and need a place to store files related to that project, contact the project's coordinator and ask them to contact us to request access to their space for your account.
Desktop scratch space
A number of our desktops have a lot more space on their local hard drives than is used by the OS install and can be allocated as scratch space. If you plan to do a lot of I/O on larger (few GB+) datasets, using your local scratch space might be a good way to speed up the process as you are getting rid of network overhead. It's also a good way to get temporary data off the main (charged) NFS server.
NOTE:
- We suggest you think carefully about what files you want to use on local scratch space, however. This space, like all scratch space, is not backed up, and it's on a single hard drive with no redundancy. Do not use it for data that is irreplaceable or would take a long time to regenerate.
If these conditions are OK with you, please contact us and request access to your local scratch space.
FAQ
I can't run ls in my home directory
I can't login to my account without pressing Ctrl-C to abort something
I can't use the window manager's file browser
- Note that these are all lumped together because they usually have the same answer
Chances are, you have a symlink in there somewhere to a scratch disk which is currently unreachable. You should never put symlinks to scratch disks in your home directory - if for some reason that disk is not reachable, you run a good risk of not being able to access your account, your home directory, or much of anything else until someone deletes it for you. Make shell aliases such as "export FOO=/scr/depot0/username" and use 'cd $FOO' if you must have a shortcut - at least then you'll be able to access your files if that disk goes down.
