Scratch disks
From Peyton Hall Documentation
m |
(Expanded the article) |
||
| Line 1: | Line 1: | ||
{{inuse}} | {{inuse}} | ||
| + | ==What are scratch disks?== | ||
Scratch disks are usually smaller (but sometimes quite large) drives where you can temporarily store data that does not need to be backed up. They may be used for intermediate stages in a data pipeline, a local copy of a dataset available elsewhere or on a non-online media (CD, DLT, DVD), or temporary storage for outputs of your work. Scratch disks are not [[Backups|backed up]] at all, though some of them are larger RAID5 disk systems which are more fault-tolerant than a single disk on a workstation. They are usually [[NFS]] mounted across all machines in the building for easy access. | Scratch disks are usually smaller (but sometimes quite large) drives where you can temporarily store data that does not need to be backed up. They may be used for intermediate stages in a data pipeline, a local copy of a dataset available elsewhere or on a non-online media (CD, DLT, DVD), or temporary storage for outputs of your work. Scratch disks are not [[Backups|backed up]] at all, though some of them are larger RAID5 disk systems which are more fault-tolerant than a single disk on a workstation. They are usually [[NFS]] mounted across all machines in the building for easy access. | ||
| + | ==How do I use access scratch disks?== | ||
| + | Scratch disks are exported via nfs and are available on any department-support Linux desktop or server via automounts. You can access existing scratch space via the following directory structure | ||
| + | /peyton/scr/hostname# | ||
| + | where <tt>hostname</tt> is the name of the host the space is exported from (minus the astro.princeton.edu) and <tt>#</tt> is the partition number so for example | ||
| + | cd /peyton/scr/depot0 | ||
| + | would cd into the first (counting from 0) scratch disk on the server depot. | ||
| + | |||
| + | If you're using space that's directly attached to your local machine, you can also access it directly via <tt>/scr#</tt> to avoid a bit of NFS overhead. | ||
| + | ==What scratch space is available?== | ||
| + | ===depot=== | ||
| + | We have two 1.1TB arrays available at <tt>/peyton/scr/depot0</tt> and <tt>/peyton/scr/depot1</tt> available to everyone in the department from any departmental Linux computer. Please ''read'' the file <tt>/peyton/scr/depot0/README_FIRST</tt> for more information on how to use this space. | ||
| + | |||
| + | |||
| + | ===Private Servers=== | ||
| + | There are a number of projects in the department with scratch disks which were purchased for the use of those projects. If you're working on a specific project (such as SDSS or ACT just to name a couple) and need a place to store files related to that project, contact the project's coordinator and ask them to contact us to request access to their space for your account. | ||
| + | |||
| + | |||
| + | ===Desktop scratch space=== | ||
| + | A number of our desktops have a lot more space on their local hard drives than is used by the OS install and can be allocated as scratch space. If you plan to do a lot of I/O on larger (few GB+) datasets, using your local scratch space might be a good way to speed up the process as you are getting rid of network overhead. It's also a good way to get temporary data off the main (charged) NFS server. | ||
| + | |||
| + | We suggest you think carefully about what files you want to use on local scratch space, however. This space, like all scratch space, is not backed up, and it's on a single hard drive with no redundancy. Do not use it for data that is irreplaceable or would take a long time to regenerate. | ||
| + | |||
| + | If these conditions are OK with you, please e-mail help@astro and request access to your local scratch space. | ||
| + | |||
{{stub}} | {{stub}} | ||
Revision as of 19:51, 22 May 2007
 | This article is actively undergoing a major edit for a short while. As a courtesy, please do not edit this page while this message is displayed. The person who added this notice will be listed in its edit history or has placed their signature above. If this page has not been edited recently (several hours!), please remove this template (or replace it with {{underconstruction}}). This message is intended to help reduce edit conflicts; please remove it between editing sessions to allow others to improve this page. |
Contents |
What are scratch disks?
Scratch disks are usually smaller (but sometimes quite large) drives where you can temporarily store data that does not need to be backed up. They may be used for intermediate stages in a data pipeline, a local copy of a dataset available elsewhere or on a non-online media (CD, DLT, DVD), or temporary storage for outputs of your work. Scratch disks are not backed up at all, though some of them are larger RAID5 disk systems which are more fault-tolerant than a single disk on a workstation. They are usually NFS mounted across all machines in the building for easy access.
How do I use access scratch disks?
Scratch disks are exported via nfs and are available on any department-support Linux desktop or server via automounts. You can access existing scratch space via the following directory structure
/peyton/scr/hostname#
where hostname is the name of the host the space is exported from (minus the astro.princeton.edu) and # is the partition number so for example
cd /peyton/scr/depot0
would cd into the first (counting from 0) scratch disk on the server depot.
If you're using space that's directly attached to your local machine, you can also access it directly via /scr# to avoid a bit of NFS overhead.
What scratch space is available?
depot
We have two 1.1TB arrays available at /peyton/scr/depot0 and /peyton/scr/depot1 available to everyone in the department from any departmental Linux computer. Please read the file /peyton/scr/depot0/README_FIRST for more information on how to use this space.
Private Servers
There are a number of projects in the department with scratch disks which were purchased for the use of those projects. If you're working on a specific project (such as SDSS or ACT just to name a couple) and need a place to store files related to that project, contact the project's coordinator and ask them to contact us to request access to their space for your account.
Desktop scratch space
A number of our desktops have a lot more space on their local hard drives than is used by the OS install and can be allocated as scratch space. If you plan to do a lot of I/O on larger (few GB+) datasets, using your local scratch space might be a good way to speed up the process as you are getting rid of network overhead. It's also a good way to get temporary data off the main (charged) NFS server.
We suggest you think carefully about what files you want to use on local scratch space, however. This space, like all scratch space, is not backed up, and it's on a single hard drive with no redundancy. Do not use it for data that is irreplaceable or would take a long time to regenerate.
If these conditions are OK with you, please e-mail help@astro and request access to your local scratch space.
