Disk space on IU's research systems

On this page:

General information

Home directories on the research systems

Following is the naming convention for home directory paths on the UITS research systems (replace username with your Network ID username):

System	Path to home directory
Big Red	`/N/u/username/BigRed`
Mason	`/N/u/username/Mason`
Quarry	`/N/u/username/Quarry`
Research Database Complex (RDC)	`/N/u/username/RDC`

When you log into any of these systems, you can access your home directory from any of the other systems by changing to the parent directory (/N/u/username), and then moving down into the desired system home directory, for example:

[davader@bh2 Quarry]$ pwd /N/u/davader/Quarry [davader@bh2 Quarry]$ cd .. [davader@bh2 davader]$ ls Quarry RDC [davader@bh2 davader]$ cd RDC [davader@bh2 RDC]$

User home directories reside on a NAS storage device with disk quotas of 10 GB per user. This quota is shared by your Big Red, Quarry, and RDC accounts, if you have accounts on those systems. If you need additional disk space, email the High Performance Systems group.

Note: If you need more than 10 GB of permanent storage, consider applying for an account on the Scholarly Data Archive (SDA). See Applying for your SDA or RFS account

Checking your quota usage

To determine your current disk quota usage, enter quota. You will see output similar to the following:

davader@Quarry: ~> quota Disk quotas for user username (uid 123456): Filesystem blocks quota limit grace files quota limit grace iupui-nas2:/vol/hd03 3468288 10137600 10485760 21216 4294967295 4294967295

"Filesystem" is the name of the file system where your home directory resides, and "blocks" is the number of 1 KB blocks you are currently using. In this example, over 3.4 GB have been used out of a quota of 10 GB. The limit signifies the absolute hard limit of disk space you can use during the default grace period (seven days). If you exceed your quota and do not reduce your disk space usage to within your quota, after the grace period you won't be able to use more than the quota. If you reach your quota while a job is running, the job will continue to run, but no output files will be appended or created.

Scratch space

Scratch disk space is available locally on each node in /tmp and on an additional large scratch file system shared by all nodes within the system.

Scratch file systems are not backed up to tape; data purged from the scratch file systems cannot be recovered. No disk quotas are enforced on the scratch file systems. You are responsible for making your own copies of the data in case of a disk failure or the need to free space.

Local scratch space: /tmp is a relatively small local file system; many programs use /tmp by default for temporary work space. You can redirect files from /tmp to other scratch disk space by setting the environment variable TMPDIR. Files in /tmp are automatically deleted once they are 24 hours old.
If your batch jobs encounter an error when trying to write to or read from the scratch file systems, a subdirectory with your username probably does not exist. Before reading from or writing to scratch file systems, check that your scratch subdirectory still exists, and create the subdirectory if necessary.
Common scratch space: Common scratch disk space refers to disk space that is available to all users and accessible to multiple systems. It is intended for short-term storage (e.g., working space for running programs and temporary storage for program output), not online archival space. Files in common scratch space are purged automatically based on age to free up space, and are not backed up. For details, see the information below for specific systems.

Big Red

Home directory: Your Big Red home directory disk space is allocated on a NAS storage device. You have a 10 GB disk quota, which is shared with Quarry, Mason, and the RDC (if you have accounts on those systems).
Local scratch: Scratch disk space is available locally on each node in /scratch (67 GB). Files in /scratch are automatically deleted once they are 14 days old.
Shared scratch: The path to your scratch space is /N/dc/scratch/username (replace username with your Network ID username).
Shared scratch space is hosted on the Data Capacitor. The Data Capacitor scratch directory is a temporary workspace. Scratch space is not allocated, and its total capacity fluctuates based on project space requirements. The Data Capacitor is mounted on IU research systems as /N/dc/..., and behaves like any other disk device. If you have an account on an IU research system, you can access /N/dc/scratch. Access to /N/dc/projects requires an allocation. For details, see Data Capacitor. Files in shared scratch space more than 60 days old are periodically purged, following user notification.

Note: Indiana University will soon replace its current Data Capacitor with Data Capacitor II, a high-speed, high-capacity storage facility for very large data sets. With 5 PB of storage, Data Capacitor II will support big data applications used in computational research. IU partnered with DataDirect Networks, Inc. (DDN) to develop Data Capacitor II, which is scheduled to be installed in the IU Data Center in spring 2013. For more about Data Capacitor II, see the November 8, 2012, press release. If you have questions about how the change to Data Capacitor II will affect your research, email the High Performance File Systems group.

Note: Big Red is scheduled to be retired from service in June 2013. Indiana University is replacing it with Big Red II, the fastest university-owned supercomputer in the nation, capable of performing one quadrillion floating-point operations per second (1 petaflop). Based on Cray XE/XK technology, Big Red II has 676 XK nodes (each containing one AMD "Interlagos" processor and one NVIDIA "Kepler" GPU) and 344 XE nodes (each containing two AMD "Abu Dhabi" processors). For more, see Big Red II.

Mason

Home directory: Your Mason home directory disk space is allocated on a NAS storage device. You have a 10 GB disk quota, which is shared with Big Red, Quarry, and the RDC (if you have accounts on those systems).
Local scratch: Scratch disk space is available locally on each node in /scratch (450 GB). Files in /scratch are automatically deleted once they are 14 days old.
Shared scratch: The path to your scratch space is /N/dc/scratch/username (replace username with your Network ID username).
Shared scratch space is hosted on the Data Capacitor. The Data Capacitor scratch directory is a temporary workspace. Scratch space is not allocated, and its total capacity fluctuates based on project space requirements. The Data Capacitor is mounted on IU research systems as /N/dc/..., and behaves like any other disk device. If you have an account on an IU research system, you can access /N/dc/scratch. Access to /N/dc/projects requires an allocation. For details, see Data Capacitor. Files in shared scratch space more than 60 days old are periodically purged, following user notification.

Note: Indiana University will soon replace its current Data Capacitor with Data Capacitor II, a high-speed, high-capacity storage facility for very large data sets. With 5 PB of storage, Data Capacitor II will support big data applications used in computational research. IU partnered with DataDirect Networks, Inc. (DDN) to develop Data Capacitor II, which is scheduled to be installed in the IU Data Center in spring 2013. For more about Data Capacitor II, see the November 8, 2012, press release. If you have questions about how the change to Data Capacitor II will affect your research, email the High Performance File Systems group.

Quarry

Home directory: Your Quarry home directory disk space is allocated on a NAS storage device. You have a 10 GB disk quota, which is shared with Big Red, Mason, and the RDC (if you have accounts on those systems).
Local scratch: Scratch disk space is available locally on each node in /scratch (19 GB). Files in /scratch are automatically deleted once they are 14 days old.
Shared scratch: The path to your scratch space is /N/dc/scratch/username (replace username with your username).
Shared scratch space is hosted on the Data Capacitor. The Data Capacitor scratch directory is a temporary workspace. Scratch space is not allocated, and its total capacity fluctuates based on project space requirements. The Data Capacitor is mounted on IU research systems as /N/dc/..., and behaves like any other disk device. If you have an account on an IU research system, you can access /N/dc/scratch. Access to /N/dc/projects requires an allocation. For details, see Data Capacitor. Files in shared scratch space more than 60 days old are periodically purged, following user notification.

Note: Indiana University will soon replace its current Data Capacitor with Data Capacitor II, a high-speed, high-capacity storage facility for very large data sets. With 5 PB of storage, Data Capacitor II will support big data applications used in computational research. IU partnered with DataDirect Networks, Inc. (DDN) to develop Data Capacitor II, which is scheduled to be installed in the IU Data Center in spring 2013. For more about Data Capacitor II, see the November 8, 2012, press release. If you have questions about how the change to Data Capacitor II will affect your research, email the High Performance File Systems group.

Note: Following a system-wide upgrade in December 2012, Quarry now runs Red Hat Enterprise Linux version 6 (RHEL 6) and uses the Modules package (instead of SoftEnv) for manipulating user environments. For more, see Information about the 2012 upgrade to Quarry at IU. If you encounter any problems or have questions, email the High Performance Systems group.

Research Database Complex

Home directory: Your RDC home directory disk space is allocated on a NAS storage device. You have a 10 GB disk quota, which is shared with Big Red, Quarry, and Mason (if you have accounts on those systems).
Local scratch: Scratch disk space is available locally on each node in /tmp (1 GB) and an additional large scratch file system named /scr (10 GB). Files in /tmp are automatically deleted once they are 24 hours old. Files in /scr are deleted when they are 30 days old. No disk quotas are enforced on /tmp or /scr.
Shared scratch: The path to your scratch space is /N/dc/scratch/username (replace username with your username).
Shared scratch space is hosted on the Data Capacitor. The Data Capacitor scratch directory is a temporary workspace. Scratch space is not allocated, and its total capacity fluctuates based on project space requirements. The Data Capacitor is mounted on IU research systems as /N/dc/..., and behaves like any other disk device. If you have an account on an IU research system, you can access /N/dc/scratch. Access to /N/dc/projects requires an allocation. For details, see Data Capacitor. Files in shared scratch space more than 60 days old are periodically purged, following user notification.

Note: Indiana University will soon replace its current Data Capacitor with Data Capacitor II, a high-speed, high-capacity storage facility for very large data sets. With 5 PB of storage, Data Capacitor II will support big data applications used in computational research. IU partnered with DataDirect Networks, Inc. (DDN) to develop Data Capacitor II, which is scheduled to be installed in the IU Data Center in spring 2013. For more about Data Capacitor II, see the November 8, 2012, press release. If you have questions about how the change to Data Capacitor II will affect your research, email the High Performance File Systems group.

Mail spool disk space usage

UITS does not provide a production mail service on any of the research systems. However, TORQUE (also called PBS) and LoadLeveler communicate via email. Mail forwarding should be configured during account creation, but if you are receiving mail on any of the research systems, you may need to set up a .forward file for your account. For help, see How do I forward my mail from a Unix account?

If you do leave email on the research systems, it may be purged. Twice a month /var/spool/mail is checked for mail files larger than 100 KB. Files larger than 100 KB are moved to /scr/mail. You may have up to two old mail files in /scr/mail. After a mail file is 30 days old, it will be purged from /scr/mail.

This is document avkm in domain all.
Last modified on January 03, 2013.

KnowledgeBase