Storage and disk space

All users accouns are given a 1GB home space that is backed up. The home space is hosted at the ACES site with which a user is affiliated. Home space directories are visible from all machines in the ACES system.

Scratch storage

ACES also provides short term scratch storage capacity for running computations with large disk space needs. However, long term backed-up storage is provided by individual groups that participate in ACES. It is administered according to their own policies.

The short term storage capacity in ACES is visible to all systems within ACES. Data stored in short term storage is automatically deleted when disk capacity exceeds 80% on the main "ds-xx" NFS servers and 50% (with a 24-hour grace period) on the compute nodes. When this capacity limit is reached datasets are deleted by age (oldest first) until disk capacity falls below the specified (80% or 50%) utilizaion. There are no space quotas on the short term storage, however, each user account is limited to one million files on each storage device (e.g. /net/ds-01/scratch-0). This file count limit is to prevent performance problems in the Linux ext3 file system format that occur when managing file systems that contain very large numbers of files.

The scratch storage devices available at present are named:

        /net/ds-01/scratch-0    /net/ds-02/scratch-0
        /net/ds-01/scratch-1    /net/ds-02/scratch-1
        /net/ds-01/scratch-2    /net/ds-02/scratch-2
        /net/ds-01/scratch-3    /net/ds-02/scratch-3
        /net/ds-01/scratch-4    /net/ds-02/scratch-4
        /net/ds-01/scratch-5    /net/ds-02/scratch-5
        /net/ds-01/scratch-6    /net/ds-02/scratch-6

        /net/ds-03/scratch-0    /net/ds-04/scratch-0

Additionally, there is local scratch storage available on each of the compute nodes in ACES. This storage is accessed using path names of the form

	/net/HOSTNAME/scratch

Please do not hog space on the "local disks" on the compute nodes. Most of the local disks (e.g. /net/aE34-500-011/scratch) are very fast (many are 15,000 RPM SCSI units) but have small overall capacities. If at all possible, please use them only while your PBS jobs have reserved those particular nodes. And, whenever possible, please cleanup your files and directories on the compute nodes after each run. As with all the scratch disks, files will be deleted when the disks become too full.

The following storage machines are "owned" by specific sub-groups within ACES. These groups have purchased RAID NFS servers for their use and they dictate who can and can't use them. If you (and/or your PIs) would like to arrange for the addition of a dedicated RAID server for your data sets, please contact us on the ACES-support list. The dedicated machines currently are: