Old Policy
Space | Size | Backup | Lifetime | Comment |
|---|---|---|---|---|
xtc | Unlimited | Tape archive | 1 year | Raw data |
hdf5 | Unlimited | Tape archive | 1 year | Data translated to HDF5 |
scratch | Unlimited | None | 1 year | Analysis results and temporary data |
User home | Unlimited | Disk + tape | Indefinite | User code |
Tape archive | Unlimited | Two copies | 10 years | Raw data |
Rationale for Switching to New Policy
The goal of the new policy based on three different storage classes was twofold:
- Allow users to have easy access (ie on disk) to the most frequently used data for a longer period of time
- Make better use of the LCLS storage resources
We do believe this is a better policy, above all for the users. We noticed that the previous 1-year policy was not enough. At the same time keeping everything on disk for 2-years is not the best use of LCLS resources. Hence LCLS decided to extend the lifetime on disk to 2-years for the most used data files. This was done by introducing quotas and by letting the users select which files should stay on disk.
We do understand the new policy requires a more active participation by the users so we provided tools to help managing the data. For example, we made it much easier to restore data from tape and move data across storage classes.
Also, the scratch directories were sometimes (ab)used unfairly. So we decided to create three scratch areas (ftc, res and scratch) with different characteristics.
Overview
Content Tools