Confluence will be unusable 23-July-2024 at 06:00 due to a Crowd upgrade.
...
Space | Size | Backup | Lifetime | Storage class | Comment |
---|---|---|---|---|---|
xtc | Unlimited | Tape archive | 1 year | Raw data | |
hdf5 | Unlimited | Tape archive | 1 year | Data translated to HDF5 | |
scratch | Unlimited | None | 1 year | Analysis results and temporary data | |
User home | Unlimited | Disk + tape | Indefinite | User code | |
Tape archive | Unlimited | Two copies | 10 years | Raw data |
6 months | Short-term | Raw data | |||
usr | |||||
Space | Size | Backup | Lifetime | Storage class | Comment |
---|---|---|---|---|---|
xtc | Unlimited | Tape archive | 6 months | Short-term | Raw data from users' DAQ systems |
hdf5 | Unlimited | Tape archive | 6 months | Short-term | Data translated to HDF5 |
scratch | Unlimited | None | 6 3 months | Short-term | Temporary data |
xtc/hdf5 | 10TB | n/a | 2 years | Medium-term | Selected XTC and HDF5 runs |
ftc | 10TB | None | 2 years | Medium-term | Filtered, translated, compressed |
res | 1TB | Disk Tape | 2 years | Medium-term | Analysis results |
User home | 20GB | Disk + tape | Indefinite |
| User code |
Tape archive | Unlimited | Two copies | 10 years | Long-term | Raw data |
Web tools have been created to facilitate users in:
The goal of the new proposed policy based on three different storage classes is was twofold:
The new policy will be accompanied by web tools to facilitate users in:
Why are you doing this to us?
We do believe this is a better policy, above all for the users. We noticed that the previous 1-year policy was not enough. At the same time keeping everything on disk for 2-years is not the best use of LCLS resources. Hence LCLS decided to extend the lifetime on disk to 2-years for the most used data files. This was done by introducing quotas and by letting the users select which files should stay on disk.
We do understand the new policy requires a more active participation by the users so we 'll provide provided tools to help managing the data. For example, we made it much easier to restore data from tape and move data across storage classes. We'll soon provide tools to compress and filter the data.
Also, the scratch directories were sometimes (ab)used unfairly. So we decided to create three scratch areas (ftc, res and scratch) with different characteristics.
Will So will all raw files be deleted by the proposed deadlinesafter 6 months?
...
...
By which date do we need to select runs with the file manager to increase the lifetime on disk of our data?
...
...
The total size of our experiment is below the quota, may we select all runs for the 2-years storage?
What will happen happens to the data under the scratch folder?Starting November 1st 2012, all
...
Can we restore raw files from tape to disk by ourselves?
Can we restore the contents of the ftc, res and scratch folders from tape?
How long does it take to restore files from tape to disk?
What is the lifetime on disk of files restored from tape?
How long do the raw data files stay on tape?
What should we store in the ftc experiment directory?
...
What should we store in the res experiment directory?
What should we store in the scratch experiment directory?
...
What goes in the usr experiment directory?
Can we ask for an increase of the 2-years quota for a particular experiment?
Can we ask for an extension of the deadline for the run selection?