...
Our current disk usage patterns are not viable going into the future. We have not budgeted enough money to buy disk at the rate we are using it. We (at least I) have blithely postulated in the past that we would find a magic bullet cut to "permanently" removed remove a large fraction of the downlinked data - which is ~>95% background. So far we have not done so, and there are no good ideas on how to yet.
Objective
I think our goal should be to have space for at least 2-3 copies of the data at any given time, and sufficient for a reasonable amount of MC.
...
Early returns from survey running are that we consume 5 TB/week in L1. A snapshot of the L1 data so far shows:
Output File | % Total |
|---|---|
Cal | 14 |
Digi | 9 |
FastMon | 4 |
merit | 3 |
Recon | 60 |
SVAC | 10 |
Recon ignoring a bunch of miscellaneous files that probably eat up a few percent of the total. Recon is the biggest elephant in this room, but not by an order of magnitude. The main uses of recon are: TKR alignment and ToT calibration; event displays; and reprocessing.
From our most recent purchases, disk runs at 0.9k$/TB and tape at 0.4k$/TB.
...
cost is $100-200k per year - ie $100k per copy of the data (or full reprocessing)
(Aug 18, 2008) We may want to think carefully before moving fastmon to tape, in case these files are useful for long term trending.
Delete recon
Sacrifice ability to reprocess quickly from recon. Changes reprocessing time from small number of weeks (1-2) up to a small number of months
...