...
- 2/20/2014 - Initial set-up of task
- 2/25/2014 - Begin trickleStream on Block 1 – entire P202 run list
- 2/28/2014 - Many problems: LSF seems to bunch up a large number of jobs before dispatching. This causes a shock wave of jobs causing a problem for AFS (reading the ~500 MB diffuse response file), and for the dataCatalog (the queries for FT2 file, and to determine the next file version for each data product). Therefore, reconfigure trickleStream to a mere dribble – no more than 300-400 jobs typically running simultaneously. This means a 1-day task has become a week-long effort.
- 3/3/2014 - bulk of block Block 1 complete. Last week developed new scheme to move all dataCatalog queries from batch jobs into jython scriptlets. These changes will be integrated prior to processing any more data. Validity check: There are the same number of runs and the same number of events in the reprocessed P203 data as in the P202 data.
Configuration
Identical with P202-FITS except:
...