Computing Newsletter
Disk Space
We continue to add about 5TB of new data a week bringing the total size of LAT data at SLAC to xxxTB (xxx L1 output, xxx MC and xxx reprocessed data). 4 new file yyyTB servers arrived at SLAC on Monday and hopefully by the time you are reading this the first one will have been installed averting the need to store older data only on tape. Although the LAT data is all kept on raid arrays with multiple redundant drives we have also acquired an addition 250TB of tapes so that we can continue to keep all data backed up in case of unanticipated disk problems.
| There are no images attached to this page. |
New user disk space (too successful?)
The new user disk space, /afs/slac.stanford.edu/g/glast/users, is gaining users and usage. Currently, 3 TB of disk space is allocated (of which about 1/2 is actually used) amongst 167 users. In addition, there are nine science groups with allocations totaling 345 GB. From this perspective, the new user space is a success.
However, a usage pattern for user space has emerged that is stressing the server. Submitting 100s of simultaneous batch jobs can cause the server to become non-responsive which, in turn, causes batch jobs to stall and eventually fail. In addition, interactive users attempting to access this space will be unsuccessful. The SLAC Computing Division has been alerted of this issue with the hope that a solution can be worked out. In the meantime, please be aware of the possibility that one can overload that server and affect other users. Batch jobs should be limited to prevent such overloading conditions. This can be done by dribbling in batch jobs a few at a time while [monitoring the server|http://ganglia01.slac.stanford.edu:8080/ganglia/glast/?m=load_one&r=hour&s=descending&c=nfs-glast&h=sulky55.slac.stanford.edu&sh=1&hc=4
]. When the CPU utilization exceed ~50%, you are entering the danger zone.
Pass 7.2 reprocessing
The first 14 months of survey data was reprocessed in October and November with the new Pass 7.2 event classification. The data sample extends from run 239557414 (2008-08-04 15:43:34 UT) through run 277596392 (2009-10-18 22:06:32 UT), spanning 6581 runs and including over 14 billion events. The C&A group is currently evaluating this reprocessed data and, depending on their findings, there may be another reprocessing cycle early next year.
Computer Security Training (SLAC)
An email went out earlier this month to all Fermi LAT collaborators who had not yet completed the required computer security training. We were informed by the SLAC Cyber Security Team that beginning in January 2010, all non-SLAC employees who had not completed this training would have their SLAC computer accounts disabled. (The deadline for SLAC employees was July 2009.) These accounts are used for interactive logins (Linux), email access, and access to a variety of other web-based services. Don't get stuck!
It is already the case (since Oct 2009) that users who need their passwords reset by an administrator must have first completed this training.
For more information on the course, or if you have questions, contact Marilyn Cariola in SLAC Computer Security at 650-926-2820 (email mcariola@slac.stanford.edu).
LAT Workbook developments
The LAT Workbook continues to evolve, expand and be updated. Some highlights since the last newsletter include: LAT GRBanalysis (new); User and group disk space; Using the SLAC batch farm; pylikelihood analysis (updated); Science Tools environment setup (update, including new SCons section); new astroserver examples; ASP Data Viewer help (updated). A full chronicle of the updates can be viewed here, http://glast-ground.slac.stanford.edu/workbook/pages/changelog/changeLog.htm, or, better yet, just browse through the Workbook - http://glast-ground.slac.stanford.edu/workbook/