PPA SCA Data Handling Group
Within the SLAC Particle and Particle Astrophysics Scientific Computing Applications group the Data Handling team is responsible for providing tools to enable scientists to efficiently process and analyze large amounts of data for a variety of experiments at the laboratory. The group works on graphical and web based monitoring, analysis and visualization tools and tools for processing, storing and cataloging High-Energy and Astrophysics data.
Team Members
- Dan Flath
- Charlotte Hee
- Karen Heidenreich
- Tony Johnson
- Max Turri
Experiments
The data handling team is working closely with the following experiments:
- Fermi Gamma-Ray Space Telescope Large Area Telescope (Fermi LAT)
- Enriched Xenon Observatory (EXO)
- Cryogenic Dark Matter Survey (CDMS)
- Detector R&D for future accelerators
- LSST
Projects
Automated processing Pipeline and Data Catalog
The automated processing pipeline and data catalog was developed for the Fermi Large Area Telescope (LAT) and has been in use since before the launch on June 11 2008. The LAT has so far recorded more than 22 billion events, sending more than 11 terabytes of heavily compressed data to Earth for detailed processing at SLAC. These processed data occupy more than 550 terabytes on disk. The pipeline has delivered more than 533 CPU years of processing for Fermi data. On average, it takes only a few hours for data to be collected onboard the LAT, transmitted to SLAC, processed, and sent on to the Fermi Science Support Center where the data are made public.
The automated processing pipeline is now in use by Fermi, EXO and CDMS. The associated data catalog is in use by the same experiments plus the international linear collider.
Web based Data Monitoring and Control
We have developed a reusable web framework for data monitoring and control. It is currently in use by Fermi and EXO.
Java based Visualization and Analysis tools
Wired event display, JAS analysis toolkit and JAIDA histograming, plotting and fitting package.
Open source Java libraries for Scientific Computing 
Other projects/tools
- LCIO - Language/experiment independent IO format.
- Collabative web based tools: Confluence, JIRA, discussion forums (forum.linearcollider.org, forum.freehep.org).
- Subversion and CVS repositories and automated build tools.
Future Technologies
We are actively exploring the use of tools such as Google Web Toolkit (GWT) for interactive web applications and the use of cloud computing for scientific data processing.