Science Tools Working Group
Have not met for 2 weeks; most likely we will not meet again until August 9. Science tools development news from the GSSC is scant this week. The network security problems were disruptive, and currently the GSSC cannot write to the CVS repository at SLAC, at least not without workarounds.
The current version of the Science Tools is v7r4, incremented last week at the end of build cycle 19. Here is what the release manager reports as the changes in release notes since the previous version, and the ScienceTools release.notes summarize the changes. Jim reports that a further update to v7r4p1 is likely within the week to take into account further updates to the definition of FT1 - see below.
Data products: Last week, I was hasty in predicting that cleaning up the format of FT1 (LS-002) was complete. Several other tweaks and one fairly major change have been pointed out as missing from FT1. These are now being tracked in JIRA. The most important change has to do with adjusting MJDREF very slightly to account for its formally being in TT but the origin of MET (January 1, 2001, midnight) being in UTC. The shift is so small that MJDREF has to be broken into 2 keywords, MJDREFI and MJDREFF for the 'integer' and 'floating point' portions. Having makeFT1 do this is straightforward, but it will break gtpsearch until it knows to look for the 2-part MJDREF.
Databases and related utilities
No news regarding the GSSC server.
I had one exchange with Jean-Paul regarding the missing START-STOP pair in the Astro Server GTIs. I think that the missing interval (for the case where the time of the first event is after the start of the first data file - which is what we'll always have) will come back. No news to report regarding command-line interface(s) for the Astro Data Server.
Likelihood analysis
Jim reports no development news. He forwarded a bug report from Jean Ballet for gtsrcmaps; tip misidentifies an image extension as not being one.
From James: "James and Analia completed a likelihood bad-fit warning. The approach suggested by Dave Thompson, in which the Poisson probability for observing N photons given that M were actually present, turned out not to be feasible for computational reasons. The formula includes raising N to the Mth power, which for even moderate sizes of M and N quickly exceeds the range of a double precision value. Instead they implemented a modified version of the scheme, in which the fractional deviation is computed and ranges are reported in which the deviation exceeds some threshold. This is clearly a work in progress because the threshold computation does not involve the predicted error, and in any case the value for the threshold needs to be tuned to real-life data. This code so far was not delivered due to the lack of repository access."
GRB tools
No news, and I'm afraid no update on the IRF normalization/gtrspgen issue reported by James last week.
Pulsar tools
From Masa: "Nothing special in the pulsar tools area; we've been developing an improved infrastructure of the time-handling classes."
Observation simulation
No news
User interface and infrastructure
From Jim: "I have started using Navid's pattern in STpolicy v1r2 for creating a central pfiles directory for all of the gt tools. This is in ST HEAD and will be a part of ST v7r4p1." This is the functionality that users of the DC2 distributions enjoyed. It also prevents needing enormously (or impossibly) long PFILES environment variables.
Source Catalog
The catalog group did not meet this week.
Science Tools Working Group
We met last week; most likely we will not meet again until August 9.
The current version of the Science Tools remains v7r3, although an increment in honor of the end of the build cycle tomorrow is in the works.
Data products: The proposed changes (cleaning up) of the FITS header for FT1 (LS-002) have been implemented in makeFT1 by Jim; among other changes this involved making makeFT1 run as an FTOOL, which gave it access to information about its own package version number, which gets written to the CREATOR keyword. I think that this is now the definition of LS-002 that we'll be going with for GRT 5.
Databases and related utilities
No news regarding the GSSC server. It has been inaccessible recently, most likely owing to the general server outage.
Jean-Paul visited SLAC last week for the conference on Web engineering and had time to meet to discuss the Astro Data Server. Jean-Paul is confident that the MySQL implementation that underlies the Astro Data Server can handle (ingest and serve) much larger data sets than DC2's, and some possibilities like generating output FITS files in parallel on a few systems have not been explored. We'll certainly extensively exercise the Astro Data Server in the coming months. Most likely Tony will implement the command-line interface to the server; after discussion with Jim and Richard this week, I see this has some interesting aspects (depending what you like) regarding whether the queries are synchronous and how they can be managed within Pipeline 2. The persistent problem with the GTIs required reingesting the DC2 FT1 files is 99.8% solved - meaning that 520 out of 521 time intervals are now returned for DC2 queries. The missing interval is the first and its absence is being looked into.
Likelihood analysis
From Jim: "I fixed a bug that occurred when trying to analyze data with zero events; and I rewrote the Amoeba class, which is used by gtfindsrc to point source position fitting, to get around a linking problem on Windows."
GRB tools
James sent this following clarification and update about the issue with Xspec analysis of GRBs that was reported last week. I have not followed up yet with questions about the response functions.
James and Dave Davis continued exploring in detail the behavior of rspgen. As a basic sanity check, Dave simulated an extremely bright power law source (to boost the number of counts) observed in a pointed observation. With the response matrix computed by rspgen, the spectum of this observation was analyzed in Xspec. When using the full range of energy selected, Xspec consistently got the wrong answer (too low) for the exponent in the power law of the source. Further experimentation revealed that the fit converges to the correct answer if one uses only the central energy range of the spectrum.
One problem they uncovered and subsequently solved was that the combined dispersion for DC2 (which includes 4 individual responses, FRONTA/BACKA/FRONTB/BACKB) was not being renormalized to 1 by rspgen. When this was fixed the behavior of rspgen improved somewhat, in that Xspec's answer was closer to the right answer, but still not statistically correct. However, at higher true energies, slices of the dispersion matrix coming from the irfs packages do not seem to be normalized to 1 to begin with, resulting in some artifacts.
To probe further, James and Dave examined the separate components of the DC2 response. They found that the effective area and dispersion seem believable, but the psf has a very odd shape, particularly for
the front events. It's still unclear what about the data set and/or the response matrix is causing Xspec to fail to converge to the right answer.
Pulsar tools
From Masa: "We are in the final stage of implementation of time handling classes in the pulsar tools."
Observation simulation
From Jim: "I modified primary HDU header keywords in the the FT1/2 FITS template files to conform to HEASARC conventions and according to our consensus decisions as to what they should be, and I modified makeFT1, gtobssim and gtorbsim to fill the FILENAME, VERSION, and CREATOR keyword with the proper values."
User interface and infrastructure
No news
Source Catalog
The catalog group met yesterday. Jean presented a comparison of the DC2 catalog with the DC2 sources that were detectable; the results are reassuring in most respects but a bit curious in others. We discussed 'test pattern' source distributions for testing source detection algorithms and a 0-th iteration of the agenda for the Catalog session at the collaboration meeting.
Science Tools Working Group
We met yesterday; most likely we will not meet again until August 9. This report summarizes activities since June 22.
The current version of the Science Tools remains v7r3.
Data products: The door is not quite closed to comment on the proposed changes (cleaning up) of the FITS header for FT1. Regarding the keywords that relate to versions of software, one question I had was whether the tools can know their own version numbers.
Databases and related utilities
Tom reports that he is benchmarking various combinations of host, memory, and disks to plan the purchase of the systems that will host GLAST data at the GSSC. I did not think to ask about the scale and time scale. Even during year 1, the GSSC will be archiving L0 data.
Jean-Paul is visiting SLAC this week, and has been doing some work on the Astro Data Server. The persistent problem with the GTIs required reingesting the DC2 FT1 files - meaning loading a database table with the FT1 quantities that the Astro server allows searching on. The loading is fast, requiring about 1 hour for all 3.3 M events. I have not heard yet how long building the index takes, or rebuilding the index after incremental updates. Jean-Paul is willing to write a command-line interface to the Astro server, to allow queries by scripts. We'll meet together with Tony Johnson tomorrow afternoon.
Likelihood analysis
From Jim: "I added a class CountsSpectra to Likeihood to encapsulate the counts spectra information (observed and model counts as a function of energy) used by the plotting option in gtlikelihood and potentially by the Poisson counts "badness-of-fit" test [see below] being implemented by Analia and James."
"I tagged pyLikelihood at v0r3p1 so that the python interface that reads the gtlikelihood par files is now available."
Also Jim has taken advantage of st_graph improvements by James for log-log plots; with the plot option paramter set to 'yes' you get (counts spectral) plots of the model components and residuals. And he has implemented the capability for reading .par files in Python version of likelihood, binned and unbinned; as of 2 weeks ago the readline library was not quite working properly.
At the request of Dave Davis and Rita, output from gtlikelihood now inclues fluxes of model components in addition to counts; these are written both to ASCII and FITS binary files. Right now this is only for point source components - for diffuse components an integral, over the ROI say, and may not be interesting.
Analia and James are looking into providing some measure of whether the model you use in gtlikelihood is actually a good representation of the data. Conceptually this kind of makes sense, but it may be a morass in terms of implementation.
GRB tools
No news, although James reports that he and Dave Davis "began looking in detail at some issues Dave
uncovered when analyzing large numbers of photons using Xspec." I don't know right now what this specifically means, although 'issues' does not typically mean something like a wonderful surprise.
Pulsar tools
Masa and James are continuing to make the pulsar tools use the new TimeRep class. Goal still remains having versions to circulate by the end of the current build cycle (July 21).
Observation simulation
Jim now has gtobssim report numbers of events generated vs. numbers 'accepted' - this is now in the mc_src_id output file. The additional information was requested by Richard to help with understanding why PeriodicSource was behaving differently for gtobssim and Gleam. (First indication was that the DC2 X-ray binaries were factors of several fainter in the DC2 data than their definitions specified.) Toby traced the problem to an implicit assumption in PeriodicSource of 1-m 2 area for the 'target' vs. 6 m 2 for the Gleam runs. Jim has fixed PeriodicSource.
User interface and infrastructure
James delivered ape (the PIL) replacement, i.e., it is part of the FTOOLS system for the coming release. He had previously estimated that integrating ape (via HOOPS, I think) into the science tools would take a few weeks after ape was done).
Source Catalog
The catalog group has not met since June 21.