Event Summary Data (FT1, LS-002)
This is a fundamental data product used by the science tools. The current definition (version of July 12, 2005) is posted on the Guidelines for Science Tools Design page maintained by Masa Hirayama.
The issues and proposed changes below primarily are with respect to the 'LAT event summary' extension in the version linked above. You may want to have that page open when you read this page. The initial concept for the contents dates at least to 2001, when we were not burdened with a detailed understanding of what would be available from reconstruction and classification and how the tools would work, and so should be updated in several respects.
The issues and proposed resolutions below, or any other aspect of the definition of the contents are open for comment (probably most effectively by editing this page rather than inserting Confluence comments).
1. Duplication of keywords
(Digel) TELESCOP, INSTRUME, EQUINOX, RADECSYS, DATE, DATE-OBS, DATE-END are duplicated from the primary header of the file. Is this necessary, or even a good idea? I propose putting TELESCOP, INSTRUME, DATE, DATE-OBS, and DATE-END in the primary header only, and EQUINOX and RADECSYS in the Lat event summary header.
2. HDUCLASS keywords
(Digel) I am not sure whether we adhere to the strict definition of a HDUCLASS1 = EVENTS file. The only column that we have in common with the specification of The Recommended Columns and Keywords for a FITS Event List is TIME.
Also, I think that with the HDUCLASS keywords we are supposed to supply a format-specifying document with the HDUDOC keyword, and a version in HDUVERS. We don't have such a document, and we already have (but do not yet use) a VERSION keyword in the primary header.
For these reasons, I'd recommend removing HDUCLASS keywords entirely.
3. TASSIGN keyword
(Digel) TASSIGN is supposed to be where the event times were assigned, and we currently have this set equal to 'SATELLITE'. This is not strictly true, as we will need some ground processing to turn ticks of the 20 MHz clock into time since the last 1 PPS signal from the spacecraft, and we'll also need to shift GPS time to TT.
Also, I don't see what good the keyword does us. So I recommend removing it.
4. OBS_ID and OBJECT keywords
(Digel) I don't think we need either of these. What OBS_ID was originally intended to represent is not clear, and a specific OBJECT is typically not relevant for the LAT.
5. MC_TRUTH keyword
(Digel) I think that the original intent was that this keyword indicate whether the data are entirely Monte Carlo truth values (i.e., actual directions, energies, etc.). We do not actually have files like this, and I propose removing this keyword. gtobsssim does add a column MC_SRC_ID to the files it generates, but this is easy enough to check for without the MC_TRUTH keyword.
6. EVENT_ID column
(Digel) This is specified as a 32-bit integer. Right now, most likely the event IDs will be assigned by the LAT as a time stamp. Most likely we will need more than 32 bits to represent them, and of course they should look more or less like some kind of integer representation of the value in TIME. Right now, I don't know what to recommend for EVENT_ID.
7. RECON_VERSION column
(Digel) RECON_VERSION is defined as a 2-byte integer to define the version of reconstruction (and classification) algorithm applied for a given event. This is probably more general than we need. In principle, a given events file could have events processsed with different versions of the software, but in practice, I do not expect that this will happen. I recommend that we make RECON_VERSION a header variable instead of a column.
Also, somebody needs to think of how we will translate versions of the geometry, calibration files, reconstruction, and classification, into some fairly compact representation. Certainly we will be closely keeping track of this information for Level 1 processing, and will have a database someplace that can tell us.
8. CALIB_VERSION column
(Digel) As for RECON_VERSION we do not need this as a column. Actually, I'd recommend combining it with RECON_VERSION.
9. IMGOODCALPROP, IMVERTEXPROB, IMCOREPROB, IMPSFERRPRED, CALENERGYSUM, CALTOTRLN, and IMGAMMAPROB columns
(Digel) These columns relate to the classifications of the events and date to DC1, in particular to implementing the Atwood cuts for background rejection. Many things came together at once in the runup to DC1 and we ended up incorporating each of these variables in the event summary file. The DC1 response functions were derived post-Atwood cuts.
For DC2, we will have a cleaner way to specify the results of the event classification, although I do not know what it is yet. We will have some kind of analog of IMGAMMAPROB. Ideally, we'll also have distilled other classifications into a few sets that map into response functions that we'll provide (e.g., front, back, and cal-only events, with good_pdf, good_energy, or dont_care).
What do you recommend here?
Speaking of classification, will we also have a flag like 'HEAVY_CR'?
10. CONVERSION_POINT
(Digel) This was originally imagined as a way for end users to decide whether they wanted to believe whether a particular event was not a charged particle and was well reconstructed. Eventually, we will have an event display server available that will make this much easier for users, who would otherwise have to find the geometry of the LAT some place.
Also, so far we have not invented a need for filtering the data on CONVERSION_POINT. In principle, we may some day have a solar flare mode, for which we will pay attention only to the inner towers and layers, but this is probably much better implemented as a kind of flag (like "INTERIOR").
I recommend removing CONVERSION_POINT.
11. PULSE_PHASE and ORBITAL_PHASE
(Digel) These are needed (and filled) by the pulsar timing analysis tools. I don't know whether they belong in the definition per se of the event file, because they can be added by the pulsar tools, but I don't feel particularly strongly if they stay. I do recommend that we make them floating point values instead of their current doubles.
12. SKYX/Y issue
(Digel) The Guidelines for Science Tools Design page lists one outstanding issue for the definition of the events file, whether to add columns that give the coordinates of the events in some coordinate projection.
The justification given for this is that with projected values of the coordinates (presumably with respect to a sensibly chosen center), tools like ds9 can interpret and bin the events into maps, and then (possibly complicated) regions can be defined for generation of response matrices. This is an analysis path (multiple overlapping point sources in Xspec) that we are not intending to pursue.
Jim has pointed out that having ds9 able to make a binned map directly with real coordinates on it (as opposed to just sky pixels) is a big convenience. Otherwise, a map first has to be generated with gtbin.
I don't know what to recommend. Having coordinates available in some (which?) coordinate projection would be convenient, as would, say, having them available in Galactic coordinates, but I am not sure where it is sensible to draw the line.
13. Live time
(Digel) Way back when (until early 2003), the events file was defined to include a column called 'Deadtime' which was to be the 'Deadtime accumulated since the start of the mission.' It was removed, partially because Masa pointed out that information that relates specifically to the LAT belong in the FT2 file. Also, live time turns out to be more convenient to work with.
I propose that we include accumulated livetime since the start of the mission as a column. It was always imagined that for very short time intervals (shorter than the update interval of the FT2 file) we would need to calculate accumulated livetimes between events in the event summary file. This also will be important for studies of solar flares and (if we are lucky) very bright GRBs, when we may be dead time limited for short periods of time.
14. Event summary++
(Digel) LS-002 is an event summary, intended to have information for higher-level analyses. The GSSC expects (reasonably) to receive this summary for every event that is telemetered to the ground (and that is not discarded early in the process as obviously background to save disk space at the ISOC).
They would also like to receive an 'extended' event summary that includes all of the variables that are used by the classification trees. That sounds fine to me, too. Right now, the specification of variables actually used by the trees has not converged, and sending the whole Analysis Ntuple would be overkill. I think that we should be able to define this data product in good detail within the next month.