...
The issues and proposed resolutions below, or any other aspect of the definition of the contents are open for comment (probably most effectively by editing this page rather than inserting Confluence comments).
Proposed resolutions for the issues below were added 29 November 2005 by S. Digel. Some issues still outstanding were also highlighted.
1. Duplication of keywords
...
(Hirayama) The repetetion of "major" keywords (such as TELESCOP) was suggested by the HEASARC FITS Working Group when we met them at NASA/GSFC on May 28th, 2003. See also section titled "Answers to the questions" under topic 006 of the latest topics of FT1 and FT2 for more details. Because the FT1 definition changed quite a bit since then, it could be useful if some of us can meet the HEASARC FITS Working Group members to discuss on this topic.
Proposed resolution: Continue duplicating the keywords between headers.
2. HDUCLASS keywords
(Digel) I am not sure whether we adhere to the strict definition of a HDUCLASS1 = EVENTS file. The only column that we have in common with the specification of The Recommended Columns and Keywords for a FITS Event List is TIME.
...
(Hirayama) The HDUCLASS and HDUCLASn keywords were added based on the HEASARC FITS Working Group's suggestion made when we met them at NASA/GSFC on May 28th, 2003. See also topic 002 of the latest topics of FT1 and FT2 for more details. I guess they will not appreciate it very much if we drop those keywords, although I can't (and shouldn't) speak for them, of course. In any case, I suggest to ask for their opinion before making a dicision to drop those keywords.
3. TASSIGN keyword
Proposed resolution: As part of a review by the HFWG, we should ask for concurrence about dropping the HDUCLASS keywords. In any case we are far enough from matching the columns of a HDUCLASS1 = EVENTS file that we shouldn't claim to match that definition.
3. TASSIGN keyword
(Digel) TASSIGN is supposed to be where the event times were assigned, and we currently have this set equal to '(Digel) TASSIGN is supposed to be where the event times were assigned, and we currently have this set equal to 'SATELLITE'. This is not strictly true, as we will need some ground processing to turn ticks of the 20 MHz clock into time since the last 1 PPS signal from the spacecraft, and we'll also need to shift GPS time to TT.
...
In any case (to drop it or to keep it), the HEASARC FITS Working Group might have a different opinion on this topic, especially from a point of view of multi-mission support.
4. OBS_ID and OBJECT keywords
Proposed resolution: Drop it unless the HFWG wants a very broad interpretation of the meaning of TASSIGN = 'SATELLITE'.
4. OBS_ID and OBJECT keywords
(Digel) I don't think we need either of these. What OBS_ID was originally intended to represent is not clear, and a specific OBJECT is typically not relevant for the LAT.
...
With reference to the fact that a satellite is not perfectly stable during a pointing, this is normal, but the keywords RA_SCX, DEC_SCX, RA_SCZ, DEC_SCZ define the boresight of the intruments and deviations from the required attitude during a pointing are tagged as bad-time intervals (the complementary to good-time intervals) and data discarded (or later reanalyzed with different methods). In other words, the basic philosophy in pointings is to have certain key parameters taken as reference and to consider deviations from them.
Proposed resolution: Drop OBS_ID and OBJECT.
5. MC_TRUTH keyword
(Digel) I think that the original intent was that this keyword indicate whether the data are entirely Monte Carlo truth values (i.e., actual directions, energies, etc.). We do not actually have files like this, and I propose removing this keyword. gtobsssim does add a column MC_SRC_ID to the files it generates, but this is easy enough to check for without the MC_TRUTH keyword.
...
(Hirayama) Adding a new column to an FT1 file for a special case may not be appreciated by the HEASARC people. I remember they don't like "variants" of an FT1 file, mainly for archival purposes. I think their discussion was like: "for the GLAST team members (who know the file contents very well), it is just one additional column, but for the HEASARC members (who do NOT want to deal with the file contents as much as possible), it is a different file format." Probably it is better to talk to them about it again, once we have our conclusion about this topic, rather than trusting my rusty memory.
Proposed resolution: Drop the MC_TRUTH keyword.
6. EVENT_ID column
(Digel) This is specified as a 32-bit integer. Right now, most likely the event IDs will be assigned by the LAT as a time stamp. Most likely we will need more than 32 bits to represent them, and of course they should look more or less like some kind of integer representation of the value in TIME. Right now, I don't know what to recommend for EVENT_ID.
...
(Hirayama) The topic is also outlined in topic 010 as well as other range problems that might interest you. Also, note that it may not be very easy to identify an event by TIME column after gtbary overwrites its contents for a barycentric correction.
Proposed resolution: Find out how the event numbers will be assigned onboard. From the ISOC-FSW ICD it isn't obvious to me. The EVENT_ID column will be distinct from time and should stay.
7. RECON_VERSION column7. RECON_VERSION column
(Digel) RECON_VERSION is defined as a 2-byte integer to define the version of reconstruction (and classification) algorithm applied for a given event. This is probably more general than we need. In principle, a given events file could have events processsed with different versions of the software, but in practice, I do not expect that this will happen. I recommend that we make RECON_VERSION a header variable instead of a column.
Also, somebody needs to think of how we will translate versions of the geometry, calibration files, reconstruction, and classification, into some fairly compact representation. Certainly we will be closely keeping track of this information for Level 1 processing, and will have a database someplace that can tell us.
Proposed resolution: I'm not sure. We probably could adopt a 'v1r2p3' string for designating the version of reconstruction - say from the version of Gleam used to make the reconstruction, but I don't think that any versioning has been worked out yet for event classification.
8. CALIB_VERSION column
(Digel) As for RECON_VERSION we do not need this as a column. Actually, I'd recommend combining it with RECON_VERSION.
Proposed resolution: Actually, on second thought, I do not recommend combining it with RECON_VERSION, but this is another example where I don't think we know yet how calibrations will be versioned. I'll at least ask the ISOC.
9. IMGOODCALPROP, 9. IMGOODCALPROP, IMVERTEXPROB, IMCOREPROB, IMPSFERRPRED, CALENERGYSUM, CALTOTRLN, and IMGAMMAPROB columns
...
In the case where there are multiple parallel trees for the classification analysis, the best cuts are probably dependent on which tree was used: this implies more variables, or fields in a variable, to describe which path the analysis took.
10. CONVERSION_POINT
Proposed resolution: We need a discrete assignment into event classes that correspond to the response functions we ultimately adopt. Right now all we have is front vs. back, and which class an event belongs to can be determined solely by conversion layer. Having a 'gamma ray-ness' column would be tempting, too, but in principle this is already related to the event classification and we wouldn't have response functions that corresponded to further cuts on the quantity.
10. CONVERSION_POINT
(Digel) This was originally imagined as a way for end users to (Digel) This was originally imagined as a way for end users to decide whether they wanted to believe whether a particular event was not a charged particle and was well reconstructed. Eventually, we will have an event display server available that will make this much easier for users, who would otherwise have to find the geometry of the LAT some place.
...
(Burnett) I agree. The filtering implied by this is already incorporated into the classification tree variablesvariables.
Proposed resolution: Remove CONVERSION_POINT.
11. PULSE_PHASE and ORBITAL_PHASE
...
(Digel) I think that we should consider the FT1 format to define what the data servers deliver. The . The PULSE_PHASE and ORBITAL_PHASE columns are specific to an analysis, and would be wasted space in the server. These columns should be added to files by the pulsar tools that fill them, just as gtdiffresp adds a column for each diffuse source in the particular source model under consideration. None of these analysis-specific columns is fundamentally part of the FT1 data.
Proposed resolution: Remove the PULSE_PHASE and ORBITAL_PHASE columns. They are analysis specific to an analysis, and would will not be wasted space in the server. These columns should be added to files by the pulsar tools that fill them, just as gtdiffresp adds a column for each diffuse source in the particular source model under consideration. None of these analysis-specific columns is fundamentally part of the FT1 datain the event summaries delivered to or from the server. Actually, Jim has pointed out that if the response functions are known and the standard diffuse emission model is also defined, the gtdiffresp columns corresponding to them could be precomputed and stored with the events. This is apparently a big computational savings from the user perspective, and I think should be done. But we aren't in a position now to even say how may different sets of response functions we will have.
12. SKYX/Y issue
(Digel) The Guidelines for Science Tools Design page lists one outstanding issue for the definition of the events file, whether to add columns that give the coordinates of the events in some coordinate projection.
...
(Foschini) Surely having ds9 able to make images directly from the event list is very useful, particularly to have a quick look. In this case, it is necessary to have full WCS keywords in the EVENTS header. But, for what I see the event list files are expected to be huge, so it will be necessary to have an executable able to make a selection of events centered on certain coordinates, with a certain radius of extraction, in a certain time region, within a certain energy band (that is already available). In that case, we can avoid putting the full WCS keywords in the EVENTS header and the executable extracting the selected photon list can add the necessary WCS keywords in the subset of data keywords in the subset of data.
Proposed resolution: Omit SKYX/Y columns. Reluctantly, I also recommend that we should not include Galactic coordinates either.
13. Live time
(Digel) Way back when (until early 2003), the events file was defined to include a column called 'Deadtime' which was to be the 'Deadtime accumulated since the start of the mission.' It was removed, partially because Masa pointed out that information that relates specifically to the LAT belong in the FT2 file. Also, live time turns out to be more convenient to work with.
...
(McEnery) I agree that we should have livetime included as a column in the events file.
Proposed resolution: Add the LIVETIME column.
14. Event summary++
(Digel) LS-002 is an event summary, intended to have information for higher-level analyses. The GSSC expects (reasonably) to receive this summary for every event that is telemetered to the ground (and that is not discarded early in the process as obviously background to save disk space at the ISOC).
They would also like to receive an 'extended' event summary that includes all of the variables that are used by the classification trees. That sounds fine to me, too. Right now, the specification of variables actually used by the trees has not converged, and sending the whole Analysis Ntuple would be overkill. I think that we should be able to define this data product in good detail within the next month.
Proposed resolution: We may be able to define now the additional variables that are actually used in the classification(s).
15. Time Issues
(Foschini) I suggest to adopt the days as time unit, that is of immediate use for scientist. This means to change the TIMEUNIT value to 'd'.
...
For TELAPSE, OBTSTART, and OBTSTOP keywords, see my comment on "17. GTI."
Proposed resolution: Keep the time units as seconds. I need to look into TIERRELA and TIERABSO still.
16. Name of tables
(Foschini) Presently there is only one keyword to identify the template of EVENTS and GTI data structure. To take into account that templates can change (particularly during the first months after the launch), it is better to add a keyword EXTVERS or something similar with the version number of the used template. Moreover, the data structure of LAT and GBM can require different keywords, so I propose to give at the EXTNAME more complicate names to take into account the different uses: for example, for the LAT events there can be EXTNAME='GLAST-LAT_EVT' and the corresponding GTI can be EXTNAME='GLAST-LAT-GTI'.
(Ballet) I don't think it is a good idea to change the names. The INSTRUME and TELESCOP keywords are there to indicate which intruments we are dealing with. Better keep standard names for the EVENTS and GTI extensions, as was done for previous missions such as Chandra or XMM-Newton. I agree to the EXTVERS suggestionsuggestion.
Proposed resolution: Keep the extension names as EVENTS and GTI.
17. GTI
(Foschini) I suggest to add a keyword GTI_NAME to the GTI header to explain the origin of GTI: for example, there could be GTI due to attitude, telemetry, or other. The final GTI can be a merging of the whole types of GTI.
...
For UTC keywords, these are generally expressed as a string: e.g. "2005-11-22T11:43:00". In this case, it is just for the end user, to have something more friendly than JD or anything else. To make the conversion, it is used a time correlation, that is a function that linearly correlates the UTC to the onboard time.
Proposed resolution: There's a lot here. I don't feel strongly about TELAPSE but I recommend that we do not adopt it because as Masa pointed out, it is not so relevant for scanning observations. I need to think more about OBTSTART and OBTSTOP. I am reminded that according to the letter of the law for GTIs, our servers should return corresponding GTIs for each query.