ft2Util_2 testing

Here I collect the results of the testing I am performing, comparing the output generated by the old ft2Util code (written by Andrea Tramacere) with the output generated by my code (ft2Util_2).

I used the version of the old code currently used in the PROD pipeline (v1r2p32), and I compiled my code and the old one against GlastRelease-v15r47p12gr21.

Fake FT2

Testing procedure

I collected a bunch of Magic 7 files for a total of ~50 days of in-orbit operation. These M7 files are the one produced by L1, in their latest versions. Thus, they should be complete (without gaps). This is, in other words, the "ideal" situation. They are not contiguous, since I've taken them directly from disk without caring for the continuity, since it shouldn't matter here.

Then I run the executables to produce the fake FT2 files (i.e., without LIVETIME), and I compared the results.  

These are the two command lines I used:

• Old version: 

  makeFT2Entries.exe -M7File $magic7 \
                      -FT2_fits_File ${filenameroot}_andrea.fits \
                      --Gleam --test-quaternion \
                      -DigiTstart $start -DigiTstop $stop \
                      -new_tpl /afs/slac.stanford.edu/g/glast/ground/releases/volume02/ScienceToolsv9r18p5/fitsGen/v6r1p1/data/ft2.tpl 

• New version: 

  makeFT2.exe -m7file $magic7 \
              -ft2file ${filenameroot}_io.fits \
              -ft2start $start \
              -ft2stop $stop

  (without specifying a template, the new version uses the template provided by the current version of fitsGen)

Results

The following plots show the results (mind the scales on the y axes, usually the differences are very small!). For every quantity q (like LAT_GEO, LON_GEO, RA_SCX, DEC_SCX...) I computed (q_old-q_new)/q_old , that is, the "fractional difference".

As you can see, there are differences, but they are very small.

The maximum difference is of the order of ~ 2%, and it is in QSJ_3. QSJ_3 is the Z component of the vectorial part of the attitude quaternion. Usually the attitude quaternion is taken directly from the ATT message in the Magic 7 file. Those messages are issued by the spacecraft 5 times per second. Anyway, there are cases when there is no ATT message corresponding to a particular time interval, thus the codes have to interpolate (or extrapolate). This usually can happen at the very beginning of a run, or because of gaps in the Magic 7 file. The differences between the two codes arises mainly from two different strategies adopted for the interpolation/extrapolation: the old code uses only ATT messages arriving in sync with an ORB message (that is, one time per second), discarding the others; the new code, instead, uses all the available ATT messages. Thus, the new code should provide better interpolation/extrapolation. Anyway, the differences are really small, and I can't think of any application where they could really matter... 

Run FT2

Testing procedure

I selected 100 runs (from 323423607 to 323989797), then for each run I submitted a different jobs to the farm. Each job performed this steps:

1. Download the Digi, Merit, Magic7L1 and DigiGap files using the datacatalog, placing them in the /scratch directory of the batch workers. 
2. Run the old code and mine, with the following command lines: 
   1. Old version: 

      makeFT2Entries.exe -M7File $magic7 \
                         -FT2_fits_File ${filenameroot}_andrea.fits \
      		   -DigiTstart $start \
                         -DigiTstop $stop \
      		   -new_tpl $AndreaTemplate \
                         -DigiFile $digifile \
                         -MeritFile $meritfile \
                         -Gaps_File $digigapfileNew version: 
      

   2. New version: 

      makeFT2.exe -m7file $magic7 \
                  -ft2file ${filenameroot}_io.fits \
                  -ft2start $start \
                  -ft2stop $stop \
                  -digifile $digifile \
                  -meritfile $meritfile \
                  -gapfile $digigapfile
      

      (without specifying a template, the new version uses the template provided by the current version of fitsGen) 
      $start and $stop are respectively the time stamp of the first and the last event in the Digi file. 
3. Run a python script which matches the entries in the two FT2 files and produces another FITS file containing the fractional difference for each quantity for each entry.
4. Move back the produced files to my user disk  

When every job finished, I simply merged all the FITS files produced by the python script in one big FITS file, and generate the plots.

Results

I show here only the results concerning the live time. The results related to the other quantities are very similar to the previous test.  

In the first panel there is the fractional difference for LIVETIME: as you can see, there are differences up to a few percent (~5%). The second panel shows a zoom of a region: we can clearly see a small growing trend inside each and every run, and some occasional larger "flares". As you can see in the third panel, those flares seems symmetrically distributed around the zero line, with a upward fluctuation followed by a downward fluctuation of the same amount. But, two fluctuation can combine giving a third fluctuation which is the algebraic sum of the former two. This also means that this should NOT usually be a problem for analysis using 30s FT2 files, or in general spanning more than few seconds in general.

To investigate the issue, I've produced a Digi file where I've overwritten the livetime counter in each event with the elapsed time. This means a dead time of zero, thus every bin in the FT2 file (except the first and the last one) must have LIVETIME = 1. Then, I run both codes on this Digi file, with the following results:

As you can see, while my code gives exactly 1 for every bin, the old code has the same behavior as with the real file, and it's clearly wrong, giving a LIVETIME greater than the elapsed time. As a further check, I repeated the same procedure imposing a dead time fraction of 0.5 (livetime=elapsed/2), obtaining the same results: