Using revised CAL asymmetry maps
Sasha has created a time series of CAL xtal response maps from on-orbit GCRs that will replace the response maps derived from ground muon calibration. We'll soon use those to measure the PSF with AGN to see if we get agreement between MC PSF and observed PSF. Good.
Note, though, that we are conflating at least four issues here, and we need to think a bit before we assign root cause and create a plan for future delivery of calibration constants. I think we need to do at least two versions of the measurement of PSF with AGN.
I describe the four issues and the two tests below.
Four potential problems in xtal asymmetry maps
The four issues are:
- Time evolution of xtal response maps.
- Poor map algorithm in the penultimate slice (i.e. slices 1 and 10 of 0 to 11).
- Coarse binning of the small/small response ratio.
- Biased ratio from FHE crosstalk when only one end fires FHE.
More details on these issues follow.
(1) Time evolution of xtal response maps.
Sasha has demonstrated that xtal response maps evolve systematically in the direction of increasing asymmetry (= increasing taper = increasing light attenuation) with time as light yield declines. This is consistent with our understanding and ground measurement of radiation damage in CsI(Tl), and the magnitude of the decline in light yield (~1% per year) is consistent with our pre-launch calculation.
See Time evolution of CAL asymmetry maps in the "(C&A 21 June please look at the following section second)" heading. The change in the slope of the xtal response with time is non-trivial in many xtals.
The slope of the xtal response measured just after launch is the same as on the ground. If the PSF was inconsistent with the MC just after launch, and the PSF isn't becoming more and more inconsistent with MC, then I think it's unlikely that the evolution of the xtal maps is important.
(2) Poor map algorithm in the penultimate slice.
See Time evolution of CAL asymmetry maps at the "(For C&A 21 June, look at the plot below first)" heading for a description of the problem.
The algorithm used in the ground xtal map generation didn't properly handle non-uniform sampling of a xtal slice by the calibrating particles, but we fixed that algorithm just around launch time. The result of that weakness in the algorithm is that the ground maps had a biased value for the asymmetry in slices 1 and 10 (of 0 to 11) equivalent to a shift of several mm. The first flight xtal maps did not have this problem because we improved the algorithm, but we never used them. The newly delivered ones use the fixed algorithm (and the trial maps that Sasha sent recently did as well), so they will not suffer from bias in slices 1 and 10. The spline will describe the actual light asymmetry near both ends of each xtal much better for these revised maps than it did with the ground maps.
My guess is that this is the problem with CAL xtal maps that couples into CTBCORE. This is the detectable difference between ground maps and post-launch maps.
Note that I also think it was a mistake to train CTBCORE on an MC dataset that had the same xtal maps used for MC generation and MC recon. Recon used longitudinal position accuracy that the actual CAL cannot possibly provide. I wouldn't be surprised to learn that this contributes significantly to the discrepancy between observed and MC CTBCORE, but that's another discussion.
(3) Coarse binning of the small/small response ratio.
The xtal response maps for the small-versus-small diode (i.e. the HE v. HE map) were created from fitting binned plus-over-minus signal ratios, but the binning was a bit coarse. The on-orbit maps are being created by fits to finer binning in the ratio. This is probably a minor effect, but it is a change from the ground maps.
(4) Biased ratio from FHE crosstalk when only one end fires FHE.
We know that when FHE fires, some small signal is coupled into the slow-shaped spectroscopy channel. As measured with the LCI calibration scripts, it has a small effect (<1%) on the energy read out from the GCFE, and we've decided to ignore it in the front-end linearity calibration constants. When FHE fires in both GCFEs (i.e. at both ends) of a xtal, the longitudinal position calculated in the xtal is unaffected, since the crosstalk is very similar in all channels.
Note, though, that it can have a larger effect on the calculated longitudinal position if only one end of the xtal has FHE firing. This can happen near one end of a xtal, where because of the light taper, the optical signal at the near end is larger than at the far end. See the plot below, which is the average over all crystals, where Sasha finds evidence for a bias of up to a few mm in the segments near the ends of the average xtal near the 1 GeV FHE threshold. Note that the bias is in the direction we expect: it is positive for segment 10, where the signal is typically ~30% bigger than the signal at the negative end (and vice versa in segment 1).
In these plots, Sasha has converted the measured asymmetry to longitudinal offset using the average xtal asymmetry slope (as usual). The segments (slices, whatever) are labeled 0 to 11, where 0 is closest to the negative face and 11 is closest to the positive face.
Tests to perform on an AGN skim or hard pulsar skim
I suggest that we need at least two test measurements of the observed PSF. Those tests are designed to separate issue (1) from (2)+(3), i.e. the time-evolution from the algorithm issues. Because we do not have maps with the crosstalk removed, we don't yet have a test for the effect of (4).
If it's important to improve the purity of the photon data sample in order to see changes in PSF with good sensitivity, I suggest we try a skim of hard (>few GeV) photons from bright pulsars and look only in the on-pulse phase intervals.
(a) Use the on-orbit xtal maps derived for the epoch of the start of the AGN skim (or epsilon before) to analyze the entire year of the skim.
Here we use the on-orbit xtal maps, i.e. maps created with the fixes to the mapping algorithm, to reconstruct the entire year of data. During that time, radiation damage to the xtals will have caused the actual response to have deviated from the maps by of order a few mm, on average, near the xtal ends.
(Of course, we could instead use maps from just after launch to increase the discrepancy. Perhaps we should do that in an additional test.)
I suspect we'll see better agreement with MC CTBCORE and PSF, with still some imperfection, and that neither quantity will evolve with time (which is what Marshall sees for PSF with ground maps).
(b) Use 4 sets (i.e. say, one new set of maps every 3 months) of xtal maps as appropriate in time to analyze the skim.
Here we use on-orbit xtal maps appropriate for each epoch. If the PSF is improved relative to test (a), then we will have demonstrated the need to map every 3 months or so. Otherwise, annual updates will likely be adequate.
Original email
This page is generated from an email I sent on 5 August 2010. The original email is below.
On 5 Aug 2010 (Day 217), at 11:57 AM PDT, J. Eric Grove wrote:
C&A,
Sasha is in the process of creating a time series of CAL xtal response maps from on-orbit GCRs that will replace the response maps derived from ground muon calibration. We'll soon use those to measure the PSF with AGN to see if we get agreement between MC PSF and observed PSF. Good.
Note, though, that we are conflating at least three issues here, and we need to think a bit before we assign root cause and create a plan for future delivery of calibration constants. I think we need to do at least two versions of the measurement of PSF with AGN.
The three issues are:
(1) Time evolution of xtal response maps.
(2) Poor map algorithm in the penultimate slice (i.e. slices 1 and 10 of 0 to 11).
(3) Coarse binning of the small/small response ratio.
The two tests, designed to separate issue (1) from (2)+(3), are:
(a) use the on-orbit xtal maps derived for the epoch of the start of the AGN skim (or epsilon before) to analyze the entire year of the skim.
(b) use 4 sets (i.e. say, one new set of maps every 3 months) of xtal maps as appropriate in time to analyze the skim.
more detail on the issues
(1) Time evolution of xtal response maps.
Sasha has demonstrated that xtal response maps evolve systematically in the direction of increasing asymmetry (= increasing taper = increasing light attenuation) with time as light yield declines. This is consistent with our understanding and ground measurement of radiation damage in CsI(Tl), and the magnitude of the decline in light yield (~1% per year) is consistent with our pre-launch calculation.
See https://confluence.slac.stanford.edu/display/CAL/Time+evolution+of+CAL+asymmetry+calibration
in the "(C&A 21 June please look at the following section second)" heading.
The change in the slope of the xtal response with time is non-trivial in many xtals.The slope of the xtal response measured just after launch is the same as on the ground. If the PSF was inconsistent with the MC just after launch, and the PSF isn't becoming more and more inconsistent with MC, then I think it's unlikely that the evolution of the xtal maps is important.
(2) Poor map algorithm in the penultimate slice.
See https://confluence.slac.stanford.edu/display/CAL/Time+evolution+of+CAL+asymmetry+calibration
at the "(For C&A 21 June, look at the plot below first)" heading for a description of the problem.The algorithm used in the ground xtal map generation didn't properly handle non-uniform sampling of a xtal slice by the calibrating particles, but we fixed that algorithm just around launch time. The result of that weakness in the algorithm is that the ground maps had a biased value for the asymmetry in slices 1 and 10 (of 0 to 11) equivalent to a shift of several mm. The first flight xtal maps did not have this problem because we improved the algorithm, but we never used them. The newly delivered ones use the fixed algorithm (and the trial maps that Sasha sent recently did as well), so they will not suffer from bias in slices 1 and 10. The spline will describe the actual light asymmetry near both ends of each xtal much better for these revised maps than it did with the ground maps.
My guess is that this is the problem with CAL xtal maps that couples into CTBCORE. This is the detectable difference between ground maps and post-launch maps.
Note that I also think it was a mistake to train CTBCORE on an MC dataset that had the same xtal maps used for MC generation and MC recon. Recon used longitudinal position accuracy that the actual CAL cannot possibly provide. I wouldn't be surprised to learn that this contributes significantly to the discrepancy between observed and MC CTBCORE, but that's another discussion.
(3) Coarse binning of the small/small response ratio.
The xtal response maps for the small-versus-small diode (i.e. the HE v. HE map) were created from fitting binned plus-over-minus signal ratios, but the binning was a bit coarse. The on-orbit maps are being created by fits to finer binning in the ratio. This is probably a minor effect, but it is a change from the ground maps.
— more detail on the two tests of observed PSF —
I suggest that we need at least two test measurements of the observed PSF. Those tests are designed to separate issue (1) from (2)+(3), i.e. the time-evolution from the algorithm issues.
(a) use the on-orbit xtal maps derived for the epoch of the start of the AGN skim (or epsilon before) to analyze the entire year of the skim.
Here we use the on-orbit xtal maps, i.e. maps created with the fixes to the mapping algorithm, to reconstruct the entire year of data. During that time, radiation damage to the xtals will have caused the actual response to have deviated from the maps by of order a few mm, on average, near the xtal ends. (Of course, we could instead use maps from just after launch to increase the discrepancy. Perhaps we should do that in an additional test.)
I suspect we'll see better agreement with MC CTBCORE and PSF, with still some imperfection, and that neither quantity will evolve with time (which is what Marshall sees for PSF with ground maps). I suspect we'll have
(b) use 4 sets (i.e. say, one new set of maps every 3 months) of xtal maps as appropriate in time to analyze the skim.
Here we use on-orbit xtal maps appropriate for each epoch. If the PSF is improved relative to test (a), then we will have demonstrated the need to map every 3 months or so. Otherwise, annual updates will likely be adequate.
Johan B created a skim of AGN photons that spans June 2009 - June 2010, but the pipeline processing of GCR data from which xtal maps are made stopped in November 2009. However, until the GCR data processing restores those data, we can't make maps that span the full epoch of Johan's skim.
—
My concern is that we'll once again jump to a conclusion about how and when to update calibration constants without thinking through the causes and consequences. We made a mistake shortly after launch in concluding that the differences between the on-orbit maps we created and the pre-launch maps we were using were small enough to ignore. No one understood the implications of the differences on the full reconstruction problem. I don't want to make a similar mistake again.
Eric