The true parameters are taken (or inferred) from the 3EG catalog assuming a power-law spectrum. See the description of the default parameterization in the Likelihood tutorial).
In order to have a quantitative estimate of how well the distributions of fit parameters match expectations, a KS test is used. In order to apply this test, we need the expected distribution for a given fit parameter, centered on or with its mode located at the value that was input to the simulations. It is not clear what this expected distribution should be, so here we make the assumption that it is a Gaussian function centered on the input value and having the same root-variance as the distribution of fitted values from the MC trials. This is not a conservative assumption, but useful in this context as it provides the best possible KS probability for any given set of trials, i.e., the real situation is worse than what is presented here.
In the tables below, the Monte Carlo distributions are described by the mean value and the root variance.
Counts maps for the Crab_Pulsar, the three anticenter sources, and extragalactic diffuse tests are prepared for the following geometry:
E min = 30, E max = 3e5 MeV, 39 bins, logrithmically spaced
RA center = 83 o, eighty 0.5 o pixels
Dec center = 22 o, eighty 0.5 o pixels
Prefactor |
|
|
Index |
|
|
---|---|---|---|---|---|
true value |
MC dist |
KS prob |
true value |
MC dist |
KS prob |
27.00 |
27.21 +/- 1.92 |
4.14e-03 |
-2.19 |
-2.16 +/- 0.06 |
1.59e-26 |
Here is a FITS image of the count map, summed over energies, used in these fits.
Prefactor |
|
|
Index |
|
|
---|---|---|---|---|---|
true value |
MC dist |
KS prob |
true value |
MC dist |
KS prob |
13.65 |
14.37 +/- 1.70 |
6.31e-25 |
-2.46 |
-2.55 +/- 0.13 |
1.87e-73 |
Prefactor |
|
|
Index |
|
|
---|---|---|---|---|---|
true value |
MC dist |
KS prob |
true value |
MC dist |
KS prob |
27.00 |
26.74 +/- 2.19 |
9.48e-03 |
-2.19 |
-2.15 +/- 0.07 |
3.61e-52 |
Prefactor |
|
|
Index |
|
|
---|---|---|---|---|---|
true value |
MC dist |
KS prob |
true value |
MC dist |
KS prob |
23.29 |
23.62 +/- 1.88 |
1.96e-07 |
-1.66 |
-1.66 +/- 0.04 |
1.69e-02 |
These analyses use this ROI file.
Prefactor |
|
|
Index |
|
|
---|---|---|---|---|---|
true value |
MC dist |
KS prob |
true value |
MC dist |
KS prob |
27.00 |
28.19 +/- 1.84 |
3.55e-62 |
-2.19 |
-2.22 +/- 0.05 |
1.39e-60 |
Prefactor |
|
|
Index |
|
|
---|---|---|---|---|---|
true value |
MC dist |
KS prob |
true value |
MC dist |
KS prob |
27.00 |
26.99 +/- 1.28 |
2.16e-01 |
-2.19 |
-2.19 +/- 0.04 |
2.44e-01 |
Here is a constrasting analysis. Using Markov Chain Monte Carlo, we sample the posterior distribution of the fit parameters for a single trial. (NB: Since the prior distributions of the fit parameters are assumed to be flat, the posterior distribution is the likelihood function.) An advantage of using the Markov Chain is that binning over any single parameter in the chain automatically marginalizes over the remaining parameters.
Error estimates for any single fit are given by taking the square root of the diagonal elements of the covariance matrix, which in turn is estimated as the inverse Hessian of the -log-likelihood. Note that the definition of the confidence interval corresponds to a specified change in the log-likelihood along a given direction in parameter space. The estimates obtained from the inverse Hessian are only accurate insofar as the likelihood surface at the local minimum can be represented as a quadratic function of the model parameters.
The blue curves are the best-fit parameter values and error estimates obtained from the inverse Hessian represented as Gaussian functions. The red curves are Gaussians fit to the histograms.
Prefactor |
|
|
Index |
|
|
---|---|---|---|---|---|
true value |
MC dist |
KS prob |
true value |
MC dist |
KS prob |
13.65 |
14.62 +/- 1.75 |
3.13e-40 |
-2.46 |
-2.52 +/- 0.11 |
5.13e-40 |
Prefactor |
|
|
Index |
|
|
---|---|---|---|---|---|
true value |
MC dist |
KS prob |
true value |
MC dist |
KS prob |
27.00 |
28.10 +/- 2.03 |
7.68e-42 |
-2.19 |
-2.22 +/- 0.06 |
1.63e-42 |
Prefactor |
|
|
Index |
|
|
---|---|---|---|---|---|
true value |
MC dist |
KS prob |
true value |
MC dist |
KS prob |
23.29 |
24.38 +/- 1.74 |
4.84e-59 |
-1.66 |
-1.68 +/- 0.03 |
4.37e-58 |
Prefactor |
|
|
Index |
|
|
---|---|---|---|---|---|
true value |
MC dist |
KS prob |
true value |
MC dist |
KS prob |
13.65 |
13.61 +/- 1.21 |
2.04e-01 |
-2.46 |
-2.46 +/- 0.09 |
1.31e-02 |
Prefactor |
|
|
Index |
|
|
---|---|---|---|---|---|
true value |
MC dist |
KS prob |
true value |
MC dist |
KS prob |
27.00 |
26.97 +/- 1.42 |
7.77e-01 |
-2.19 |
-2.19 +/- 0.05 |
7.37e-01 |
Prefactor |
|
|
Index |
|
|
---|---|---|---|---|---|
true value |
MC dist |
KS prob |
true value |
MC dist |
KS prob |
23.29 |
23.39 +/- 1.32 |
1.96e-01 |
-1.66 |
-1.66 +/- 0.03 |
2.49e-02 |
As before, the blue curves are the best-fit parameter values and error estimates obtained from the inverse Hessian represented as Gaussian functions; the red curves are Gaussians fit to the histograms.
The ROI file.
Prefactor |
|
|
Index |
|
|
---|---|---|---|---|---|
true value |
MC dist |
KS prob |
true value |
MC dist |
KS prob |
1.60 |
1.48 +/- 0.06 |
0.00e+00 |
-2.10 |
-2.10 +/- 0.03 |
5.79e-08 |
Prefactor |
|
|
Index |
|
|
---|---|---|---|---|---|
true value |
MC dist |
KS prob |
true value |
MC dist |
KS prob |
1.45 |
1.67 +/- 0.05 |
1.53e-34 |
-2.10 |
-2.21 +/- 0.02 |
3.97e-33 |