Versions Compared

Old Version 12

changes.mady.by.user Alessandra Filippi

Saved on

compared with

New Version Current

changes.mady.by.user Alessandra Filippi

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Here is an overview of the steps needed to work out the internal alignment of the full vertex detector. Each step is described in more detail below.

  1. Production a Millepede input binary
  2. Run millepede using mille.bin creating a resulting file with correction to the constants, e.g. millepede.res
  3. Create a new compact detector description with the new corrections, e.g. compact_new.xml
    1. java -cp hps-distribution-3X.0Y.4Z-SNAPSHOT-bin.jar org/hps/svt/alignment/BuildMillepedeCompact -c compact.xml millepede.res -o compact_new.xml 
    2. Use option "-r" to replace instead of adding to existing constants in the compact.xml
  4. Create a new detector with this new compact_new.xml file.

Production of a Millepede binary input file

BEWARE: the following procedure (old python-based procedure) is valid only for hps-java versions < 4.2 (oct2018). From this version on, the binary file to be provided as input to Millepede is produced automatically upon the reconstruction, so steps 2-5 are skipped. Go to the section "New procedure" for updated instructions.

Old python-based procedure (hps-java < 4.2 oct18)

The Millepede input binary file is obtained through the following steps:

  1. Reconstruct real data with GBL (to this purpose, you need to include the GblDriver and GblOutputDriver in your steering file).

    1. For Montecarlo data:

      1. run the readoout with the steering file (for instance, for test run data: HPS2014ReadoutNoPileup.lcsim)
      2. run the recon with the steering file (for instance, for test run data: HPS2014OfflineNoPileUp.lcsim)
    2. For real data you can use as steering file: /org/hps/steering/recon/EngineeringRun2015FullReconGbl2.lcsim
  2. Check that at the end of reconstruction a out.gbl ascii file (or, named as you decided in the GblOutputDriver) is produced.
  3. Remember that by default the geometry is taken from the database. If you want to force the use of your own geometry, you must provide it in the compact.xml file in a given detector. For MonteCarlo data, set the run number to zero during readout and reconstruction. This is done adding the flag -Drun=0 when running the readout. For real data, use the -DdisableSvtAlignmentConstants flag.. IMPORTANT: remember to re-compile hps-java before running each time you change the compact.xml file! (this is the most common error).
  4. The out.gblfile is read by a python procedure. You must download with git the current version of the software from the github repository as descrived in the following. This will create a hps-gbl directory. After having configured your account and username for git usage, issue the following commands:(the second and third command need to be issued just upon installation, and they are needed since you will have to download something else from github -see step 1 of the next subsection- and some directories with needed utils are shared in the github repository).
  5. remember to install the GBL software, if you already haven't it. In a directory parallel to hps-gbl download the GBL software using svn:(or check the newest release, and get it). To compile it:
    • cd GeneralBrokenLines/cpp
    • mkdir build; cd build
    • cmake ../
    • make install
    • make doc (if you want it)
    Note: if you have installed the latest cmake version, it probably won't compile. You must prevent the compilation to search for C++11 support (the default for newest cmake). To do this, you have to set as compilation flag -std=c++0xadding it to the c++ compilation line. Either you do it in the cmake configuration files, or (quickest) you add by hand this flag at the end of the CXX_FLAGS line, in the two files:
    • GeneralBrokenLines/cpp/build/CMakeFiles/GBL.dir/flags.make
    • GeneralBrokenLines/cpp/build/examples/CMakeFiles/GBLpp.dir/flags.make
  6. the gbl python procedure reads the out.gbl file and prepares the binary read by Millepede. You mustrun python from the hps-gbl directory. This is the shortest syntax (-h shows all possible options):
    • cd hps-gbl
    • python gbltst-hps.py [-h] out.gbl
    A heap of pdf files are produced containing plots of several quantities for top/bottom halves, with long names that should be self-explaining (but at the moment they are not and they are too long, this must be improved). You will also file a .root file containing the single root histograms, and a .ps file containing a summary of the plots ready to be printed.
    The file gbltst-hps.py contains the instructions to extract the useful information on tracks and hits from the ascii file and write the input file for Millepede. If you want to add/modify some of the output plots/histograms, you have to modify both the gbl_plots.py file (in which they have to be booked) and the gbltst-hps.py file, in which they have to be filled.
    Note: root must be compiled including the python support, otherwise python stops with an error complaining about root libraries missing. A good idea is to put in your profile and instruction to run automatically $ROOTSYS/bin/thisroot.(c)sh, which provides the correct root-python environment and libraries for your system.
    At the end of python run, you should also find a MilleBinaryISN.dat file (the name could slightly change), which is the input file to be read by Millepede.

 

Running Millepede

...

...

  • cd hps-mille
  • ./runMP.py -i../hps-gbl/milleBinaryISN.dat -M NAMES

...

  1. 1-6: layer number
  2. A: axial, S: stereo
  3. h: hole, s: slot
  4. t: top, b:bottom
  5. _t: translation, _r: rotation
  6. u, v, w: coordinates on the sensor reference system

...

    1. whatever steering file (usually the most update version will work)  but remember to include the mentioned GblOutputDriver drivers as mentioned above, if they are not present (usually they are not, for production purposes):
      <driver name="GBLOutputDriver"/>
      ...something else in the lcsim file...
       <driver name="GBLOutputDriver" type="org.hps.recon.tracking.gbl.GBLOutputDriver">
               <debug>0</debug>
               <isMC>false</isMC>   <!-- use true for MC data here -->        
                <gblFileName>${outputFile}.gbl</gblFileName>
      </driver>
      Of course, for straight tracks use a proper steering file.
    2. For the purpose of alignment you are recommended to drop ghost hits, including in the HelicalHitDriver driver the following line:
      <rejectGhostHits>true</rejectGhostHits>

  2. Check that at the end of reconstruction a out.gbl ascii file (or, named as you decided in the GblOutputDriver) is produced.
  3. Remember that by default the geometry is taken from the database. If you want to force the use of your own geometry, you must provide it in the compact.xml file in a given detector. For MonteCarlo data, set the run number to zero during readout and reconstruction. This is done adding the flag -Drun=0 when running the readout. For real data, use the -DdisableSvtAlignmentConstants flag and a proper run number to select other calibration constants. IMPORTANT: remember to re-compile hps-java before running each time you change the compact.xml file! (this is the most common error).
  4. The out.gbl file is read by a python procedure, called gbltst-hps.py. You must download with git the current version of the software from the github repository as described in the following. This will create a hps-gbl directory. After having configured your account and username for git use, issue the following commands:
    (the second and third command need to be issued just upon installation, and they are needed since some directories with utility files are shared with other software packages which will be described later).
    Once you have downloaded the code, remember to install the GBL software, if you already haven't it. In a directory parallel to hps-gbl download the GBL software using svn:
    • svn co http://svnsrv.desy.de/public/GeneralBrokenLines/tags/v01-16-04 GeneralBrokenLines
      (or check the newest release, and get it). To compile it:
    • cd GeneralBrokenLines/cpp
    • mkdir build; cd build
    • cmake ../
    • make
    • make install
    • make doc (if you want it)
      Note: if you have installed the latest cmake version, probably it won't compile. You must prevent the compilation to search for C++11 support (the default for newest cmake). To do this, you have to set as compilation flag -std=c++0x adding it to the c++ compilation line. Either you do it in the cmake configuration files, or (quickest) you add by hand this flag at the end of the CXX_FLAGS line, in these two files:
      • GeneralBrokenLines/cpp/build/CMakeFiles/GBL.dir/flags.make
      • GeneralBrokenLines/cpp/build/examples/CMakeFiles/GBLpp.dir/flags.make
  5. The gbl python procedure reads the out.gbl file and prepares the binary read by Millepede. You must run python from the hps-gbl directory. This is the shortest syntax (-h shows all possible options):
    • python gbltst-hps.py $GBLFILE --name $OUTNAME --ntracks $MAXTRKS --nopause --save
      You can provide the input gbl filename, the output file name and the number of tracks to be processed as logical names in a shell script.
      A heap of pdf files are produced containing plots of several quantities for top/bottom halves, with long names that should be self-explaining (but at the moment they are not and they are too long, this needs to be improved). You will also file a .root file containing the single root histograms, and a .ps file containing a summary of the plots ready to be printed.
      The file gbltst-hps.py contains the instructions to extract the useful information on tracks and hits from the ascii file and write the input file for Millepede; it reruns GBL on the tracks. If you want to add/change some of the output plots/histograms, you have to modify both the gbl_plots.py file (in which they have to be booked) and the gbltst-hps.py file, in which they have to be filled.
      See the help (or, better, the code) for indication to further functionalities:
      •  python gbltst-hps.py --help
      Use the --batch flag if you not running interactively (if you are submitting the job on a batch system, for instance) or you do not want the output histograms to pop out when the elaboration is finished (they can be annoyingly invasive on your screen for about a minute).
      At the end of python processing, you should also find a MilleBinaryISN.dat file (the name could slightly change), which is the input file to be read by Millepede.
      Keep in mind that this procedure is very RAM consuming (all track informations are stored in memory once read from the GBL ascii file). Therefore, in general it is not a good idea to run many multiple sessions, or even single sessions with many events, on the same processor, unless you want to risk your system be stuck. 
      Note on root compilation: root must be compiled including the python support, otherwise python stops with an error complaining about root libraries missing. A good practice is to put in your profile and instruction to run automatically $ROOTSYS/bin/thisroot.(c)sh, which provides the correct root-python environment and libraries for your system.
      Note on straight tracks processing. If you want to process straight tracks (no magnetic field) you need to used a slightly different python script:
      • python gbl-hps-ST.py  $GBLFILE --save --nopause
      Of course, you will void histograms for quantities related to helical track parameters, impact parameters, momenta etc.
      Note on truth MC information: for the moment being, if the GBL procedure is being run on Montecarlo data (flag --mc), no output is extracted for MC truth quantities. This needs to be fixed (it was working up to a certain hps-java version, need to be checked if it is a problem of info extraction from the banks by the python script, or it is something more related to the way the information is stored in the readout/reconstruction of the simulated data).
  6. Once you run GBL on the out.gbl file, you can check the quality of the alignment/geometry studying residuals, kinks, pulls and several other useful distributions. All histograms are contained in a file named gbl_$OUTNAME_*.root (the full name can be very long and depends on the used flags). A set of useful root macros can be found here:The master branch contains macros which are compliant to root v5.34, the branch root6 contains macros working with root6 as well (this is the development branch). In the git bundle you can find a README file with some indications about how to run the macros, and what they do.
    Note: due to a reversed sign in the magnetic field coordinates in the GBL procedure, the charge of tracks in the produced rootfile is reversed as well (keep this in mind when analysing this output). In many cases, also the u-v axes have to be flipped to provide the correct orientation of the sensors in the space (this is already done almost everywhere in the macros). By inspecting the trend of residuals and distributions one can judge which are the most critical degrees of freedom to float in the following Millepede iteration.

 

New procedure (hps-java > 4.1 oct18)

The Millepede input binary file is obtained directly upon running hps-java provided in the steering file the GblOutputDriver driver is called, and some options are added to  GblRefitterDriver. The following procedure must be followed:

  1. Include GblOutputDriver in your steering file and add the following options to GblRefitterDriver:
    1. <driver name="GBLOutputDriver"/>
    2. <driver name="GBLOutputDriver" type="org.hps.recon.tracking.gbl.GBLOutputDriver">
    3. <driver name="GBLRefitterDriver" type="org.hps.recon.tracking.gbl.GBLRefitterDriver">

              <storeTrackStates>true</storeTrackStates>

               <writeMilleBinary>true</writeMilleBinary>

                <milleBinaryFileName>millepedeData.bin</milleBinaryFileName>

       </driver>

    in this way the output binary file to be read by Millepede will be called millepedeData.bin. A further root ouput file, named after -Doutputfile (outputfile.root, the default being GBLplots_ali.root) will be produced, containing residuals and several distributions to be analyzed to judge the alignment quality. See point 3.
  2. Check that at the end of reconstruction the files millepedeData.bin and outputFile.root (GBLplots_aliTo check the quality of the alignment/geometry studying residuals, kinks, pulls and several other useful distributions. A set of useful root macros can be found in github https://github.com/afilippi67/DataQualityMacros: the relevant suite to analyze the new histograms (which have different namings as compared to the "old" procedure) is in branch root6_newReco (the branch root6 is for the "old" procedure) for 2015/2016 geometry configuration, in branch alignment2019 for 2019 setup (one more layer). In the git bundle you can find a README file with some indications about how to run the macros, and what they do. Note that some histograms, if compared to "old" procedure, can be missing (in particular, those related to track momenta corrections or output helix parameters).
    Note on histogram file naming: the macros are set to extract a proper tag label for the output file from the input file name. The tag name is taken as all characters between "/" and "_" (so both the characters are mandatory in the filename), they should be renamed properly if the underscore is missing (due to a change in the default name, from in the steering file). 

Running Millepede

  1. Once you have the binary file, millepede is ready run from the hps-mille directory. This directory is setup downloading the Millepede software by github using the following commandRemember to compile the fortran sources of the MillepedeII software. It comes with the git bundle (or you can download it from https://www.wiki.terascale.de/index.php/Millepede_II) but you have to compile it in your system beforehand (note: with gfortran you might have to slightly modify the Makefile by hand, because the code version is frozen and not aligned anymore to more modern gfortran versions/libraries).
  2. To run millepede use the following commands:
    • cd hps-mille
    • ./runMP.py -i../hps-gbl/milleBinaryISN.dat -M NAMES
    where NAMES is a list of parameters coded via the following regexp: L(1-6)[AS]?[hs]?[tb]_([tr])([uvw]) having the following meaning:
    1. 1-6: layer number
    2. A: axial, S: stereo
    3. h: hole, s: slot
    4. t: top, b:bottom
    5. _t: translation, _r: rotation
    6. u, v, w: coordinates on the sensor reference system
    if some of the parameters preceded by "?" are omitted, both the choices are selected
    Millepede produces as output, among several files, the millepede.res file which contains the corrections found by Millepede for the floated parameters.
    To keep track of the subsequent iterations, it is wiser to store them file that can be read automatically by the runMP.py procedure. The file is called floatoptions.py and it contains a series of "options", each of them containing a list of sensors to be floated in the same or multiple iterations, coded following the regexp pattern described above. The user can add as many options as needed, ordering them by consecutive numbers which are then used to label a "switch number" used to select the needed option upon running the procedure. The corresponding command to be issued is:
    • ./runMP_batch.py --run --files milleBinaryISN.dat -s switchNumber -b
    The output file will be labeled with the title string provided in the option definition and the sequence of desired floating operations (keep it simple because when its gets too long -usually for several iterations in one go or multiple sensor floatings- the procedure breaks). Usually a run with Millepede takes some minutes on about 1M tracks, with a reasonable amount of floated parameters, so it is not a time consuming step.
  3. Note on binary input files. 
    1. Millepede can run on multiple input files and they should be processed all together and concatenated properly. This has been tested with the latest available MPII which is present already in this fork of hps-mille:https://github.com/pbutti/hps-mille  (will be harmonized with Alessandra's and Per's repos soon). runMP.py has an option (-l) to give a listfile containing the path to all the millepede binary files in order to run on all of them at the same time. 
    2. An equivalent way to concatenate multiple files is to use cat to concatenate them to get cumulated statistics (keep in mind that it can only process 1.5M tracks in total, so keep under control the size of your samples). What you need to do, for instance when you want to provide a mixed sample of curved+straight tracks, is to use a single input file which merges several other partial binary files. To produce it you just need to concatenate the single files, like:
      • cat mille_straight1.dat mille_curved1.dat mille_straight2.dat ... > mille_mixed.dat
    3. When merging curved and straight track sets together, try to sort them evenly in order to get an homogeneous composition of the final sample.  

Create a new compact based on Millepede corrections

The new geometry file is produced in the hps-mille directory. Use the following python wrapper that reads the millepede.res file and writes the compact_new.xml file (it replaces the procedure describes in point 3a of the introductory notes in this section):

  • ./buildCompact.py [-h] -j JARFILE -c compactFile_youWantToUpdate.xml -r millepede.res -t

JARFILE is the name of the hps-java jarfile you used/will be using to reconstruct your data files with the aligned (new) geometry. You have to provide the name of the geometry file used for the initial reconstruction of the data, with respect to which the millepede offsets will be added to produce the new geometry file. The flag -t is needed as some of the parameters in the compact.xml file are already been modified by some expressions introducing physical constraints (for global alignment).

ATTENTION: all rotation offsets have to be summed, which means their sign doesn't have to be flipped when building the new compact.xml file. In some cases two "-" signs appear the the xml file: don't worry, this behaves as expected in the  java code.

 

Create a new detector based on a new compact.xml file

The instruction on how to create a new detector given your new .xml geometry file can be found here: Detector Geometry Overview, section: Adding A New Detector. Remember to rename the compact_new.xml file into compact.xml, write a few lines of clever/useful comment in the .xml file to remember which kind of geometry this is, and recompile hps-java before running with the new geometry. If you want to to process straight tracks with the new geometry, remember to modify properly the compact.xml file setting the constBFieldY variable to zero, and commenting out the magnetic field map file definition in <fields> while activating the constant BoxDipole field.

 

That's it! If you got to this point, either you already find a perfect alignment or you are ready to restart the full procedure from point 1 of the list at the beginning of this section, for an uncountable number of iterations.

 

...

 

Note on beam spot integration in Millepede

BEWARE: not recommended for beginners: at your risk! note that the procedure for the moment being does not provide satisfactory results, so this paragraph is kept for reference purpose mainly)

If you want to integrate the beam spot information in Millepede, which means to include two fictitious sensors whose center coordinates have to be adjusted to make tracks from the top and bottom halves converge to the same point, you have to provide the (provisional) target coordinates and their geometry frame since the beginning of the procedure. Therefore you need to:

  • include in the steering lcsim file the information for the beam spot (which you will modify in the following iterations by hand) providing the following parameters in GblOutputDriver: 

    <beamspotScatAngle>0.005</beamspotScatAngle>
    <beamspotWidthZ>0.05</beamspotWidthZ>
    <beamspotWidthY>0.2</beamspotWidthY>
    <beamspotTiltZOverY>0.26</beamspotTiltZOverY>
    <beamspotPosition>0.0 -0.11 -0.05</beamspotPosition>

    where you will need to modify the coordinates of the beam spot position (and widths)in subsequent iterations. These are the default values introduced for alignment version 4.4 optimized   on 2015 curved tracks. Moreover, you need to activate the inclusion of the beamspot adding the line 
    <addBeamspot>true</addBeamspot>
    By default, the beam spot coordinates are not added and the reconstruction assumes it to be located in (0,0,0). With this add-on the beam spot coordinates and track parameters at the additional 0-layers are added in the out.gbl ascii file, for further processing by gbltst-hps.py.
  • include the --beamspot flag upon running the gbltst-hps.py script to extract the "layer 0" information from the gbl ascii file
  • activate the implementation of the beam spot when running Millepede: this is done using the flag --BSC (which imposes constraints in pairs between the top and bottom degrees of freedom for the additional zero layers). You can also introduce the --SC flag to constrain parameters within a 50 microns precision (this can be done even without introducing the beam spot). In the floatoptions.py file, or in the list of parameters to be floated you can provide by hand, you can introduce L0t and L0b layers (axial and stereo), with the desired directions to be floated. The Millepede id's for each of the 0-layers are the following:
    • top:
      • u translations: 11198: L0t stereo, 11199: L0t axial
      • v translations: 11298: L0t stereo, 11299: L0t axial
      • w translations: 11398: L0t stereo, 11399: L0t axial
    • bottom:
      • u translations: 21198: L0b axial, 21199: L0b stereo
      • v translations: 21298: L0b axial, 21299: L0b stereo
      • w translations: 21398: L0b axial, 21399: L0b stereo
    Rotations for 0-layers can be dropped as not meaningful.
  • Millepede writes in the .res file the offsets found for these new layers, but you cannot produce a new geometry out of them: that is, they cannot be read by the buildCompact.py procedure that produces the new geometry including the MP offsets, which can include information about the physical layers only. Therefore you need to translate the obtained information, which is in the 0-sensor reference system, to the tracking reference system, in which the beam spot coordinates introduced in the .lcsim file are defined. To do this, you should beforehand perform the proper reference system transformation from 0-layers reference system to the tracking one. As an approximation, however, the information from the impact parameters can be exploited. To do this:
    • prepare a new geometry with the modified parameters (all floated layers BUT layer-0)
    • re-run reconstruction with previous beamspot, get the mean value and sigmas of the d0 and z0 distributions produced after running the gbltst-hps.py script, and use the mean value between top and bottom (half sum with signs) as new input beam spot coordinates in the lcsim steering file. Remember that in the tracking system <d0> corresponds approximately to beamspotPosition_y, and <z0> to beamspotPosition_z (while beamspotPosition_x=0).
    • introduce the new beam spot coordinates in the .lcsim file and repeat the procedure until a reasonable convergence is found (it may take very long).

 

 

 

 

 

 

 

...

Create a new compact based on Millepede corrections

The new geometry file is produced in the hps-mille directory. Use the following python wrapper that reads the millepede.res file and writes the compact_new.xml file (it replaces the procedure describes in point 3a of the introductory notes in this section):

  • ./buildCompact.py [-h] -j JARFILE -c compact_new.xml -r millepede.res -t

JARFILE is the name of the hps-java jarfile you used/will be using to reconstruct your data files with the aligned (new) geometry.

The flag -t is needed as some of the parameters in the compact.xml file are already been modified by some expressions introducing physical constraints.

ATTENTION: all rotation offsets have to be summed, which means their sign doesn't have to be flipped when building the new compact.xml file. In some cases two "-" signs appear the the xml file, you need to replace the "- -" string with a "+" (not sure of the overall effect if the correction is not applied - this must be fixed).

 

Create a new detector based on a new compact.xml file

The instruction on how to create a new detector given your new .xml geometry file can be found here: Detector Geometry Overview, section: Adding A New Detector. Remember to rename the compact_new.xml file into compact.xml, write a few lines of clever comment in the .xml file to remember which kind of geometry this is, and recompile hps-java before running with the new geometry.

You are ready now to restart the full procedure restarting from point 1 of the list at the beginning of this section.