Page History
...
- thr_low: 120
- thr_high: 250
- son_min: 7
Hover the mouse pointer over the Bragg peaks to study the intensities. The sum of the Bragg peak pixels are above 500 ADUs. Set the following values:
- Set atot_thr: 500
If you have trouble finding peaks due to high background scattering, you should turn on median background subtraction. In the Experiment Parameters panel, set the Image properties to "median background corrected ADU".
Let's launch the peak finder on the cluster for a small number of events to see how this set of parameters performs. The output directory on the Peak Finder panel should already be automatically set to: /reg/d/psdm/cxi/cxitut13/scratch/<username>/psocake
...
The default setting will analyze run 10 on psanaq with 24 CPUs. Number of events to process set to -1 analyzes all the events.
For this demo, let's analyze only 50 events with 2 CPUs.
- Run(s): 10
- Queue: psanaq
- CPUs: 2
- Number of events to process: 50
Click "Launch peak finder”. You can check the status of launched jobs by typing "bjobs" in your Terminal. This will display the job ID and the status amongst other information.
A logfile of the peak finding is also saved under the same directory, .<jobID>.log. Note the "." in front of the logfile name.
If the status stays in PEND or SUSP mode for awhile, then you may want to kill the jobs. To kill a batch job, type "bkill <jobID>". For more information, please refer to: Submitting Batch Jobs
Click "Refresh" in Small Data panel to see the progress. You should see a red plot showing event number vs number of peaks found. When peak finding finishes, the plot will turn blue.
Note: You can launch peak finding jobs on multiple runs when you are ready to analyze the entire experiment, e.g. Run(s) 1,10:13 will launch peak finding on runs 1, 10, 11, and 13.
The CXIDB file is generated in the output directory in the following format: <experiment name>_<4-digit run number>.cxi (i.e., /reg/d/psdm/cxi/cxitut13/scratch/<username>/psocake/r0010/cxitut13_0010.cxi)
Two virtual powder patterns will also be generated in the run directory:
1) cxitut13_0010_maxHits.npy: Maximum pixel values for hits found
2) cxitut13_0010_maxMisses.npy: Maximum pixel values for misses
If the hit finding parameters are good, cxitut13_0010_maxMisses.npy should not contain many Bragg spots. You can look at these images by using the "Load image" button in the Image Control panel.
At CXI, the detector geometry may not be centred exactly. You can fix the centre by clicking "Deploy automatically centred geometry" in the Diffraction geometry panel. This uses cxitut13_0010_maxHits.npy and cxitut13_0010_maxMisses.npy.
Jumping to interesting images based on the number of peaks
Once you have submitted the peak finder job, let's plot the number of peaks found for each event by clicking the "Refresh" button. The refresh button will sometimes not work if your .cxi file is busy writing.
In the small data panel, you should see the CXIDB filename:
- filename: /reg/d/psdm/cxi/cxitut13/scratch/<username>/psocake/r0010/cxitut13_0010.cxi
- dataset: /entry_1/result_1/nPeaksAll
"/entry_1/result_1/nPeaksAll" is an array containing number of peaks found for each event.
This should load a plot of all the peaks found so far per event. Click "Refresh" to update the plot even if your batch job is still running.
You can click on the red marker in the plot to jump to the corresponding events. These single crystal diffraction patterns of lysozyme seem to have between 15 to 60 peaks.
Indexing crystals
First things first, crystal indexing requires an accurate detector geometry. Latest CXI geometry files can be found here: Geometry history
Detector panels can manually adjusted using: calibman
If you are on a psana machine, you can run CrystFEL programs by setting up your environment:
| Code Block | ||
|---|---|---|
| ||
source /reg/g/cfel/crystfel/crystfel-dev/setup-sh
CFDEPDIR=/reg/g/cfel/dependencies
export PATH=${CFDEPDIR}/bin:$PATH |
Secondly, we need to know the diffraction geometry before we can index. It is often the case at the beginning of a beamtime that the detector distance to the interaction point (coffset) is not precisely known (We are talking about sub-millimetre precision), and we can use the Diffraction Geometry panel to find this distance.
- In the diffraction geometry panel, set "Detector distance" to 156mm.
- In the indexing panel, tick "Indexing on". If indexing succeeds, the integrated (predicted) peaks will be marked with magenta circles. These triple rings represent the integration radius. A magenta triangle means wait. If indexing fails, you will see a magenta X.
- If you see magenta circles and magenta unitcell appear, congratulations! You have indexed your first pattern using psocake.
- Try incrementing this distance in 1mm step till the unit cell parameters are as close as possible to lys.cell. The optimum detector distance is around 158mm.
Everytime the "Detector distance" value is changed by the user, psocake converts the psana geometry (in /reg/d/psdm/cxi/cxitut13/calib/CsPad\:\:CalibV1/CxiDs1.0\:Cspad.0/geometry/10-11.data) to a CrystFEL geom file (in /reg/d/psdm/cxi/cxitut13/scratch/<username>/psocake/<runNumber>/.temp.geom).
Indexing panel uses CrystFEL to index the diffraction patterns, so the input parameters in the indexing panel should be familiar to you if you've used indexamajig before.
CrystFEL geometry: This geometry file is automatically converted from our psana geometry to CrystFEL geometry for you. Feel free to look inside .temp.geom. If you have a CrystFEL geometry file that you know is good, you can simply type it in. Psocake will never modify this file even if you change the "detector distance" in the diffraction geometry panel. (Just don't name your geometry .temp.geom, it will get overridden). You can also deploy the CrystFEL geometry as a psana geometry by clicking "Deploy CrystFEL geometry" in the indexing panel.
Integration radii: These 3 numbers define the radius of two concentric rings about each Bragg spot. Inner ring is used to integrate the Bragg spot and the outer ring is used to estimate the background. Try adjusting these numbers and see what is being integrated on screen. It should be large enough to fit a Bragg spot inside the inner ring.
PDB: If you have a CrystFEL unitcell, you can constrain the indexing algorithms to look for this unit cell.
Indexing method: Default is mosflm-noretry, dirax. "retry" is used to speed up mosflm (it can take few seconds).
Tolerance: These 4 numbers define how much wriggle room you want for indexing. 5, 5, 5 are the tolerance level for unitcell axes a, b, c. 1.5 is the tolerance level for the angles alpha, beta, gamma.
Extra CrystFEL parameters: You can enter extra parameters for indexamajig in this field. It will be appended at the end of the command line, e.g. --profile will turn on the processing timing information.
Let's try to index another diffraction pattern at event 44.
- In the experiment parameters panel, set Event Number to 44.
- You should see the magenta triangle appear again. Wait few seconds and hopefully you will have indexed another pattern.
Hopefully, you have indexed this diffraction pattern. Notice that the unitcell parameters are a bit off compared to what is expected. Let's load a CrystFEL unitcell file to help the indexer along.
- In the Indexing panel, set the PDB field to: /reg/d/psdm/cxi/cxitut13/scratch/psocake/lys.cell
| Code Block | ||
|---|---|---|
| ||
CrystFEL unit cell file version 1.0
lattice_type = tetragonal
centering = P
unique_axis = c
a = 77.05 A
b = 77.05 A
c = 37.21 A
al = 90 deg
be = 90 deg
ga = 90 deg |
You should notice that the reindexed result will match the known unit cell parameters more closely.
To check which queues are free, type the following in the terminal:
| Code Block | ||
|---|---|---|
| ||
$ bqueues | grep ps
psnehhiprioq 117 Open:Active - - - - 0 0 0 0
psfehhiprioq 117 Open:Active - - - - 0 0 0 0
psnehprioq 116 Open:Active - - - - 0 0 0 0
psfehprioq 116 Open:Active - - - - 0 0 0 0
psnehq 115 Open:Active - - - - 0 0 0 0
psfehq 115 Open:Active - - - - 520 276 244 0
psdebugq 114 Open:Active - - - - 0 0 0 0
psanaq 112 Open:Active - - - - 7466 6612 854 0
psnehidleq 70 Open:Active - - - - 96 0 96 0
psfehidleq 70 Open:Active - - - - 104 0 0 104
psanaidleq 70 Open:Active - - - - 552 432 0 120 |
At the time of writing this documentation, psanaq is quite busy processing 7466 jobs. 6612 jobs are pending. 854 jobs are currently running. For more information on which queue you are allowed to use, see Submitting Batch Jobs
Click "Launch peak finder”. You can check the status of launched jobs by typing "bjobs" in your Terminal. This will display the job ID and the status amongst other information.
A logfile of the peak finding is also saved under the same directory, .<jobID>.log. Note the "." in front of the logfile name.
If the status stays in PEND or SUSP mode for awhile, then you may want to kill the jobs. To kill a batch job, type "bkill <jobID>". For more information, please refer to: Submitting Batch Jobs
You can check the status of your peak finding job here: /reg/d/psdm/cxi/cxitut13/scratch/<username>/psocake/r0010/status_peaks.txt
Also, click "Refresh" in Small Data panel to see the progress. You should see a red plot showing event number vs number of peaks found. When peak finding finishes, the plot will turn blue.
Note: You can launch peak finding jobs on multiple runs when you are ready to analyze the entire experiment, e.g. Run(s) 1,10:13 will launch peak finding on runs 1, 10, 11, and 13.
The CXIDB file is generated in the output directory in the following format: <experiment name>_<4-digit run number>.cxi (i.e., /reg/d/psdm/cxi/cxitut13/scratch/<username>/psocake/r0010/cxitut13_0010.cxi)
Two virtual powder patterns will also be generated in the run directory:
1) cxitut13_0010_maxHits.npy: Maximum pixel values for hits found
2) cxitut13_0010_maxMisses.npy: Maximum pixel values for misses
If the hit finding parameters are good, cxitut13_0010_maxMisses.npy should not contain many Bragg spots. You can look at these images by using the "Load image" button in the Image Control panel.
At CXI, the detector geometry may not be centred exactly. You can fix the centre by clicking "Deploy automatically centred geometry" in the Diffraction geometry panel. This uses cxitut13_0010_maxHits.npy and cxitut13_0010_maxMisses.npy.
Jumping to interesting images based on the number of peaks
Once you have submitted the peak finder job, let's plot the number of peaks found for each event by clicking the "Refresh" button. The refresh button will sometimes not work if your .cxi file is busy writing.
In the small data panel, you should see the CXIDB filename:
- filename: /reg/d/psdm/cxi/cxitut13/scratch/<username>/psocake/r0010/cxitut13_0010.cxi
- dataset: /entry_1/result_1/nPeaksAll
"/entry_1/result_1/nPeaksAll" is an array containing number of peaks found for each event.
This should load a plot of all the peaks found so far per event. Click "Refresh" to update the plot even if your batch job is still running.
You can click on the red marker in the plot to jump to the corresponding events. These single crystal diffraction patterns of lysozyme seem to have between 15 to 60 peaks.
Indexing crystals
First things first, crystal indexing requires an accurate detector geometry. Latest CXI geometry files can be found here: Geometry history
Detector panels can manually adjusted using: calibman
If you are on a psana machine, you can run CrystFEL programs by setting up your environment:
| Code Block | ||
|---|---|---|
| ||
source /reg/g/cfel/crystfel/crystfel-dev/setup-sh
CFDEPDIR=/reg/g/cfel/dependencies
export PATH=${CFDEPDIR}/bin:$PATH |
Secondly, we need to know the diffraction geometry before we can index. It is often the case at the beginning of a beamtime that the detector distance to the interaction point (coffset) is not precisely known (We are talking about sub-millimetre precision), and we can use the Diffraction Geometry panel to find this distance.
- In the diffraction geometry panel, set "Detector distance" to 156mm.
- In the indexing panel, tick "Indexing on". If indexing succeeds, the integrated (predicted) peaks will be marked with magenta circles. These triple rings represent the integration radius. A magenta triangle means wait. If indexing fails, you will see a magenta X.
- If you see magenta circles and magenta unitcell appear, congratulations! You have indexed your first pattern using psocake.
- Try incrementing this distance in 1mm step till the unit cell parameters are as close as possible to lys.cell. The optimum detector distance is around 158mm.
Everytime the "Detector distance" value is changed by the user, psocake converts the psana geometry (in /reg/d/psdm/cxi/cxitut13/calib/CsPad\:\:CalibV1/CxiDs1.0\:Cspad.0/geometry/10-11.data) to a CrystFEL geom file (in /reg/d/psdm/cxi/cxitut13/scratch/<username>/psocake/<runNumber>/.temp.geom).
Indexing panel uses CrystFEL to index the diffraction patterns, so the input parameters in the indexing panel should be familiar to you if you've used indexamajig before.
CrystFEL geometry: This geometry file is automatically converted from our psana geometry to CrystFEL geometry for you. Feel free to look inside .temp.geom. If you have a CrystFEL geometry file that you know is good, you can simply type it in. Psocake will never modify this file even if you change the "detector distance" in the diffraction geometry panel. (Just don't name your geometry .temp.geom, it will get overridden). You can also deploy the CrystFEL geometry as a psana geometry by clicking "Deploy CrystFEL geometry" in the indexing panel.
Integration radii: These 3 numbers define the radius of two concentric rings about each Bragg spot. Inner ring is used to integrate the Bragg spot and the outer ring is used to estimate the background. Try adjusting these numbers and see what is being integrated on screen. It should be large enough to fit a Bragg spot inside the inner ring.
PDB: If you have a CrystFEL unitcell, you can constrain the indexing algorithms to look for this unit cell.
Indexing method: Default is mosflm-noretry, dirax. "retry" is used to speed up mosflm (it can take few seconds).
Tolerance: These 4 numbers define how much wriggle room you want for indexing. 5, 5, 5 are the tolerance level for unitcell axes a, b, c. 1.5 is the tolerance level for the angles alpha, beta, gamma.
Extra CrystFEL parameters: You can enter extra parameters for indexamajig in this field. It will be appended at the end of the command line, e.g. --profile will turn on the processing timing information.
Let's try to index another diffraction pattern at event 44.
- In the experiment parameters panel, set Event Number to 44.
- You should see the magenta triangle appear again. Wait few seconds and hopefully you will have indexed another pattern.
Hopefully, you have indexed this diffraction pattern. Notice that the unitcell parameters are a bit off compared to what is expected. Let's load a CrystFEL unitcell file to help the indexer along.
- In the Indexing panel, set the PDB field to: /reg/d/psdm/cxi/cxitut13/scratch/psocake/lys.cell
| Code Block | ||
|---|---|---|
| ||
CrystFEL unit cell file version 1.0
lattice_type = tetragonal
centering = P
unique_axis = c
a = 77.05 A
b = 77.05 A
c = 37.21 A
al = 90 deg
be = 90 deg
ga = 90 deg |
You should notice that the reindexed result will match the known unit cell parameters more closely.
Once you are happy with the detector geometry and indexing parameters, you can click "Launch indexing”.
- Run(s): 10
- Sample name: lysozyme
- Queue: psanaq
- CPUs: 24
- Keep CXI images: On
Indexing will take some time to complete. If successful, you should see a stream file in: /reg/d/psdm/cxi/cxitut13/scratch/<username>/psocake/r0010/cxitut13_0010.stream
You can check the status of your indexing job here: /reg/d/psdm/cxi/cxitut13/scratch/<username>/psocake/r0010/status_index.txt
Psocake saves the detector images of only the hits in the .cxi file. It is likely that you may want to reindex these files to optimize the indexing rate. If you anticipate that you have finalized the indexing parameters, set 'Keep CXI images' to Off. It will delete the detector images in your .cxi file which will free up your precious disk space for doing other things.
As with peak finding, you can launch indexing jobs on multiple runs by specifying runs in the Run(s) field.
Detector centering
Let's check whether your detector is well centered with respect to your beam. You want the centre to be as accurate as possible (at least to a pixel accuracy) for high indexing rates (>40%).
Load the powder rings generated by clicking the "Load image" button in the Image Control panel. Open "cxitut13_0010_maxHits.npy". Adjust the intensity as necessary.
Draw resolution rings by ticking "Resolution rings" in the Diffraction Geometry panel. You can change the ring resolution by typing number in "Resolution (pixels)". Type 165 and see whether your powder rings overlap with the resolution rings. If they do, then the detector is centered. If not, then you can click on the "Deploy automatically centred geometry" to recenter your detector. If you are unhappy with the results, you can use "Deploy manually centred geometry" which will shift the detector centre to the centre of the green ROI circle.
Once you are happy with the detector geometry and indexing parameters, you can click "Launch indexing”.
- Run(s): 10
- Sample name: lysozyme
- Queue: psanaq
- CPUs: 24
- Keep CXI images: On
Psocake saves the detector images of only the hits in the .cxi file. It is likely that you may want to reindex these files to optimize the indexing rate. If you anticipate that you have finalized the indexing parameters, set 'Keep CXI images' to Off. It will delete the detector images in your .cxi file which will free up your precious disk space for doing other things.
As with peak finding, you can launch indexing jobs on multiple runs by specifying runs in the Run(s) field.
Indexing will take some time to complete. If successful, you should see a stream file in: /reg/d/psdm/cxi/cxitut13/scratch/<username>/psocake/r0010/cxitut13_10.stream
Jumping to indexed images
Once you have completed indexing a run, you may want to look at the indexed events.
In the small data panel, type the CXIDB filename:
- filename: /reg/d/psdm/cxi/cxitut13/scratch/<username>/psocake/r0010/cxitut13_0010.cxi
- dataset: /entry_1/result_1/index
You can click on the red marker in the plot to jump to the corresponding events.
Once you are satisfied with indexing all your runs, please remember to backup your .cxi files in the /res directory of your experiment. The scratch folder will get wiped after few months (Data Retention Policy).
The stream file is small enough to transfer back to your institute for post-processing (CrystFEL tutorial) and phase retrieval (PHENIX).
For phase retrieval, you can use Phenix and CCP4 by sourcing the following lines:
| Code Block | ||
|---|---|---|
| ||
# Phenix source /reg/common/package/phenix/phenix-1.10.1-2155/phenix_env.sh # CCP4 source /reg/common/package/ccp4/ccp4-7.0/bin/ccp4.setup-sh |
Bug/Comments:
Please send bug reports/comments:
yoon82@slac.stanford.edu
Overview
Content Tools