In most cases, the producer file should only be modified between the two flags:
########################################################## ## ## User Input start --> ## ########################################################## ... ########################################################## ## ## <-- User Input end ## ##########################################################
This user section is split in two parts, one for run-dependent parameters, usually mainly involving the more complex area detector analysis, which is likely to vary from run to run, and a run-independent part, which allows for the addition of EPICS PVs or analog inputs/outputs for example.
Run-independant variables
This will describe how to tweak some of the default data:
This is done mostly in the block in the producer file below (<smalldata_tools>/producers/smd_producer.py).
########################################################## # run independent parameters ########################################################## #aliases for experiment specific PVs go here epicsPV = ['s1h_w'] #tt calibration parameters (if passing None, init will use values used during recording ttCalibPars = None # (or [] or [p0, p1, p2] #aioParams=[[1],['laser'],[1.],[0.]] aioParams=[]
EPICS PVs
Should you have user motors e.g. for sample motion for which you would like to save the position or if you are using a lakeshore for temperature control or have other remotely controlled data you would like to save, you will need to add those to the epicsPV line. For the exact names, ask the beam line staff. Please note that 's1h_w' is only an example/placeholder. You can get the list of possible EPICS variables names/aliases by using
"detnames exp=<expname>:run=<run#> -e"
This will only work when you have the analysis environment setup by calling:
source /reg/g/psdm/etc/psconda.sh
Please note that these PVs will appear in event-by-event, but they are NOT time matched. The time values are close, but can differ by up to second, usually it's better than that, but it is not reliable.
Timetool calibration parameters
If we take a tt calibration run, we obtain a set of parameters for the conversion of the peak position on the timetool in pixel to ps. This should then be used in the DAQ and nothing needs to be done. Should you realize that this steps has been missed for at least some of the data or have you extracted better parameters, you can put the obtained parameters into the ttCalibPars parameter. They will be used for any new productions (e.g. for reprocessing of runs taken before the calibration was in place). If it is set to "None" or an empty list, the calibration saved in the data is used.
In some hutches (XPP), this can be ignored, as the calibration constants are already implemented upstream and the ps time-tool values are reliable.
Analog Input
Values read in from our analog inputs are timestamped at 120Hz and saved in the data using the channel name. You can select channels to be saved, give them an alias and possibly apply a (linear) conversion.
aioParams=[[1],['laser'],[1.],[0.]]
First argument (necessary if using): list of channel numbers of interest (from 0-15)
Second: argument (necessary if using): list of channel names
Third: argument (optional, defaults to 1.): conversion factor
Fourth: argument (optional, defaults to 0.): offset
Addition of the time tool or Wave8 traces
########################################################## ## User Input start --> ########################################################## ##adding raw timetool traces: #defaultDets.append(ttRawDetector(env=ds.env())) ##adding wave8 traces: #defaultDets.append(wave8Detector('Wave8WF')) ########################################################## ## <-- User Input end ##########################################################
Time tool analysis
XTCAV values
Run-dependent variables: Area detectors analysis
In order to keep the filesizes small to avoid issue when analysis the smallData files, we try to extract the important information gleaned from areaDetectors at an event-by-event basis and only save these pieces of data. Implementation examples of each of the analysis functions that are readily available can be found in <smalldata_tools>/producers/smalldata_producer_template.py
.
Please reach out to your data and controls POC to discuss analysis needs for your experiments. The POC will make sure the functions you need are prepared in the main producer file, and you should only have to change the argument's values, such as region of interest or thresholds, according the ongoing experimental conditions.
Generally speaking, the parameters for each analysis functions are set from a run-dependent logic as shown here:
def getFuncParam(run): """ """ if isinstance(run,str): run=int(run) ret_dict = {} if run<21: func_dict = {} func_dict['param1'] = <param> func_dict['param2'] = <param> func_dict['param3'] = <param> ret_dict['detname'] = func_dict else: func_dict = {} func_dict['param1'] = <param> func_dict['param2'] = <param> func_dict['param3'] = <param> ret_dict['detname'] = func_dict return ret_dict
Several analysis functions or detectors can be set up this way.
An example: Regions of interest (ROIs)
In the smd_producer.py, the function to define the ROIs goes as follow:
def getROIs(run): """ Set parameter for ROI analysis. Set writeArea to True to write the full ROI in the h5 file. See roi_rebin.py for more info """ if isinstance(run,str): run=int(run) ret_dict = {} if run<21: roi_dict = {} roi_dict['ROIs'] = [ [[1,2], [127,394], [655,923]]] # can define more than one ROI roi_dict['writeArea'] = True roi_dict['thresADU'] = None ret_dict['jungfrau1M'] = roi_dict elif run>20 and run<43: roi_dict = {} roi_dict['ROIs'] = [chip22] # can define more than one ROI roi_dict['writeArea'] = True roi_dict['thresADU'] = None ret_dict['jungfrau1M'] = roi_dict elif run>42: roi_dict = {} roi_dict['ROIs'] = [chip22] # can define more than one ROI roi_dict['writeArea'] = True roi_dict['thresADU'] = None ret_dict['jungfrau1M'] = roi_dict return ret_dict
Besides, the ROI definition, the parameters are the following:
writeArea
: whether to write the full ROI to file or only the statistics (intensity sum, and other statistics)
thresADU
: pixel intensity threshold. Any pixel below that value are set to 0
Use of SmallDataAna_psana to select a ROI
For the use of the interactive features of SmallDataAna_psana, we recommend to start it in an ipython session as interactive grabbing of user input is currently not implemented via the notebook. The first step is to create an average image that shows the feature of interest well.
From from smalldata_tools top level directory, run the following command to launch an IPython shell with pre-loaded a data analysis class for the relevant run
./producers/runSmallDataAna -r <#> [-e <expname>]
The average image can be then made by running
SDAna In: anaps.AvImage()
This will by default create an average image of 100 events of an area detector. The method will list and prompt for a detector name if more than one was recorded.
The full command with all its options can be looked via using the "?" feature of python:
SDAna In: anaps.AvImage?
More details on the interactive IPython capabilities can be found in the section Working with Images from camera/areadetectors
A ROI selection utility can then be used:
SDAna In: anaps.SelectRegion()
This will by draw the image and let you select a rectangular area of interest with the mouse. At this point, you click on two corners of the ROI. Once you are happy with your selection, it will print the numbers you need to enter in the producer file. Below both what the terminal session looks like (here with an optional selection of different plot limits, by default 5% and 99.5% are used). Figure 2 will come up when you get asked if you are happy with the selection which appears on the right side of that figure. "n" means you can to click again until you are satisfied. "y" will print the ROI as it should be entered in the smd_producer file.
A new figure will also show the ROI again, this time obtained from the raw data rather than the image (a technical detail). The more striking different is that the scaling/limit are now taken from only the ROI rather than the whole image.
SDana In [3]: anaps.SelectRegion(limits=[5,99.8]) plot AvImg_cs140_rob using the 5/99.8 percentiles as plot min/max: (1603.9, 1994.84) Happy with this selection: y ROI: [[1,2], [55,76], [308,335]]
More details on the different analysis function can be found under 1.2 Area Detector treatment with DetObject.