Troubleshooting common issues with DAQ
Issue | Solution/Workaround |
---|---|
DAQ not saving scalar data (BSA and non BSA) and getting F2_EventClass error | This is caused by a bad PV in one of the lists |
DAQs returning no matches and UniqueId_RBV PV not updating + fiducial rate stuck and/or red | Reboot the server |
DAQ (1D or 2D) fails and shows the following error: Error using repelem http://physics-elog.slac.stanford.edu/facetelog/show.jsp?dir=/2024/24/15.06&pos=2024-06-15T21:36:00 | This happens when a camera trigger is not set up correctly A workaround is to find a camera that is on the same trigger and add it to your DAQ Ask Sharon for help if this isn't clear |
Bugs and Robustness
Add bugs here, newest entries on top:
Live planning notes
Goal: Update DAQ to HDF5
- What has already been done?
- We have a working version of the HDF5 DAQ, and now we are working on making it robust
- Sharon has shown that HDF5 file writing in "Streaming" mode is very fast.
- Now we want to extend capability for scans so that we have 1 file per camera.
- How to save scan image data into HDF5 file?
- Use PVs:
- extraDimSizeN = Number of shots per step
- nExtraDims = Number of scan dimensions (0 = simple DAQ, 1 = 1D scan, 2 = 2D scan . . .)
- extraDimSizeX = Number of steps for Dim 1
- extraDimSizeY = Number of steps for Dim 2
- extraDimSize3 = Number of steps for Dim 3
- . . .
- These PVs implicitly control the NumCapture PV
- NumCapture_RBV = extraDimSizeN * extraDimSizeX * extraDimSizeY * . . . (depending on number of nExtraDims)
- Use PVs:
- In this scheme, we have to carefully control the triggers so that each scan step gets exactly the right number
- We will use a new feature to do this:
- EVNT:SYS1:1:PMAQCTRL.H = extraDimSizeN = Number of shots per step
- EVNT:SYS1:1:PMAQCTRL.E = 0/1 to stop/start triggers to cameras
- Need to toggle back to 0 after every step
- Set EVNT:SYS1:1:PMAQCTRL.H back to zero at end of scan
- We will use a new feature to do this:
- What happens if the camera does not receive all triggers?
- Then scan steps are kind of messed up . . . think about how best to handle this with pulse ID
- Prefer "NDArrayUniqueId" to be "NDArrayPulseId"
- Make caput/lcaPut/lcaPutSmart consistent
- "BSA buffer" for UV spectrometer IOC?
- Find a reliable fix to shot counting issue
- checkShots is not counting the correct number of shots at some steps. we think the DAQ is just not waiting long enough for the shot number to go up, because the shots do get saved eventually
- for E332 shift on 5/30, we just added a while/pause loop to force DAQ to wait for shot numbers to go up
- if we decide to keep this functionality we should at least add a break for the while loop but for now it is working again
DAQ tests E320 shift 4/30/24
Testing TIFF/HDF5 switch
- take a DAQ with TIFF and try to load it in DAN
- results look normal
Testing HDF5 DAQ
- take a DAQ with HDF5, simple DAQ, 20 shots, 1 camera (LBG_LFOV)
- all 20 shots saved
- test with 1000 shots
- saved 999 shots
- took a little over 2 minutes
- add more cameras (LBG_LFOV, EOS2, GAMMA1)
- saved 993 shots for each camera
- ~2 min 10 seconds in total
- try streaming
- 1000 shots
- failed TEST 7480
- try streaming but with 10 shots
- success with all matches
- try streaming with 200 shots
- success with 197 matches
- try streaming with 700 shots
- says DAQ failed on step 1 but 641 matches
- try streaming with TIFF, 700 shots
- saved 155 matches
- took a little more than 3 minutes
- try a 1D scan, 10 shots, 3 steps
- did not restore to initial value
- Spec_Quad_E
- 28 matches
- trying a 2D scan, 10 x 2 x 2
- saved 40 shots
- trying 2D scan with streaming
- saved 40 shots
- try to open HDF5 images in DAN
- error when "Plot Correlation" button pushed, expecting a TIFF image
- Error in hlpGetImage with “imgType”
Bugs and Robustness
Issue | Person | Priority | Status | Notes |
---|---|---|---|---|
Load/clear config buttons don't work | Low | |||
Warning: Unable to save App Designer app object. Save not supported for matlab.apps.AppBase objects. | Sharon | Low | Seems to be harmless |
DAQ crashes and takes down the whole SIOC when total number of shots exceeds 2200 (no streaming, try with streaming on) | Reproduced this with PM03 Has to be more than 5 cameras | |||
Background not being saved | Sharon | High | DONE |
Implement HDF5 file saving for images | Sharon | High |
DONE | ||||
N/A | ||||
Calculate how much space on disk is required | High | |||
Install 10G switch and NAS in S10 | Arjun | High | Planning | |
Check what happens if new DAQ is started before old one is finished (failed abort) | High | |||
Fix binning and ROI for CMOS cameras | Medium | |||
Can't get magnet values on first DAQ? | Medium | |||
Check that the beam rate is not equal zero if rate set to "BEAM" | Spencer | Medium | ||
Fix "Use_PV" functionality. | Sharon | Medium | DONE | |
Fix camera triggers if DAQ fails. | Sharon | High | DONE | |
"Fix cameras" button disabled if DAQ hasnt run yet | Sharon | Medium | DONE | |
"Fix cameras" button should use default values? - yes | Sharon | Medium | DONE | |
Timestamping fails on certain ROI settings? | Spencer | High | DONE | |
Upgrade all machines in S20 to 10G connection | Spencer | Medium | DONE | |
Camera binning messes up pulse ID finding | Spencer | Medium | DONE |
Feature requests
Issue | Person | Priority | Status |
---|---|---|---|
Add check/remove bad PVs and notify user in an obvious way | Sharon | ||
Add nonBSA arrays so we can get UV spectrometer (Claudio request) | Sharon | High | DONE |
DAQ Monitor GUI | Sharon | High | DONE |
Find a way to globally trigger a modifier bit (e.g. "DUMP_2_9"). | David | ||
Scan function that pauses until you click 'ok' | |||
Generate "camera config" file including trigger info and add a save/restore button. Monitor if this finished ok or not. | Sharon | ||
Load/Save Config files. Default config files. | Claire | ||
Robustness issues. . . | Spencer | High | In progress |
DAN features | Sharon | ||
Documentation/commenting | Spencer | High | In progress |
Fix "Use_PV" functionality. | Sharon | Medium | DONE |
...