ESTB 2016/Sep-Oct T545 + T539

Participants: 

LBNL: Katie Dunne, Timon Heim, Peilian Liu, Ben Nachman, Maurice Garcia Sciveres
SLAC: Nicole Hartman, Peter Kim, Martin Kocian, Rob Mina, Francesco Rubbo, Su Dong
ANL: Sergei Chekanov
KEK: Koji Nakamura 

Shift Schedule (tentative) 

Event Log

Sep/19-26 prep      

Sep/19: Previous dissembled Caladium is reassembled by Test Facility in the ESTB tunnel but parked on the side as beam pipe was reinstalled for a down stream experiment (somewhat of a surprise for us) this week. Caladium mover control electronics is still missing and to be installed next week. We checked the re-connections and found a few errors and missing links which got corrected. Connected the double chip module 94-04-03 as DUT with upgraded DUT CAT6 data cables and tried pixel digital calibration which worked as expected. Reordered the power supplies on the remote power strip to match the original panel in EPICS.     

Sep/20: Checking with Carsten about what is really going on this week, it turned out T-542 (sFlash) is taking the entire period Sep/22-Sep/26 so that the originally agreed Sep/26 for us has evaporated and we were never informed of that change. 

Sep/23: Checked with sFlash at ESA control room. The intense scene confirmed the fear that they will probably need the entire period, including Monday.

Sep/26: sFlash is continuing with their runs so that we are unlikely to have any chance to run for Monday, considering that Caladium still needs to be brought onto beam line after beam pipe removal. So Monday is expected to mainly doing lab test of the first RTI module to get proper configurations prepared and check other preparation items.   

Sep/27

• 8AM MCC meeting mentioned ESA beam delivery problem for sFlash on Monday but eventually took very good beam data. sFlash cleanup.
• 10AM ESA became permitted access. 
• Caladium moved onto beamline after removing section of beampipe and mover electronics box reinstalled. Chiller level low, refilled. DUT stage also back on and hold current is nominally driven. Caladium powered up current anomaly traced to Clock-Jtag cables swapped. Missing TLU-NI USB cable located and reconnected. Caladium auto-trigger run and standard DUT (mounted 93-04-03 IBL double chip module back) both operating.        
• Noticed IBL double-chip DUT LV PSU setting sagged from 1.5V->1.2V when module current went up to 0.5A. Realized this was due to the current limit of 0.5A appropriate for single chip modules. Raised current limit to 0.8A stablized voltage through threshold tuning exercise.    
• FE65_P2 readout arrived from LBNL. Mike Dunning/Juan Cruz laid two DUT ethernet lines for the two readout PCs to connect to the ESA 104 private subnet. Mike granted fixed IP addresses: 172.27.104.52 esadutlbnl1 (riverhorse) and 172.27.104.53 esadutlbnl2 (inkfish), gateway is 172.27.104.1 and mask 255.255.252.0.   
• Various cameras setup - see main portal.

Sep/28

Plan for today:

• Priority is to debug FE65_P2 readout integration with telescope (Timon, Ben) and taking telescope whenever needed,
• Install the 2nd RTI SC module into the ESA DUT seat to do in situ threshold tuning (Sergei,SD).
• Threshold tuning for the first RTI SC module at group C lab (SD,Francesco,Rob,Nicole)
• HV remote control/mon with EPICS (Ben,SD with help from Mike)
• possible telescope geometry optimization (SD)
• Presentation [slides] of our plan and beam requirement to MCC (SD) at the 4:15pm accelerator weekly planning meeting next to the accelerator control room at the ground floor of B52. Everyone is welcome.      
• Discuss preparation status, run plan and shift assignment at ESA control room at 5pm. 

Activities:

• RTISC2B replaced the 93-04-03 as the RCE DUT to get threshold tuning going for it.
• When attempting to move the DUT closer to beamline we finally realized the oddity of the vertical position was due to the DUT stage mounted upside down. Juan corrected the mounting and now position close to be central.
• Sergei installed a script duplicate.sh in the moduleconfig directory on esadutdaq1 so that one clone an existing module's RCE DUT config directory into a new module's starting configuration with underlying sud-directory file name to follow the new module name.    

• The Research Yard construction people are unhappy about us rushing across the area in speed so that we are currently disallowed to go to ESA from the Research Yard ground floor through Gate 17/SSRL route. Everyone should go to ESA from Gate 30 to the top level counting room then down the stairs to ESA hall. If we have to drive in with equipment, we can give Keith Jobe a notice to arrange the passage via Research Yard. 

• Timon exercised first integration of YARR readout infrastructure with EUDAQ: YARR Producer correctly booted up in EUDAQ Run Control. Correctly receiving configure transition and acknowledge completion while actual configuration content and event loop handshake still in progress.     

• Re-orientated the RTI module by mounting inside the L to give some space for the HV LEMO bend not touching EUDET. The HV socket facing perpendicular to the PCB is not test beam friendly. TImon mounted the FE65_P2 nad looks like RTI and FE65_P2 are reasonably aligned in height and horizontal position to go together. Just some DUT stage fine shifts should bring the DUT to beam.

• Segei's threshold tuning for RTISC2B is giving high noise at 500e. Realized we didn't tape the sensor window. Proper blacktake hard to find. Kluged some thicker tape for now. Simple threshold scan didn't get sensible results. To be tried again tomorrow.   

Sep/29

Plan of the day:

• ESA search for returning to controlled access is either ~9am or ~11am. First beam by noon ? 
• Preparation the RTISC2B + FE65_P2 pair DUT suite for first beam: 
  • DUT stage fine adjustment to align DUT into beamline and horizontal move of telescope into beam. Visually locate FE65_P2 window wrt RTI. 
  • Adjust overall EUDET planes to pack closer and narrow the DUT region. Needs an LEMO-L for the RTI HV socket to avoid HV cable bowing out to EUDET. 
  • Proper black tape for RTI sensor window and recheck threshold tuning with taped window.
  • Further auto-trigger integration of YARR with EUDET.
  • EUDAQ config file for the DUT suite 
• Continued threshold tuning for RTISC4A in the lab to get ready to go in later, and cross check on noise effects seen on RTISC2B.   
• First beam activities
  • Time-in of FE65_P2 devices if DAQ event handshake fully functional. If no new device interpreter with EUDAQ online mon, offline dumps sufficient to verify ? Hopefully the known lab test delta of 3 clk ticks with FE-I4 will save us from a long chase. 
  • Beam window size, location and rate adjustment.  

Activities:

• 8am MCC meeting clarified that the ESA search is probably going to be 11am-noon, not the originally hoped 9:30am. We declared ready to run right after ESA search. It was brought up that a few weeks ago some ESA user from Japan brought in an irradiated sample without declaring to RP which caused quite some disturbance that even temporarily shutdown the ESA operations. Lia's question on this caused a major overhaul that required us to send updated participant list, and TestFac experiment review form and RP in person interview etc. before getting approval to start at ~2pm.
• 13:00 - ESA search has been completed and the cave is now set to controlled access.
• DUT stage fine adjustments brought DUT aligned with Caladium MIMOSAs, and whole telescope moved horizontally into the beam line. 
• RTISC2B appears have gone into very strange state and cannot be calibrated even after removing the rough taping. Brought in RTISC4A to replace it and sensor window properly taped. Threshold calibration runs but the noise rather high at ~400e, with some systematic structure also. Suspected bias voltage and seems to measure 0V on the HV ch-0 used. Switched over to ch-1 can at least drive the voltage up. Maurice concerned that the leakage current seemed a bit high >>few uA for very small starting bias voltage. However, cranking up to -53V (these Micron 300um sensors actually need -80V to deplete) didn't bring much more leakage current to settle at 4uA. Left at this setting to try to run. Sergei managed a standard 3000e tune with nice narrow dispersion. The noise is still high at ~400e. Also checked the various grounds and decided we can leave the HSIO2-PSU ground hook off since the HSIO-FE cable is now proper CAT6 with ground. This didn't make any difference. Decided to just take this tune as running config (RTISC2B__cern__25.cfg) - calib directory stayed with RTISC2B but this is really RTISC4A.    
• Tried EUDET ni_autotrig runs but somewhat confused by the behavior: without starting up the TLU producer, the autotrig setup actually runs with event build flowing and histograms in monitoring updating, although trigger/particle counts remain 0. When added the TLU producer, particle counts still go up but no trigger and no event build.  
• 18:05: Carsten called and informed us about the delay. The people before us wrapped Al foil around the emergency buttons which of course upset a lot of people. They stopped ESTA operation until it is resolved, which should be in the next hour.
• Beam finally arrived ! We don't seem to get any trigger. Decided to ask for access to look at the trigger and change RTI sensor HV control at the same time. One problem with trigger quickly found: the Caladium Date Reducer box data link to TLU was misplaced on  the Data Reducer box side to the port right next to the 6 MIMOSA data cables, but it really should be plugged into the channel at the far end. Fixing this resolved the TLU producer anomaly above.   
• The remote control to the HV in the NIM crate (the orange one in the middle) now works via the KB 9130 power supply.  There is a panel in the epics for T-542 for controlling the low voltage.  The granularity is pretty good - by setting 0.08 V (=80 V in HV), we get 0.086 measured (and displayed on the monitor).  The only problem is that the current is in A so we can't monitor it (need to change to uA).  Should ask Mike tomorrow during the day for help with the quick fix.
• After we moved HV control and set voltage -53 > -86V volt, Sergei rechecked the RTI sensor threshold scan noise and found noise reduced from ~400e to ~300e. This is consistent with the Micron sensor depletion voltage of 70 - 80V. 
• Upon restart with beam, we still don't get the beam triggers, although auto trigger runs with TLU now works correctly.  
• 21:19: Accelerator shift change, beam energy changed and was off for 20min, back now. Will take access to investigate trigger problem.
• ESA access again to examine the beam trigger. Traced the trigger cable from the DUT rack to the patch panel area and placed a yellow tag on it. It is connected to the row of BNC plugs saying "To B061-01-06" socket J5. Brought scope to the DUT rack and placed a BNC T on the trigger cable to check it on the scope by moving over the rack top camera.   Upon restart with beam, we saw from the camera that there are blips of pulses at ~5 Hz as expected and negative polarity as expected from NIM but the pulse height is much lower than the -1V NIM. We though the trigger signal might be available at the counting house but the "beam spill" plug had nothing on it. Toggling the trigger panel channel #10 SiTrack- Caladium Enable on/off one can see the 5 Hz blip goes on/off together with it which suggests the trigger is still hooked up to the right channel but too low an amplitude for the TLU box.  Couldn't make much progress without Mike. 
• Decided to quit for the night and reconvene tomorrow when Mike Dunning is around. 
• New calibration is RTISC2B/config/RTISC2B__cern__93.cfg for 89V shift volts.  The main features of this calibration: Threshold distribution has mean 2988e, with sigma 68e. The noise is 288 e.

Sep 30

• Some description and tutorials how to take the calibration scans is https://atlaswww.hep.anl.gov/asc/wikidoc/doku.php?id=itk:testbeam. All figures corresponds to the previous scan on Sep 29 ( RTISC2B/config/RTISC2B__cern__93.cfg)
• 5.00: New calibration file for RTI module is RTISC2B/config/RTISC2B__cern__158.cfg.  It was noticed that the noise level increased to 400e, and the  analog scan has more loses on the left edge.