Event log

Jul/06

• Caladium back to beam line, but chillier not yet on and g-2 platform post is blocking beam line to the dump.
• Cleaned up the remnants of g-2 from DUT rack. Mounted DUT rotation stage. Reconfigured the web cams and updated the confluence portal list.
• Queuing one BK Precision 9130 supply for survey to exit ESA for let Mike Dunning working on its RS232 interface for EPICS.  

Jul/07  

Operations team: Maurice Garcia-Sciveres, Katie Dunn, Su Dong

Maurice's CMOS sensor shipped to SLAC from Sensor Creator's Inc directly. A 'camera' like standalone readout unit connecting the sensor via a thin flex. Beamline support scheme worked out with Matt McCulloch with CMOS sensor taped behind the 94-04-03 double chip FE-I4 module. Vertical readout chain to allow easy rotation. Ethernet port address 192.168.2.1.

• Doug McCormick turned on chiller, moved aside g-2 stage and reordered web cams. 
• Alignment encountered several issues: coarse Y appeared to be not stopping and having strange pauses during moves. Coarse X is at hard limit and we are still ~2cm off. Fine XY panel scale factor didn't look right. Rotation goes scarily fast.  
• Caladium commissioning got stuck on NI crate not ping-able. Traced the network route for NI crate and DUT Sun server to table below. Early stage found NI crate power dropped - tracked to loose power cable at the back, but firm reconnect didn't get it back still. tried to swap NI crate's Belden channel to 22 but made no difference.

• Prep-prod HSIO-2 X1 moved into ESTB tunnel DUT rack. Network table needed update of HSIO-2 MAC address. 
• CMOS cam ethernet readout needs to find a host server. Sun server esadutdaq1 both ports occupied. 
   

Jul/08

Operations team: Maurice Garcia-Sciveres, Katie Dunn, Su Dong

• Doug McCormick and Juan Cruz moved the whole Caladium to the south (-x) by 25mm to center the telescope to beam and still in the middle X coarse range. Also adjusted the yaw.   
• Following recommendation from Matthias Wittgen and Martin Kocian to add a NetGear G105 router between Sun server and readout devices such as HSIO2 and the CMOS cam so that the single DUT readout ETH1 port of the Sun server can take in multiple DUT readout sources in a private network. To avoid the complication of changing the CMOS cam IP config, the RCE and Sunserver private network IPs are modified to create a common private network:

• Mike Dunning and Juan Cruz helped to debug the NI crate unreachable issue. Belden channel moved to 22 and again verified made no difference. Hooked up a monitor to NI crate discovered it got stuck in an unbooted state, presumably due to uncontrolled power downs. Followed its own long self-recovery procedure and eventually booted up normally and final became ping-able.    
• Matthias Wittgen offered a recipe to open a ssh tunnel from a counting room node to the CMOS cam via esadutdaq1 
  1. Window 1 on ar-esaux2
     ar-esaux2 > ssh -L8080:192.168.2.1:80 tbslac@esadutdaq1 
  2. Window 2 on ar-esaux2 open a web browser (firefox didn’t work for me, but konqueror did)
     ar-esaux2 > conqueror
     Navigate to  http://localhost:8080
  and installed ftp to esadutdaq1 to allow copying of data.
• Mike and Juan further debugged the XY stage scaling parameters to correct them to the right values. Movement steep on some on them also slowed down (especially the rotation). Mike showed how the advanced option panel can help control the movement rata and soft limits. However the problem of current position display lagging until next move is still there. Needs to use the advanced option panel to to manual refresh to check current position. 
• Upon retry of EUDAQ integration, RCE producer always aborts at start of run. Running calibGui reveals connection error and a suspicious of DUT private network IP address for the Sun server. Debugging with Martin Kocian uncovered that the environment variable ORBHOST for the host node esadutdaq1 used by RCE DAQ also needs update to follow the DUT private network change above. The default definition in ~/daq/rce/scripts/setup-gen3.sh is updated to point to 192.168.2.3. This brought whole EUDAQ to run with the SLAC_1dut_FE14B configuration (1 module both FEs). 
• Ready for beam !     
• Upon requesting to start beam MCC found out that the primary LCLS experiment at CXI was initially unwilling to share out the 5 Hz beam, partly because their overall beam delivery for the last day was rather rough. 1Hz setup appeared to be not available. However, CXI eventually agreed to release the lowest possible rate at ~3.5Hz so we had the first run. Whenever we are not running we should inform MCC to take away the beam to give back the 5%. 
• Run 269 is the first run with the LBNL CMOS taped at the back of the regular double-chip module 94-04-03, started out at perpendicular to beam. CMOS is untriggered. MCC didn't have dump signal to guide setup quality, but we could tell it was a rather wide spread spot and about 10 particles per MIMOSA per shot. For the last 1/4 of the run, the module is rotated by~45 degrees but not sure if still aligned within beam.
• The CMOS camera timing window can only open up to 1ms so that it was only 1-2% duty cycle. Got an 5V->3V BNC+resistor voltage divider from Mark Freytag to try to run triggered mode. Also got a 1M Ohm resistor to series a bias voltage LV drive. Installed both on the CMOS camera.
• Upon requesting to restart beam, MCC complained that some vacuum valve for ESA section didn't look right and wondered how we could operate like that. Indeed the beam spot looked very wide and nearly uniform over the whole MIMOSA area.However we were getting ~10 particles per beam crossing on each MIMOSA. This is actually OK for us so asked MCC not to bother tuning it.   
• NI producer repeatedly got stuck and no events were built while beam triggers were piping through at 5Hz. Eventually went back to start from the very beginning of NI setup to get everything going again.
• Run 277 with DUT at ~45 degrees and CMOS camera in triggered mode. Beam appeared to be hitting left X edge of the FE-I4 module. Moved the module to -X in 5mm steps several times during the run to confirm hits gradually moving to the center of DUT. Still not seeing hits on CMOS camera. Not clear if the TLU TRIG output was really getting into the camera trigger line.         
• Saved monitoring histograms to ~tfuser/eudaq/bin and archived at ~tfuser/Desktop/2016July_onlinemon/.
• Decided to retrench on the CMOS camera for now. Dismounted and packed up for survey exit ESA. Several things to prepare for retry next week:
  1. Activating another beam trigger line to allow independent trigger time delays while keeping pairing FE_I4 and Caladium data in time, or still using the same primary trigger line to make a duplicated copy at DUT rack and find a way to do 25ns step delays one of the derived triggers. The primary trigger must be before the actual beam crossing.   
  2. Verify triggering pulse electrical spec at LBNL bench tests
  3. Pulser and scope is ESA to examine trigger signals  
  4. Keithley to measure microA level current on bias supply ?

Jul/13

Operations team: Ben Nachman, Su Dong

• Lab cold tests obtained good tuning for both LUB1 and LUB2, even 1500e for LUB2, at HV=600V.
• Presented plan for 2nd period to MCC at 4:15pm meeting: ESTB_MCC_7_13_2016_SD.pdf
• MCC meeting had brief discussion of possible effect of ESA stealing 5Hz beam. Some effects of missing pulse causing irregular effects for pulse after that a Low E beam experiment, but high E beam experiments never cared. However, still needs to verify this for MFX/LM51 tomorrow, before 9am so that both dat and night shift beam to ESA would be possible hopefully.
• Lab testing pushed up HV. For LUB1, 1KV immediately trips the 1mA max current, while 900V still hanging there but current still slowly creeping up. For LUB2, 1KV started out at 700uA (-29C) but current slowly creeping up and after several hours it eventually tripped and temperature went up to -26C. 

Jul/14

Operations team: Ben Nachman, Su Dong

• Moved lab setup of cold box with LUB2 to ESA.
• The dry CO2 chamber is reversed from May to be on the corridor side while electrical connections from rack side.
• LV VDD=1.5V. I=0.33A after config.
• One temperature probe (Ts) taped to the Cu cooling tape tail from the DUT, the other temperature probe (Tg) in cold gas. 
• Leakage current high initially but temperature was just -10C. 2nd access raised HV to 800V leakage current 830uA  at Ts= -23C.
• After ensuring only ch=0 for RCE activated in readout, calibration ran, but threshold distribution is broad. Returned and got a quite decent RMS now. Saved to LUB2/config_rad_np_0_335: 1500e threshold, 8000e=10 for TOT.
• Made EUDAQ config SLAC_1dut_rad_FEI4A.conf
• Alignment was still tricky. Fine-Y adjustments not holding up the position. Any adjustment would trigger a long sag. Mike/Keith both thought the motor holding current may be too low. Will be looked at tomorrow before 9am by TestFac.
• Because DUT X/Y transposed, we cannot see correlations any more between MIMOSA and DUT as correlation only plots X vs 'X'.
• Horizontally we would like to move Fine-X towards +X (rack wall) by ~1cm to align with EUDET but that reached limit and cold box cable exit is touching Caladium plane frame.
• Requested access ~4:45pm. Put a ~2cm block spacer in the cold box to move DUT towards -x to recenter better in X wrt Caladium and space out the cable exit from Caladium. Also jacked up the DUT box by hand to be close to center in Y. Topped off dry ice.
• MCC notified us that LCLS MFX requested change of beam energy up to 14 GeV (but secondary always 11 GeV) which will affect our rate for a while. After reconfigure the beam spot shape changed completely with more definite edge and more uniform. Previous beam spot had a sharp hot spot of on the top right corner and more diffused spread below that.    
• 6:45pm first serious data run 294. HV=800V, leakage current 200uA. Ts=-34C, Tg=-29C. ~350 hits/MIMOSA, 150 hits/evt in APIX. DUT beam window: Local X=40-75 which is 35 pixel ~8.5mm; Local Y=30-330 which is 300 pixels ~15mm. MIMOSA beam window X=200-1000 also ~15mm; Y=20-460, also 8mm.  
• Noticed some 'scars' on the DUT which are straps of area with clearly lower hit density than neighboring areas.
• Ended a good run 294 to request lower rate. MCC had no clear recipe to reduce rate other than collimating to smaller beam size which doesn't really help us as it is the hit density that matters. They are trying to reach Toni Smith for advise. Ryan tried to tune this from upstream adjustments and succeeded to bring the rate down to 30 hits/shot at DUT with relatively small change in beam spot shape. We needed to make a few mm adjustments to keep both MIMOSA and DUT beam window fully contained.
• 8:34pm 2nd run 299 with DUT hit rate ~30/frame. Temperature climbing back up Ts=-32C, Tg=-27C. Leakage current 250uA. MIMOSA beam window X=150-700 (10mm) and Y=full span 0-570 and perhaps a bit more (11mm). DUT window 'X'=30-74 (11mm), 'Y'=40-260 (11mm) a bit trapezoidal.   
• 9:06pm started a new run 300 when last run reached 10K event limit. Exactly same condition. Ended 21:42.   
        
  

• Requested access to change DUT tilt. Had to pull out the block to tighten the rotation screw and retape the DUT Ts temperature probe. Cold box stage sagged again which needed manual repositioning. Topped off dry ice. 
• Upon restarting to find the location of the DUT from remote motors, the whole DUT stage sagged to Ymin again. Only saw a lot of out of time noise hits at local X min and max edges (this was probably due to temperature not yet low enough).
• Turned off beam to do a low TOT tune of 1500e threshold and 400e per TOT bit. Calib tuning kept crashing. 600e still crashing, while 800e existing tuning still worked. 
• Gaming with Y-fine stage to hold up the cold box at ~5mm below nominal (any further up push will cause full scale sag down). Cold box lid is opening a significant gap.
• Now a bit colder, returned to previous configuration and brought beam back. Noise lower. Moved DUT fine-X by -6mm, starting to see streaks. Data populating lower 'X' half of the DUT but did not dare to jack up the cold box.
• 11:39pm started run 303 for the first streak data. They looked a bit shorter than expected typically. DUT data rate similar to MIMOSA at ~80 pixel/frame, but there are many streak clusters. Pixel TOT peak at ~5 (5*800e=4000e). This is a bit larger than expected from Bichsel calculation that predicted peak at ~3300e for 50micron silicon even without radiation damage ( foldspec_elec_11GeV_50mu.png ). 
• 00:20am ended run. Let MCC took away beam. Remote power down of HV+LV.