Versions Compared

Old Version 5

changes.mady.by.user Christopher O'Grady

Saved on

compared with

New Version 6

changes.mady.by.user Christopher O'Grady

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • each jungfrau runs linux (blackfin?)
  • unconfigured current draw is 0.8A, configured current draw is 3A for a 0.5M
  • evr trigger goes into "trigger in" lemo input on back of camera
  • ethernet goes to CDS subnet in DAQ lab (a couple of small switches lying around have this subnet)
  • has both software and firmware
  • ethernet goes to CPU
  • fiber goes to FPGA
  • data streams out of FPGA via UDP (we need jumbo frames, but not pause frames, might need other ethtools settings, we think intel/myricom nic settings are the good at the defaults)
  • the FEZ interface that receives the EVR multicasts (and transmits the bulk data to the DSS nodes) needs interrupt coalescing (may depend on the nic).  maybe 75us for myricom and 50us for the intel nic?  the coalescing matters on nodes reading out large detectors, or large numbers of small detectors
  • jungfrau segment level
    • event builds multiple modules 
    • stitches together 8 asics from each 0.5M in a natural order
  • sometimes need to update software/firmware
  • can update firmware via CPU, but if that fails have to jtag, but it's altera.  altera is installed on machine daq-det-standalone (had to do this more with the early jungfrau's)
  • each 0.5M silicon has a serial number that can't be read out via the DAQ
    • these serial numbers are used by mikhail to lookup calibration constants, but must be managed administratively
  • use the fiber farthest away from the ethernet
    • most module fibers are "green" when locked, but one of them is red unfortunately
    • in daq lab fiber goes to daq-det-jungfrau (also used for epix's)
  • second fiber is used for high-rate bonding which isn't supported yet
  • on det-daq-jungfrau machine fez is 172.21.59.53
  • the serial number of the module (used by Mikhail's calibration lookup) is the mac address of the module (Dan reads these from the modules)
  • the detector fiber is hooked up to one of the other interfaces
    • use ethtool to look for link locked (ethtool enp5s0)
  • standalone executable to read out jungfrau's (same code as segment level)
    • see also Running DAQ devices standalone 
    • want to run as detopr because executable creates temporary files then there can be permissions problems cleaning them up to allow a future instance to run
    • executables live in /reg/g/pcds/dist/pds/current/build/pdsapp/bin (no env setup necessary)
    • /reg/g/pcds/dist/pds/current/build/pdsapp/bin/x86_64-rhel7-opt/jungfrauStandAlone -P 32410 -H 10.1.1.105 -m 00:60:dd:45:66:df -d 10.1.1.55 -s det-jungfrau-31 (use -h for help)
      • -P is port number on the detector side
      • -H is the host IP address of the fiber interface
      • -m is the mac of the host fiber interface
      • -d the ip address of the detector
      • -s hostname of the detector control interface (what's in netconfig)
    • look at cnf file to get params: "grep jungfrau-31 /cds/group/pcds/dist/pds/det/scripts/det.cnf"
    • if this works, camera is working. daq config can be messed up.
    • pds/jungfrau/DataFormat.hh shows the structure of the UDP packet.  framenumber is trigger number, and packet number goes from 0-127.

Generate one frame in standalone (some free-running trigger):

...

  • If no jungfrau triggers seen (or intermittent) and using a rhode-schwarz supply (used for 0.5M and 1M, but not 4M) often need to hook up an extra "chassis ground" 

4M Idiosyncracies

  • numbers shown for each 0.5M above are the serial numbers
  • read out on two nodes because too much data for one (daq-cxi-jungfrau01 and 02)
  • 40Gb nic in each machine, setup at 4 10Gb interfaces.  MPO cables get broken out very near the detector into LC.
  • 4 segment level processes (2 per node, one per quad) to allow more cpu parallelization
    • "CxiDs1/0/Jungfrau/0" -S 0,2,8  flags in .cnf show the "parent" detector id and the modules of this on (-S 0,2 means 0,1 and -S 2,2 means 2,3)
    • intercepted in the DSS nodes which puts together the 4 pieces into one CxiDs1/0/Jungfrau/0 detector in the final .cnf
    • pdsapp/tools/JungfrauSegBuilder.cc does this.  Included in Recorder.cc.  does something on both configure and l1accept.  FrameCacheIter is holding pieces while before they are memcopied onto the end