Jan. 22, 2021

with Alex R., Alex W., Zach L., Jana T., Chris F., Matt W., Silke N., Chris O.

Discussion

georgi: 100eV photon scan range over 100 seconds. 1eV/s. few microeV change per shot at 1MHz. need precision 100 microev? need to take out shot intensity (normalization)

Alex R. says encoder has 100 microev resolution, scan even faster than 1eV/s required.

Corresponds to 10kHz measurements.

silke says that old lcls1 can't be reused. old usdusb one: relative encoder (quadrature pulses) new one: absolute encoder

alex says: new encoder can only read out at 10kHz, maybe 20kHz.

zach says: add a second encoder? relative quadrature encoders faster, maybe a MHz?

silke says: "usb4" relative encoder reader can go up to 5MHz, but maybe not enough space in the vacuum environment. alex found another renishaw that goes up to 15MHz.

zach asks: why not put it on the external shaft? georgi replies not straightforward to use that as the position of the grating. zach/Alex say it could be calibrated

cpo says: interpolate? georgi says the turnaround point would break interpolation. cpo says: mechanical motor turnaround time either fast or slow on timescale of 10kHz sampling should work. in particular, turnaround time feels likely much slower than 10kHz sampling period?

alex w.: could add time-staggered 10kHz encoders to artificially increase rate

georgi says: need to understand the upper limits. interpolation may be too short-sighted. fast encoder is best.

alex wallace renishaw: https://www.renishaw.com/en/tonic-uhv-incremental-encoder-system-with-rslm20-linear-scale--11155

ankush encoder document: https://docs.google.com/document/d/1Ho1IpT4903wkUoJMV06z5hou9to5CfAgtJBDPdBl6I4/edit?ts=600b806d

→ (Silke) this lists the USB version of the encoder we originally used the PCI version for. We now have a box as Ankush laid out, except that it 'splits' the signal, so has in&out for each encoder channel. This device its listed towards the end of Supported Control & DAQ Devices

Matt says we may have an existing board that could do the readout/timestamping

Summary of Options/Information

10kHz with interpolation?
multiple encoders?
- combine absolute/relative (may be complexity issues)
- 2 staggered slower encoders
need fpga timestamping
100 microev resolution requirement

	Option	max rate	work to be done	drawbacks/other consideration
A	absolute encoder used for relevant stage	10-20(?) kHz	controls: rework PLC to send data (current solution likely ~1kHz) DAQ: likely FPGA timestamping
B	absolute encoder used for relevant stage, interleave with second absolute encoder	20-40 kHz	as above	need to ensure both encoders read back equivalent data, intersperse jitter from both?, same latency
C	add a relative quadrature encoder on shaft	5 or more MHz	DAQ: need board to read signal & timestamp	complicated and possible not very reproducible mapping of measured & relevant motion
D	add a relative quadrature encoder in vacuum	5 or more MHz	DAQ: need board to read signal & timestamp engineering: figure out how to add relative encoder

To do

fill out the table options and costs (alex and silke)
talk to mechanical engineers (Georgi)
meet in two weeks

Feb. 5, 2021

Alex R., Georgi D., Chris F., Jana T., Matt W., Zach L., Joe R., Chris O.

Delay-stage scan discussion:

It was decided that for now RIX will not do on-the-fly delay stage scans, instead using standard DAQ step scans of an electronic delay (which incur some dead time when the delay is changed)
Georgi reserves the right to revisit this decision in the future

MHz encoder discussion:

Feels like option D (see above) is doable according to conversation between Georgi and Danny Morton
- they meets with Axilon next week to verify (manufacturer of mono assembly))
Use this as our current default solution, but can still change (final decision in weeks)
Matt W. and Ryan H. think the camlink converter box could be used for readout
- quadrature encoder "frame format" is understood and relatively simple (absolute encoders are more difficult)
- quadrature encoder readout can be triggered
- might need either a new camlink converter board with new connectors or some "connector transition"
currently no use case for high rate absolute encoders

120Hz Encoder Plans

- plc is in the fee alcove and the encoder itself is in the fee

- (decision) we put a dedicated daq evr in the controls plc node or controls evr node or drp/ctl node?
- evr gets driven by xpm fiber so trigger stops on endrun
- ttl cable to plc
- both evr and plc use BNC connectors
- drp executable joins udp packet to timestamp from kcu (reuses TDetSim firmware). It is new UdpEncoder, which inherits from XpmDetector (Detector Class Diagram)
- (decision) question: do we have udp direct connection or go through the router? router is easier, but could drop the first packet and be off-by-one (can’t detect off by one with dropped shots). direct? cpo feels we should avoid the router.
- “direct” may mean a switch where we control the traffic? (ATCA (8 sfp, 8 rj45) or top-of-rack with a new vlan?) could use it for xtcav as well.
- ready "early march”
- clearreadout means: kcu clears its fifos, drp clears out UDP packets on configure/endrun
- ric idea: IF we could send out 1 trigger on beginrun then we could use that as the starting for future frame counters to detect a missing first packet. not clear how to send that 1 trigger.

Mail message from Alex:

SXR servers are in R06. There’s likely space in there.
https://docs.google.com/spreadsheets/d/1woH6NHS4IrOvr2zDVZD4R8x73moTXAps1PBqADUauC4/edit#gid=1690903792

Our, or at least my understanding was that we can use an trigger output from any nearby host with an EVR. To be specific:
- PLC: B940-009-R07
- EVR: B940-009-R06 (whatever host we have running the exit slit camera)

I assumed the point/beauty of the EVRs is to distribute these trigger sources so it doesn’t have to originate from any specific node. We were going to include a counter in the payload so we could address any off-by-one issues. I guess it’s more complicated!

Sorry, Chris, I didn’t realize this thread had taken up residence elsewhere otherwise I would have responded sooner. I try to keep an eye on the pcds-it list but it’s kind of a blindspot for me.

The question I have at this point is, where does the Ethernet/<whatever medium we end up choosing> have to go to deliver the UDP packet?

MHz Conversation with Georgi and Zach (May 4, 2022)

May 4, 2022

the absolute encoder will continue to be used
perhaps add a second high-rate (maybe even MHz?) relative encoder
in november either we have to increase the current absolute encoder rate, or do the "interpolation". beam starts at a few kHz.
feels like a good chance we need interpolation mode since the MHz encoder does not yet exist
existing absolute encoder may trigger at 500Hz or 1kHz, but no more
existing absolute encoder has estimates for velocity, acceleration, jerk, maybe more? zach can include these in the data stream fairly easily so we can see how well it does
for psana we will need to have an encoder readout trigger the "start" of a batch (for parallelization) similar, but different from, integrating detectors where data triggers the "end" of a batch.
- is there a batching conflict between integrating detectors (e.g. andor) an absolute encoder? may need to trigger the encoder at a higher-harmonic of the andor? but maybe still a conflict because one marks the end of the batch while the other marks the beginning?

MHz Discussion (June 10, 2022)

June 10, 2022 dan morton, larry ruckman, matt weaver, nick waters, zach lentz, cpo, chris ford, mona

dan has been interfacing with axilon (cologne, germany)
stick with renishaw
axilon designs brackets and cable routing and provides renishaw (
digitized signal comes out
lead times long
encoder is in the FEE (just east of entrance on massive granite)
need Larry/Matt/TID to do the custom timestamping: pcie with FMC and SFP for timing system
what are the cable-length constraints? maybe do a wave8-style box or a dev board (only one of these at the moment?)? prefer dev board at the moment. comes out as PGP to KCU1500 in the DRP.
do we need a second controls interface to this box?
- requires an "IOC" (more work for controls group)
- may make it easier for the scientists to use
need to be able to reset the counter to zero (to get absolute-encoder style behavior)
dan says MHz is overkill, but psana needs MHz.
may need errors sent back to DAQ
need to decide on a table-top model for TID prototype. looking at page 14 of the manual (and page 7, which shows max speed vs. clock and resolution)
- Interpolation factor: RIX would like 1nm (20KD). may decrease range?
- Alarm format and conditions: A (line driven E output; all alarms)
- Clocked output option: 50MHz
- Options: A (This is what axilon suggested, Larry thinks he can adapt)
cable going into Larry's box will be as on page 5 of the manual
cable lengths on page 9 look good.
diagram on page 9 of the manual shows the analog signal getting converted to digital by TONIC interface. Larry needs to supply 5V.
Dan will purchase encoder and second one for Larry prototyping.
schedule (need to ask rix scientists). mechanical install in January 2023.
account numbers: ask rix scientists
ask rix scientists: if we are going to do this, do we still need to "extrapolate" with the existing absolute encoder?

From axilon:

“…the encoder intended to be used is:
-             Read head: T2601-15M or T1611-15M (this depends on availability, the arc can be resolved with a linear and a rotary encoder head, it is actually more a matter of how the scale is mounted and that will be done on an arc)
-             Interface: Ti20KDA06A, that is 20,000x interpolation to get the highest resolution. Note, on the interface one has to check with controls what options they may want to see, for that see below link. What is of importance are the last for letters (i.e. currently the …A06A, this is our usual default):
o            First A: this is for alarm outputs. I have to admit, I have no real clue on that, this needs to come from controls
o            06 is the max output frequency. 6 MHz in this case, often controllers have a limit of what they can take as maximum. The 6MHz is for vibrations check more than sufficient
o            Second A: some options regarding what kind of signals are output, also here this is a controls related question.https://www.renishaw.com/media/pdf/en/257d836362cd4e8189908521185ff4bd.pdf”

Encoder Simulator

Chris Ford has an encoder simulator in lab3, with documentation here: UDP Encoder Interface (rix Mono Encoder). He also has committed lab3-caf-encoder.cnf to lcls2 GitHub in psdaq/psdaq/cnf/.

High Rate Mono Encoder Interpolation Options

Goal: keep the parallelization happy (both ami and mpi-psana) so that every shot has an encoder value (either real or interpolated)

~5kHz of measured data is roughly the maximum readout rate
interpolation will make the data better

Example motor motion:

Options:

DAQ interpolation. Ugly DAQ C++, may want to change later. Works for AMI
- "freezes" the interpolation to what is done at DAQ time: but doesn't preclude someone going back and doing a better analysis later
- what if what we think is noise is actually real vibration of the mono grating. in that case interpolation loses important information. one would test this by doing the analysis both with the raw data and with "smoothed" data to see which yields better physics resolution
only send events with measurements to AMI? requires a ric-style python script to select AMI events
- limits statistics in AMI. in principle could get 5kHz.
- in future could avoid limited-statistics issue by going to 1MHz relative encoder
- cpo worry: we already need such a script for RIX to send the andor events to AMI. two issues:
  - that script becomes more complex trying to do two things
  - cpo thinks: if the (rare) andor events don't have an encoder value it will make the analysis (or ami display) tricky. would need to have "trigger overlap"
(doesn't work for real-time analysis like ami) psana interpolation in-memory
- on SMD0 core (would dramatically complicate our most complicated psana event-builder code)
- psana "broadcasts" all the encoder data to all the cores. quite messy and would affect performance unless SlowUpdate broadcast at 10Hz was feasible
(doesn't work for real-time analysis like ami) psana pre-processing interpolation written to a new xtc2 file
- could also analyze the shape above and write out very little information (10 numbers?): regions 1 (flat),2(sloped),3(flat),4(sloped),5(flat).
(doesn't work because of chaos caused by batching, deadtime, arbitrary number of cores, load-balancing) DAQ repeats without interpolation.
- Ideally xtc2 would be able to say 3 (repeats 5 times) 7 (repeats 5 times). xtc2 can't say this easily.
- (preferred) or another version would be 3(deltatime=0),3(+deltatime),3(+deltatime),3,3,7,7,7,7,7,.... each core could do it's own interpolation. provides flexibility and scalability. extra data volume isn't too bad. will make user code more complex (they will have to "buffer")
- Chris Ford's thought on this "debinning": Encoder Debinning

Toy Example of DAQ Interpolation Option

need matrix inversions (linear-regression for a polynomial fit) using eigen software? Example timestamps/values:

ts  1 2 3 4 5 6  7
val 5   7   9   11

multithreaded option

core 0 is handed event ts=2 (fit vals=[5,7] to get answer 6)

worry about:
1) when can we free the memory at the beginning of the circular buffer?
2) core 0 needs to wait for ts=3,val=7 data to show up before doing the fit

two possible drp versions:

high-rate drp hands out different events to different cores (e.g. 60)
simple drp code that is more "single-threaded"

what version is the encoder code? copied PVA detector perhaps (75% confident)?

single threaded option

if encoder is "simple" version above.

why single thread should be OK:

fits only need to be done at a low "real encoder value" rate (100Hz->5kHz) multi-threading isn't necessary.
polynomial (quadratic or 3rd-order?) calculations a+bx+cx**2+dx**3 (plus have to compute the time "x") have to be done at 1MHz. hopefully can avoid multithreading as well, at least up to 100kHz

main thread algorithm:

wait for ts=3
launch fit subthread (or maybe don't with ts=[1,3] to calculate result for ts=2 this addresses worry (2) above
main thread watches for completion of the fit-subthread and knows when to "delete" ts=1 (addresses worry (1) above)
main thread hands out ts=[3,5] for the next fit

starting plan:

standalone C++ code first to test 5kHz fits with eigen and 100kHz polynomial calculations
order of polynomial would be "#define" or command-line-option
another #define would be number of points to include in each linear-regression

Page tree

RIX Encoder Discussions