Page History
| Table of Contents |
|---|
with Alex R., Alex W., Zach L., Jana T., Chris F., Matt W., Silke N., Chris O.
Discussion
georgi: 100eV photon scan range over 100 seconds. 1eV/s. few microeV change per shot at 1MHz. need precision 100 microev? need to take out shot intensity (normalization)
...
Matt says we may have an existing board that could do the readout/timestamping
Summary of Options/Information
- 10kHz with interpolation?
- multiple encoders?
- combine absolute/relative (may be complexity issues)
- 2 staggered slower encoders
- need fpga timestamping
- 100 microev resolution requirement
Option max rate work to be done drawbacks/other consideration A absolute encoder used for relevant stage 10-20(?) kHz controls: rework PLC to send data (current solution likely ~1kHz)
DAQ: likely FPGA timestamping
B absolute encoder used for relevant stage,
interleave with second absolute encoder
20-40 kHz as above need to ensure both encoders read back equivalent data,
intersperse jitter from both?, same latency
C add a relative quadrature encoder on shaft 5 or more MHz DAQ: need board to read signal & timestamp complicated and possible not very reproducible mapping of measured &
relevant motion
D add a relative quadrature encoder in vacuum 5 or more MHz DAQ: need board to read signal & timestamp
engineering: figure out how to add relative encoder
To do
- fill out the table options and costs (alex and silke)
- talk to mechanical engineers (Georgi)
- meet in two weeks
Feb. 5, 2021
Alex R., Georgi D., Chris F., Jana T., Matt W., Zach L., Joe R., Chris O.
...
- Feels like option D (see above) is doable according to conversation between Georgi and Danny Morton
- they meets with Axilon next week to verify (manufacturer of mono assembly))
- Use this as our current default solution, but can still change (final decision in weeks)
- Matt W. and Ryan H. think the camlink converter box could be used for readout
- quadrature encoder "frame format" is understood and relatively simple (absolute encoders are more difficult)
- quadrature encoder readout can be triggered
- might need either a new camlink converter board with new connectors or some "connector transition"
- currently no use case for high rate absolute encoders
120Hz Encoder Plans
- plc is in the fee alcove and the encoder itself is in the fee
...
SXR servers are in R06. There’s likely space in there.
https://docs.google.com/spreadsheets/d/1woH6NHS4IrOvr2zDVZD4R8x73moTXAps1PBqADUauC4/edit#gid=1690903792
Our, or at least my understanding was that we can use an trigger output from any nearby host with an EVR. To be specific:
- PLC: B940-009-R07
- EVR: B940-009-R06 (whatever host we have running the exit slit camera)
I assumed the point/beauty of the EVRs is to distribute these trigger sources so it doesn’t have to originate from any specific node. We were going to include a counter in the payload so we could address any off-by-one issues. I guess it’s more complicated!
Sorry, Chris, I didn’t realize this thread had taken up residence elsewhere otherwise I would have responded sooner. I try to keep an eye on the pcds-it list but it’s kind of a blindspot for me.
The question I have at this point is, where does the Ethernet/<whatever medium we end up choosing> have to go to deliver the UDP packet?
MHz Conversation with Georgi and Zach (May 4, 2022)
A Jira ticket about this:
| Jira | ||||||
|---|---|---|---|---|---|---|
|
...
- the absolute encoder will continue to be used
- perhaps add a second high-rate (maybe even MHz?) relative encoder
- in november either we have to increase the current absolute encoder rate, or do the "interpolation". beam starts at a few kHz.
- feels like a good chance we need interpolation mode since the MHz encoder does not yet exist
- existing absolute encoder may trigger at 500Hz or 1kHz, but no more
- existing absolute encoder has estimates for velocity, acceleration, jerk, maybe more? zach can include these in the data stream fairly easily so we can see how well it does
- for psana we will need to have an encoder readout trigger the "start" of a batch (for parallelization) similar, but different from, integrating detectors where data triggers the "end" of a batch.
- is there a batching conflict between integrating detectors (e.g. andor) an absolute encoder? may need to trigger the encoder at a higher-harmonic of the andor? but maybe still a conflict because one marks the end of the batch while the other marks the beginning?
MHz Discussion (June 10, 2022)
June 10, 2022 dan morton, larry ruckman, matt weaver, nick waters, zach lentz, cpo, chris ford, mona
...
“…the encoder intended to be used is:
- Read head: T2601-15M or T1611-15M (this depends on availability, the arc can be resolved with a linear and a rotary encoder head, it is actually more a matter of how the scale is mounted and that will be done on an arc)
- Interface: Ti20KDA06A, that is 20,000x interpolation to get the highest resolution. Note, on the interface one has to check with controls what options they may want to see, for that see below link. What is of importance are the last for letters (i.e. currently the …A06A, this is our usual default):
o First A: this is for alarm outputs. I have to admit, I have no real clue on that, this needs to come from controls
o 06 is the max output frequency. 6 MHz in this case, often controllers have a limit of what they can take as maximum. The 6MHz is for vibrations check more than sufficient
o Second A: some options regarding what kind of signals are output, also here this is a controls related question.https://www.renishaw.com/media/pdf/en/257d836362cd4e8189908521185ff4bd.pdf”
Second High-Rate Encoder Meeting (Nov. 28, 2022)
(with Ruckman, Herbst, Lentz, Dakovski, Wallace, cpo)
- Axilon will install in FEE in December
- Georgi wants it in DAQ by end of 2023
- Quadrature output, oversample counter at 186MHz, sample that at 1MHz
- front-end board (not a computer) KCU105 receiving (like wave8)
- need to order now (16-week lead time)
- should ask hard x-ray hutches if they want any of these
- does TID need a test encoder? they think it's ok to use an oscilloscope to capture a few ticks of the encoder (May Ling wants to do this anyhow to test connections)
- this encoder will be in the FEE, so not always accessible
- will absolute encoder plus interpolation be as good as the high rate relative encoder? answer: we will have to look at the photon "energy resolution".
- Can the absolute encoder go to higher rates? Zach writes:
"With an alternate hardware configuration, the absolute encoder can run faster than we currently run it, but it cannot ever get close to 1MHz.
So, if the goal is to measure the position at full beam rate, we definitely need the alternate encoder hardware as planned.
The BiSS-C protocol allows a maximum clock rate of 10MHz, and this is theoretically the maximum data rate in bits. The full position is sent with every BiSS-C cycle. Therefore, we'd need to allocate less than 10 bits per position to have a chance at hitting 1MHz. In practice there are >40 bits per position including status bits, error bits, and checksums. There is also some additional mandatory dead time. There are some documented tests on the internet of running these sorts of encoders up to 33kHz or so, but nothing near the 1MHz mark."
Meeting on Dec. 12, 2022
An email message I sent after the meeting:
...
The creator of the RTM board that splits the encoder signal was @Olsen, Jeff J. (TID)."
FEE Fiber Status
From Jing Yin (via email)
...
Nov 8, 2023: Found rack 3 on the HXR of the FEE. 3 single-mode cassettes on BACK of rack (hard to access): 6 fiber pairs to B940-009-R06-FOD3-U1, 6 pairs to B940-009-R06-FOD3-U2, 6 pairs to B940-009-R06-FOD3-U3.
High-Rate Encoder Touch-Base (Aug 22, 2023)
Questions:
- will send one number per shot
- keep it simple for now (Larry/Namit have all encoder signals hooked up for future fancy behavior)
- 32-bit or 64-bit number? (how many total ticks)
- Zach thinks less than 4 billion ticks (maybe even less than 1 million)
- Larry says that the firmware bandwidth usage doesn't change in either case
- start with 32 bits for now to save daq disk space and network space
- can start daq-integration now?
- don't need daughter code (need xilinx dev board kcu105)
- rogue code GitHub repo? https://github.com/slaclab/high-rate-encoder-dev
- timestamping: done in kcu105 using the usual weaver/ruckman module
- use pgp4 6Gbit/s for
- increased trigger jitter ok? Larry says only 20-30ps RMS, so that's fine
- handling limit/set-up/alarm signals
- data sheet: https://jira.slac.stanford.edu/secure/attachment/30151/L-9517-9426-03-D_Data_sheet_TONiC_UHV_EN.pdf
- Nick says set-up is least important (says encoder readout is aligned: says that read-head is at the right distance from the strip with the ticks). this should be a one-time setup, requires breaking vacuum to fix. we think this information comes from some of the X,E,P,Q,Z signals.
- P,Q are limit (are these the end of the readout strip? X is setup, E is alarm. Z is reference-mark (like a "home" position?)
- Georgi feels reference-mark is important (although we should probably use the companion absolute encoder for this and for limit-protection)
- Alarm feels important too (corrupts data, either because going too fast or signal level too low, so should never happen)
- don't use an IOC: the DAQ will crash if the alarm goes off and absolute encoder takes care of limits/reference-mark
- Larry will include all the bit (and a latched error bit (counter?)) in the 64-bit DAQ data word, but DAQ will nominally only use alarm bit
- Larry counts alarms and reference-mark
- Not clear if we have a reference mark or not: ideally would check. Nick will email axilon to try to find out. Could check also check empirically.
- the companion absolute encoder can be used instead of the reference-mark.
- Georgi thinks absolute encoder should protect the "limits"
- we guess that it's the same for limit and alarm
- might need to latch some of these (and make them clearable)
- when does rix need interpolating absolute encoder?
- 3 months (november?)
- when does rix need high-rate relative encoder?
- spring 2024
- metastability:
- Larry reports that the way the encoder works this should not be an issue
- maximum speed (before skip-encoder-mark errors):
- spec sheet says 0.032 m/s is the max speed, but we need to convert this from linear to angular
- 1 count per nm
- 150urad per count
- multiply/divide as needed: https://www.wolframalpha.com/input?i=0.032+m%2Fs+*+%281+%2F+1nm%29+*+%281urad%2F150%29
- = 0.213 rad/s is the max speed
- We typically run this under 20 urad/s which is orders of magnitude below the calculated limit
MHz Encoder Discussion (Oct 17, 2023)
cpo and Larry
Signals:
- P,Q are limits
- E is error (encoding error, typically from going too fast, or could be misalignment?)
- A,B are for calculating position
- Z is home
...
event builder has blowoff, as usual
Installation To Do List
- run 3 SM fiber pairs (2 plus a spare) from granite to R03 (Jyoti)
- figure out kcu105 mounting structure on the granite (Jyoti)
- figure out the power for kcu105 (Jyoti)
- verify that Larry agrees connector and length are OK (Nick recommends less than 3 meters) (chris)
- patch/test (mona)
FEE Installation Pictures
The connector to the FMC:
...
The single-mode patch panel:
Encoder Simulator
Chris Ford has an encoder simulator in lab3, with documentation here: UDP Encoder Interface (rix Mono Encoder). He also has committed lab3-caf-encoder.cnf to lcls2 GitHub in psdaq/psdaq/cnf/
High Rate Mono Encoder Interpolation Options
Goal: keep the parallelization happy (both ami and mpi-psana) so that every shot has an encoder value (either real or interpolated)
...
- DAQ interpolation. Ugly DAQ C++, may want to change later. Works for AMI
- "freezes" the interpolation to what is done at DAQ time: but doesn't preclude someone going back and doing a better analysis later
- what if what we think is noise is actually real vibration of the mono grating. in that case interpolation loses important information. one would test this by doing the analysis both with the raw data and with "smoothed" data to see which yields better physics resolution
- only send events with measurements to AMI? requires a ric-style python script to select AMI events
- limits statistics in AMI. in principle could get 5kHz.
- in future could avoid limited-statistics issue by going to 1MHz relative encoder
- cpo worry: we already need such a script for RIX to send the andor events to AMI. two issues:
- that script becomes more complex trying to do two things
- cpo thinks: if the (rare) andor events don't have an encoder value it will make the analysis (or ami display) tricky. would need to have "trigger overlap"
- (doesn't work for real-time analysis like ami) psana interpolation in-memory
- on SMD0 core (would dramatically complicate our most complicated psana event-builder code)
- psana "broadcasts" all the encoder data to all the cores. quite messy and would affect performance unless SlowUpdate broadcast at 10Hz was feasible
- (doesn't work for real-time analysis like ami) psana pre-processing interpolation written to a new xtc2 file
- could also analyze the shape above and write out very little information (10 numbers?): regions 1 (flat),2(sloped),3(flat),4(sloped),5(flat).
- (doesn't work because of chaos caused by batching, deadtime, arbitrary number of cores, load-balancing) DAQ repeats without interpolation.
- Ideally xtc2 would be able to say 3 (repeats 5 times) 7 (repeats 5 times). xtc2 can't say this easily.
- (preferred) or another version would be 3(deltatime=0),3(+deltatime),3(+deltatime),3,3,7,7,7,7,7,.... each core could do it's own interpolation. provides flexibility and scalability. extra data volume isn't too bad. will make user code more complex (they will have to "buffer")
- Chris Ford's thought on this "debinning": Encoder Debinning
- (too complex) treat the encoder as an integrating detector
- won't work with ami which doesn't use Mona's batching
- creates a lot of constraints with other integrating detectors: their trigger rates have to be multiples of each other with no phase difference.
Toy Example of DAQ Interpolation Option
need matrix inversions (linear-regression for a polynomial fit) using eigen software? Example timestamps/values:
| Code Block |
|---|
ts 1 2 3 4 5 6 7 (1MHz) val 5 7 9 11 (100Hz) |
multithreaded option
core 0 is handed event ts=2 (fit vals=[5,7] to get answer 6)
...
what version is the encoder code? copied PVA detector perhaps (75% confident)?
single threaded option
if encoder is "simple" version above.
...
- standalone C++ code first to test 5kHz fits with eigen and 100kHz polynomial calculations
- order of polynomial would be "#define" or command-line-option
- another #define would be number of points to include in each linear-regression
Event Builder
Information we have for event-build:
...
Starting point software: let's tweak encoderV1. BLD might have code that deals with disparate rates (would need to talk with Matt).
Issues
- how do we enable the low-rate readout group since the tpr process doesn't participate in the collection mechanism? rough proposal: set in cnf file (so shared by both tpr and daq), timing system would program, and control would somehow display
- currently caf needs the relative rates of the two readout groups. would be nice if it could be "learned" somehow using the dgram env bits with readout group info, but deadtime can eliminate some of those events.
Interpolation Algorithm Iteration
Discussion on July 10, 2023
...
protecting buffering:
- ideally size the pvQueue so it is "bigger" than the dmaQueue. then deadtime protects
the pvQueue from overflow.
o pvQueue is a low-rate queue, while dmaQueue is high-rate
- 2044 dma buffers +10% (204) for extra firmware buffers
- then we need (2044+204)/350 pvQueue buffers
Resetting Interpolating Encoder Data
Discussion on Oct. 4, 2023 with caf/claus/weaver/cpo
...
Overview
Content Tools