May 16, 2017
mark
- getting open cluster working in lab2 drp node
mikhail
chris
- lab2 pgp loopback is running
- 0.5 Mpixel epix calibration is 30ms and peak-finding is about 60ms (although maybe our algorithm can be sped up). doesn't bode well for 4MP epixM
sioan
- led's are counting on FPGA eval board
chuck
- working on SZ absolute error of +-30 doesn't seem to affect indexing
silke
May 9, 2017
clemens
- pgp in lab2
mark
- thinking about reusing astronomical package
mikhail
chris
- setting up test stand
- still understanding trigger vs. filter setting
silke
- distracted by turning things on
sioan
- moving from devel board to pgp card
- no progress on sxrk4816
chuck
- sfx: trying sz with 60 ADU
- spi: users like veto, binning, but not general lossy compression (e.g. sz)
- spi: data will be more sparse because of lower fluence (not true for sfx)
May 2, 2017
clemens:
- working on other stuff. need to pick out new project.
- handed off drp event distribution to mark
mark:
- working on drp event distribution
mikhail:
- working on other stuff (fiber diffraction)
sioan
- able to program image for xilinx eval board!
- still working on sxrk4816 SZ compression
- identified turner expt in oct for data reduction
silke
- will do determination of z distance of detector
- will see if beam centers work for tim analyses
- working on time tool calibration issues
chris
- understanding DAQ L3T code/control
Apr 25, 2017
sioan:
- has fpga hardware
- starting to use Herbst makefile
- sxrk4816 is on back burner (using absolute sz errors)
- continue to identify experiments where we can reduce
mikhail:
- working on fiber diffraction
yoon:
- finished indexing the 1% error (191k patterns) (vs 200K without the error). but might be missing a couple of runs which could explain difference.
- redo with absolute errors.
- wrapped up SPI wavelets. binning is better than wavelets.
chris:
- want to look at DAQ code for triggering
clemens:
- working on DRP event distribution
Apr 4, 2017
sioan:
- vivado hls: C-style programming done
- matlab is next
silke:
- filtering with ipimb
- mike kozina expt analysis (hoffman expt) to test filtering
chuck:
- indexing selenium, in progress
- jungfrau: 8kHz @4MegaPixel now, with 16 images depth
- epix: 25kHz in 2019, @2MegaPixel
- vfccd: 5kHz @4MegaPixel
mikhail:
- working with meng
chris:
- sz
- save for a day
clemens:
- data in and out of pgp
Mar 28, 2017
sioan:
- still working on compiling hello-world
- still working on sxrk4816 with clemens
silke:
- cleaning up ring analysis
- get a t0 difference-image and talk with Tim
- think about ipimb filtering
chuck:
- student (zhen) working on 1% SZ error on selenium sad
- hao-yuan working on wavelet for SPI, feels messy (throws away photons) leaning towards binning
clemens:
- working on distributing PGP events
cpo:
- explore the idea of keeping the data for a day
Mar 14, 2017
cpo:
- zhenglin results
- upcoming meetings with users
sioan:
- working sxrk4816 analysis
- continuing to work on FPGAs
mikhail:
- working on cxif5315 analysis
- ask meng if hit-finding will work for her experiment for DRP
clemens:
- added vectorized KNL code
- added silke's pyfai GPU measurements
- working with Dan/Jack on handing out
- how does NVME buffering work
silke:
- beam centering: hough, ransac, scipy optimize 1-ring fitting
- canny uses soble filter
- looked at skbeam. canny works better.
- hough transform took optimizing
Mar 7, 2017
chuck:
- playing around with SZ. difficult because doesn't integrate into python.
- simulated SZ by adding 1% error
- sees common mode shift. surprising, not understood
- indexing rate is 98% similar. looks promising
mikhail:
- fiber diffraction
sioan:
- committed spatial example to drp eval
- looking at effect of SZ on georgi's expt
- maybe work with clemens on sxrk4816
- mtg with Ryan Herbst on Mar 8
chris:
- zhenglin doing the legion peak-finder
- cuda peak-finding looks worse than cpu
- zhenglin will send out updated results
silke:
- finished pyfai
- working on center-finder
- some people want to run center-finder multiple times during the run
- looking at the hough transform in skimage (needs a boolean)
- hough transform is SLOW. working on it.
- difficulty with conda and interactive plotting
clemens:
- integrating silke's pyfai numbers from gpu onto his plots
- will work on legion
- next project: work on the infrastructure for handing out events to different cores
Feb 28, 2017
mikhail:
- looked at skbeam centering algorithm
- sz compression
silke:
- try SZ compression on some XPP/XCS datasets
sioan:
- try SZ compression on some SXR/AMO dataset (works trivially with georgi's dataset)
- spatial FPGA code working
clemens:
- angular integrations: background is not symmetric (scattering from masks etc). take difference of t0 (no signal) and later data (with signal) and that is symmetric and allows determination of beam center. beam centers changes every half-hour so calibrations would have to be quick.
- have results for CPU/KNL
- found benchmark of EPICS common-mode correction, found performance problem in ndarray.
- looked at intel vtune
chuck:
- wavelet stuff takes 3ms on 250x250 image with pywt
- visually physics results look OK
chris:
- working with legion people on gpu-based two-threshold lag
Feb 21, 2017
sioan:
- working on spatial. good progress
clemens:
- found slowdown for python bin count: memory copy due to bad types
- went through nersc presentation slides
- need to use the fast local ram on knl
- to do: how much does it cost to get 10GB/s through a node?
- working with timbvd on beam-position centering
- why is it hard to do in angular integrations in real-time? detector corrections
mikhail:
- working on sz from argonne
- difficulty running examples, but fixed
- only runs on floats
- timing measurements for sz
- automatic centering algorithm in skbeam/xicam/silke's huff algorithm
chuck:
- away at LBL xtallography workshop: pessimistic about drp
- unit-cell not known
- bad pixels, which change over time (good enough, but room for improvement in our dark runs)
- compare bad-pixel maps from super-users to dark-run results
- geometry
- python scaling problem work
- students look at pywavelets: zero-pads for performance reasons
Feb 14, 2017
Chuck:
- Tried SPI wavelet compression: 4x (on top of a 4x binned image)
- quantifying the error at goal: 3Angstrom resolution (currently 9nm)
Mikhail:
- looking at igor/matt compression
- much faster than gzip, but limit is 2x compression
- igor used multi-threading, matt did not
- to do: enter igor stuff in spreadsheet, work on argonne package
silke:
- work on pyfai
- ran on gpu102
- compared with henrik bincount method
- csr ocl is 4x as fast 1x henrik code, their numpy version is very slow (30x slower than henrik's)
- turn it into a metric (standardize on 1024x1024 single-precision "bincount" method, if pyfai will do that)
clemens:
- benchmarking phi
- python is 8x worse than C
- to do: test at SLAC and test with matrix multiply
- to do: run all corrections and see if we can get 10GB/s
sioan:
- running spatial builds
Feb 7, 2017
sioan:
possible users to contact:
- nora berrah
- dan higley
- mankowsky
- urs staub (elsa abreu)
- marcus kubin (slow andor)
- emmanuel jal
- sam vinko
- christoph bostedt
to do:
- finishing populating list
- will work on algorithms or data transfer
- come up with examples where we need shot-by-shot position measurements with a reticle
silke:
cases where people rerun analysis:
- some people reran roi's 3 months after
- some people had a crappy detector and had to rerun analysis
possible users to contact:
- timbvd
- mike kozina
- reis / mariano
- henrik
- ulvestad
- marco
to do: will work on algorithms or data transfer
mikhail:
- szip (proprietary)
- lzf+shuffle (1.5 for calibrated, 1.7 for raw)
- shuffle (5%-20%)
- best: gzip with shuffle (2.2 for calibrated)
- wait on SZ compressor results
- look at igor algorithm
- picking a feature-extraction
clemens:
- comparing python/C++: still working on it
- zmq/mpi/gasnet
- gasnet: 3 year old python bindings
- setup mtg with elliott/wonchan?
- talk with DAQ about DRP re-configuration
Jan 27, 2017
mikhail:
- henrik algorithm
- filipe maia algorithm
- running on gpus/knl
- evaluate compression package from argonne that amedeo will send out
clemens:
- why only 10GB/s with bincount?
- why c++ 10x faster?
- scaling plot with cores
silke:
- understand required info for each expt (talk to timbvd about real-time angular integration difficulties)
sioan:
- populate amo/sxr 2016 table like Silke
cpo:
- evaluate pyfai
- work on gpu peakfinding with legion with zhenglin
- purchase knl/gpu/nvme boxes
Overview
Content Tools