This document is a guide for ACR staff for the current best practices when operating the FACET-II linac. For recent news and links to ops procedures visit the MCC wiki FACET-II hot page (requires SLAC intranet access).

Shift protocol

• At the start (or end) of each day/swing/owl shift: save machine configs (SCORE & SCP)
• At least twice per day (typically once on swing shift and again at ~0700): record the standard machine characterization
• Because of the more dynamic nature of the FACET-II program and the difference in science objectives, keeping track of the physics logbook is critical
• Take regular notes 
  • It's still important to make sure critical program info is communicating in the MCC elog
  • If much of the days live note-taking happens elsewhere, copying a summary entry over from the physics log is useful too
  • It's especially important that the EOIC has a sense for how the program has progressed, so they can write an accurate shift summary to then communicate to the rest of AD
• You should expect at least one FACET APBO physicist to be attendant to the ACR each day to provide expert support (see: call rotation)
• When tuning the machine please make notes in the physics log or MCC log
  • Doesn't have to be super-detailed, an accounting of knobs tried and what did/did not work is what is important.
  • Prevents duplicated effort, avoids machine configuration changes that aren't understood

Standard machine characterization

Periodically, the following set of measurements should be taken & logged to characterize beam quality. Up-to-date beam measurements are critical for making informed decisions when running the accelerator.

• Measure & update linac ENLDs (in L2 and L3)
• Save machine configuration
  • SCORE save-the-world
  • SCP config macros
  • Reference orbit (from the orbit display)
• Emittance measurements in the following areas:
  • L0 (injector)
  • L2 (LI11)
  • L3 (LI19)
• Measure the beam size at the final focus
  • IPWS1 X and Y measurements
  • XTCAV bunch length (recommend taking >3 measurements to get a sense of per-measurement error)
• Record beam images from the following screens:
  • PR10571
  • PR11375
  • SYAG
  • DTOTR2

Loss monitoring

During beam delivery, losses & secondary radiation should be kept ALARA

• BCS LIONs, BPMs, toroids, the PLIC, S20 PMTs and RDMs should all be monitored during MDs and user delivery
  • Toroids and the PLIC are on the Beam transport summary CUD
  • LION signals are on the facet PPS/BCS display
  • S20 PMTs/RDMs are on the S20 CUD
• Significant losses should be investigated and tuned out
• Occasionally, experiments (e.g. E300 PWFA) will generate significant radiation in S20
  • Ensure before any experiment work begins that beam transport is clean
  • Continue to monitor loss signals for increases above your baseline expectation

Critical software & controls infrastructure

There are a number of background processes that must remain operational for the controls system to function

• Alpha
  • Mainframe for all SLC micros, if this dies it will take the entire SLC control system with it
  • Single, catastrophic point of failure
• SLC → AIDA-PVA → EPICs communication
  • You can check on the aidapva, aida_slcklys, aida_slcbpm statuses in the SCP (main index → network micro index); if they have a red status, these process will need to be restarted
• Watcher processes
  • In EPICS, physics apps → GOTO watcher
  • Everything on this panel should be green, if a watcher is off/dead/not needed it should be bypassed by its owner
• MDL feedforward
  • You can see the status of this process on the klystron CUD and in the watcher panel 
  • Stabilizing the MDL phase is critical for maintaining performance of the whole accelerator
  • Minimizing actual phase drift keeps systems dependent on phase measurements more robust over long time scales
• S20 differential pumping system (DPS)
  • DPS watcher and upstream & downstream turbopump statuses are on the S20 CUD
  • If you ever see this red, inform an APBO or AARD physicist

Supporting Machine Development

• FACET usually has proportionally more time spent on machine development and tuning compared to LCLS
• The level of direct involvement during MDs may vary, depending on the nature of the tasks required
  • Physicists/users should communicate the motivation and nature of the MD program
  • A high-level understanding of the day's objectives is important for communicating with other shifts, though the nitty-gritty of physics data taking might be less relevant
  • If you feel excluded from activities, please speak up or let the EOIC know

Supporting FACET-II users

• Don't assume that users are OK with/aware of any abnormal beam condition you observe
  • Make sure they are informed of what - if anything - is inhibiting delivery
    • e.g. a jump in backgrounds in the IP, a large orbit, losses of RF, or feedback stability problems with klystrons
• Make sure you understand what the (approximate) beam requirements of the current experiment are
  • User shift plans should be linked in the physics log & communicated to you by the FACET team
  • Advice should be provided on tuning strategies for given configs; if you're unsure of the goals, ask a physicist
• Remember to enforce jurisdictional boundaries in the accelerator
  • You as the operator are responsible for the stable and successful delivery of the e- beam from sector 10 to the IP
  • Users are responsible for operating experiment equipment and choosing beam requirements

Handing off to experimenters

• ~20min before users start:, populate the table of handoff parameters with the following measurements:
  • Emittance measurements from L0-L3
  • IPWS1 sigX & sigY
  • XTCAV sigZ
• Save a machine config, including the orbit
• Note the time when user beam delivery begins

What to do after experiment shifts end

• Note the time when users finish
• Verify explicitly with users that they have retracted/made-safe any experimental devices on the beamline
• Verify that all experiment-specific machine configuration changes have been undone, and if not, make sure the list of what to change is communicated
• Restore the final focus to its nominal config (50cm betas at IPWS1) unless instructed otherwise
• Compare orbit & losses to the pre-delivery config; investigate if if there are large discrepancies
  • Losses may indicate that experimental devices were accidentally left in the beam or that magnets were left in a funky state

On-call physicist

The FACET-II MD schedule lists the current on-call physicist for the day. This person is the first point of contact for any technical questions & issues regarding beam delivery. The phone numbers for all accelerator physics & beam operations (APBO) group members are in the ACR phone tool and the SLAC directory.

• This does not replace the usual escalation mechanisms for requesting help from support engineering groups & communication with the LAF program deputy
• If you're unable to reach the on-call person for a blocking issue, attempt to contact other members of the APBO group

Beam Tuning

• In general, you should tune by proceeding in the order of energy → compression → trajectory → optics
• The ultimate end goal is to optimize the beam parameters that are delivered to the IP, intermediate measurements are just a guide
• Once you find the knob that works, the next step is to revisit all the knobs that didn't work before

Target machine parameters (for 1.6nC operation)

The table below lists the nominal target beam parameters in the linac for 1.6nC, single bunch operation. These targets are based on historical results and not what the design/theory predict the optimal performance to be. (Note: bunch length is characterized by measured sigZ in um for L0 and S20 (where there are TCAVs), and using BLEN monitor read backs in AU elsewhere)

Loss reduction

• In general, maintaining clean transport of the e- beam is more important than having the best possible measured beam quality.
• If you notice losses rising while tuning, you should identify & walk back changes that caused them, If increases in losses don't correlate obviously with tuning changes then look for tripped or malfunctioning devices.

Transverse

The general strategy when optimizing the linac is to identify where in the linac emittance growth is most significant, and then intervene as close upstream as possible.

• When tuning emittance, one should measure frequently
  • If taking multi-wire measurements, often repeat scans of a single wire can more quickly reveal the effect of a parameter change
• Emittance growth in the linac is dominated by two factors: uncancelled dispersion and transverse wakes
  
  • Due to these strong transverse wakes, steering will have a significant effect on the emittance growth of the beam
  • Wire scans that show a flat-topped, less gaussian profile → indicates dispersion → try CQs
  • Wire scans that show a tail/shoulder/double-peak → likely wakefield effects → try steering
• Steering advice
  
  • Try to minimize the rms strength of all correctors used
  • It's good to mix one-to-one steering (i.e. to the next BPM) with steering to minimize the rms orbit in a whole region
  • If you see a coherent oscillation, a corrector near the most-upstream zero-crossing of the oscillation should have the right betatron phase to close the orbit
  • Steering in non-periodic/dispersive regions (i.e. in the injector or inside BC14) is not necessarily forbidden, but should be undertaken with great care

Injector Tuneup

• Steering (ignore the very first BPM)
  • going though DL10 cleanly is extra important
• DL10 sigE minimization
  • This measurement is either taken at the beginning of a run, or the L0-B phase is set based on historical precedent.
  • Rather than start with a full measurement, checking PR10711 by hand is a good smoke test if you think the L0-B phase is wrong
• SOL121, QA10361 and QA10371
• Beta matching usually produces good results provided the input emittance measurements are good
  • If Bmag > ~1.5, consider more hand-tuning before measuring again
  • If the beam is very mismatched, a good strategy is to try to shape the beam to the design expectation while looking at PR10571
    • In the design lattice, betaX = 2*betaY at PR10571→ sigX = root2*sigY, so we expect the beam image to be ~1.41x wider than it is tall
    • Adjust quads until the beam is as small as possible in a ~3:2 aspect ratio, then measure emittance
    • reiterate until you get something roughly matched (Bmag <1.5 ideally)

Linac Tuneup

• Linac orbit
  • When steering in the feedback measurement/actuation regions (LI11-12 in L2, LI18-19 in L3), disable the transverse feedbacks and (if needed) update their setpoints after orbit correction 
  • BC11 and BC14 launch/exit angles should be as flat as possible, with the smallest rms orbit inside the chicane as well
  • Small bumps early in L2 can have a dramatic impact since the beam is still <1 GeV
  • Jitter-based or energy-scan dispersion measurements can provide useful information about where in z to target steering
• Dispersion corrector quads
  • BC11: CQ11317, CQ11352
  • BC14: CQ14738, CQ14866

S20 Tuneup

• Steering
  • Don't steer in BC20 and downstream unless (1) a facet physicist requests it or (2) you really know what you're doing
  • Due to the flexible nature of the S20 optics and the aperiodic nature of the lattice & BPM placements, steering downstream of BC20 is typically challenging 
  • It's most important to have a flat trajectory through the IP, as reported by the two BPMs that bracket it
  • ~5-800um BPM readings elsewhere are typically a required compromise to achieve this
• Sextupole family movers
  • symmetric/antisymmetric moves with whole families can be useful if single-mover tweaks aren't yielding anything (see decoder ring)
  • The sextupoles movers are typically used to minimize the X/Y waist & cancel or linearize dispersion, in practice it's mostly a guessing game
  • Make sure to measure frequently, and note good settings
  • There are "gold" sextuple positions that can be restored to with reasonable accuracy
  • S1E - S3E is the last change for optical corrections before the final focus, so settings here 
• Small momentum offsets at BC20 (typically +/-500um on BPM 2445)
  • This is typically something that's done empirically when the flat trajectory causes losses or otherwise indicates that x = 0 is not the correct zero-energy point.

Longitudinal

The longitudinal setup has fewer inputs that the transverse phase space, and correspondingly only a small number of knobs (linac phases). In general if linac phases are coherent and feedbacks are working, the compression setup should require less frequent touch-up than the transverse does.

• Note: while the bunch length is regulated at BC11 and BC14, the actual target location is in S20.
  • The BC20 BLEN is not regulated by a feedback, however the S20 pyrometer signal is a critical passive diagnostic.
  • Due to dynamic range limitations, this detector tends to saturate around peak compression (fix incoming...)
• Compressions scans (see MCC wiki for notes on how to take this measurement)
  • Typically compression scans are needed when configs are built/optimized. If compression
• L2 phase adjustments (0.5deg steps) or small adjustments to BC14 BLEN setting
  • This can also be accomplished by adjusting the BC14BLEN feedback set point, however extra settle time should be allocated for feedback convergence
• Complementary phase adjustments in L1 & L2 (again 0.5deg steps, move phases together to keep the same downstream BLEN reading)
• Due to strong Wakefields (especially in the over compressed short-bunch running mode), starting with the correct compression setup is important for making transverse tuning gains stick.

Tuning in novel operating modes

2-bunch

• This setup is significantly more complicated due to interactions between both bunches
  • That is, the two bunch beam cannot be through of as the drive + witness in a simple superposition
• So far, our best strategy for transverse optimization has been to treat the two-bunch system as a single (weirdly structured) beam, and attempt to wrangle it with the usual methods
• Making the normal measurements of two-bunch beams (emittance, bunch length) can be significantly more complicated
  • Many measurements are prone to failures of image processing or optimization under these conditions
  • Typically determining the true parameters of the beam requires synthetic reasoning using multiple sources (images, scan data, synchrotron emissions etc)
• One of the most important quantities for the two-bunch setup is the bunch spacing. Tuning to optimize and stabilize bunch spacing is currently a bit of a dark art...

High charge (>2nC)

• Presently, this mode only delivers beam to TD11 and not beyond
• The usual injector tuneup techniques apply here, but emittances will typically be larger simply by virtue of the increased charge

Measurement Quality Control

Exercise careful judgement when evaluating measurements, most numerical methods and optimization algorithms are prone to false positives and are as susceptible to bad input data as any program. Often even obtaining good data for a scan is challenging, depending on the operating mode (e.g. XTCAV bunch length measurements at peak compression).

Wirescans

• An ideal beam profile should be clearly gaussian, with minimal tails/skew (i.e. left/right asymmetry) and kurtosis (i.e. not pinched at the the bottom)
• Very high backgrounds can cause wirescans to underreport the actual beam size as part of the gaussian is "buried" in the noise, and only a narrower peak portion is visible to the fitting algorithm.
• Very high beam jitter will translate to noise in the PMT signals and thus larger error in the final scan. BPM jitter correction on the GUI can subtract some of this error using shot-by-shot normalization.
• The scan range of the scanner motor should be wide enough to fully capture ~3-4sigma of the beam. If too much signal is clipped on the left/right the fitting algorithm will start to diverge.
• If the wire scan ranges seem to need changes, its a good idea to make sure the orbit in the area of the wires has not changes significantly before shifting the wire scan ranges. If the ranges need to be changed please document those changes.

Beam Images

• Be aware of secondary sources of radiation that could be visible on the screen (e.g. IR laser light, synchrotron radiation), depending on what part of the beamline you are imaging
• Be aware of beam images that appear too perfectly gaussian, as this could actually indicate problems with the camera focus

Emittance Measurements

• Make sure to evaluate the input beam size measurements (wires or images) as described above
• For quad scans:
  • The best practice is to use a large (~10-15+) number of quad settings, if you are willing to wait
  • If the emittance fit is just capturing the "linear regime" (i.e. the beam sizes just look like a V shape instead of parabolic), you should narrow the range of measurement to actually capture the nonlinear part of the beam size evolution
  • On PR10571 you should expect the X and Y waists to be near the same magnet setting, if not, this means the beam is mismatched
• For multi-wire scans:
  • The GUI shows the expected betas/sigmas at each wire – when optimizing emittance, look at the ratio of the 4 wire sizes to each other in comparison to the design values.
  • The average wire sizes are proportional to the overall emittance, and the ratio of sizes from wire-to-wire contains information about how well matched the beam is

Bunch Length scans

• It's important to make sure the TCAV is correctly phased to within +/-2deg before starting, otherwise the streaking will also kick the beam which will distort the streaked beam image on the screen
• Ideally, the beam images should be streaked symmetrically at either zero crossing. If one crossing streaks the beam more than another, that indicates the incoming beam is tilted (i.e. x(z) or y(z) are not flat)
• If you are struggling to get good scans, it may be necessary to spend time doing transverse tuning to eliminate beam tilt/uncancelled dispersion to improve the quality of beam streaking

Long-duration Measurements

• Measurements that take longer than ~10-15min are on the same timescale as many sources of machine drift
  • The MDL Feedforward also actuates once every 5 minutes, and can disrupt linac phase scans that run for longer than that time
• It can be helpful to randomize the order of scan settings (instead of linear/zigzag scanning) to minimize the convolution of drift with changes due to the scan
• The most significant source of machine drift on long timescales is thermal, so the optimal time to take certain long measurements is driven by weather