LCLS Online Physics Applications.

From 20000ft

This document describes a possible architecture for physics applications for the LCLS. From the Requirements for High level Software Applications

High-level applications packages refers to the controls software used by physicists and
accelerator operators to:

• tune or optimize the beam,
• to keep the beam running stably in the optimized state,
• monitor performance for long-term optimization
• to diagnose problems with machine performance
• detect, prioritize and notify of fault conditions

Here we outline the support for accelerator optics modeling in XAL, and in the exiting online SLC control system, and include recommendations for a technology track, the process for XAL modeling, how SLC and XAL modeling can be brought together to meet the commissioning schedule, and highlight issues that will require attention. It is assumed that both the SLC modeling environment and SCP applications will be replaced by XAL based equivalents over some period.

Desktop Hardware and Filesystem

Our target desktop processors will be x86 CPUs running RedHat linux, with the GTK window system (see Details). The executables may be housed on either AFS or NFS filesystems (see Filesystem). Each user (including control room heads) additionally requires their own configuration file area - the precise configuration seen by each head may be unique therefore. This is a feature of XAL and the other desktop technologies we'll use. That configuration file area will be NFS because a long-lived executable (>25hrs - the AFS token lifetime) must be able to write to it at any time.

X11

XAL (that is JFC/Swing) and SWT/Jface applications may be used on any X11 equipped workstation (Windows PC, Solaris) with some performance degradation because JFC/Swing performs poorly over X11 (even in Java >=1.4). These apps could be run "natively" on Windows (since Swing is pure Java, which is platform independent, and any SWT components could be delivered for Windows too, but the added complexity of synchronizing filesystem resources between the Unix filesystem and the Windows filesystem probably makes this option undesirable - see Redflags. Hence, Proposal: no native Windows apps, Windows use only over X.

Overall User Interface

Use Case: A user (physicst or operator) types the name of the main application, say "lips" ("LCLS integrated physics environment" so we can use a word in this doc) at a Linux console. A GUI based application launches. This application can launch any EPICS display (DM, dm2k etc), any XAL application, the SCP (on MCC), or any EPICS extension application like archive browser, alarm handler watchdog, stripchart, cmlog browser etc. These are launched on the correct host for that application or display. The "local" messages generated by each application appear in a console window dedicated to that application - "global" messages appear in the jcmlog browser window. (This functionality has already been developed - see figure 1.)

Systematics of the User Interface

Figure 1 shows a screenshot of representative user applications for LCLS (a larger version of this picture is here). The main application shown is a minimally modified Eclipse Rich Client (that is, basically Eclipse out of the box) in the Resource Perspective, where the Eclipse Workspace has been configured to launch various kinds of program. A number of execution modes are represented (in-process, out-of-process, using Swing and SWT GUI frameworks, plus EPICS display technologies).

Kinds of Eclipse launch modes used by "Lips":

1. Eclipse external launching (EPICS displays, archive viewer, and MCC SCP are shown launched, but many are available: eg matlab, xterms, DECterms on MCC, Elog, Physics Log).
2. Eclipse launching external java SWT/Jface application (jcmlog shown in bottom right)
3. Eclipse launching internal java SWT/Jface (same VM) application (aida probe shown bottom right)
4. Eclipse launching external java JFC/Swing (XAL) application (NOT SHOWN YET)

Note that a list of common EPICS displays is directly accessible from the project window on the LHS. In fact any display can be added or deleted from the list trivially - the list is kept in each user's Workspace config.

Graphical User Interface Frameworks.

In addition to the overall lattice modeling framework and interface to device control, XAL provides one GUI framework (based on JFC/Swing). All the existing XAL applications using thise framework will be provided through "lips" such, such as Scan ("Correlation Plots"), SCORE ("configs"), XIO ("z-plots"), or just the XAL root application.

Additionally lips may include applications which use the modelling and control aspects of XAL, but use some other GUI framework, such as matlab applications, or applications that use SWT/Jface for implementing the user interface view and controls, for instance the jcmlog browser. These are run in a sub-process of the lips executable with no resource splitting.

The distinction the user sees between these applications is relatively minor as long as they are running on Linux to a local display, so window repaints are relatively fast for Swing based apps.

Additionally applications may be written which leverage the Eclipse Rich Client Platform, in which case they run in-process and in the same VM as Eclipse. A trivial example shown is the Aida probe shown in bottom left, which was started from the "Controls" menu. Such applications may be expected to run with better performing use interfaces (though there is some controversy over this point in the literature) and have native look and feel (as opposed to JFC/Swing apps that always have the Swing look and feel of the operating platform).

See also Application Framework Architecture

Basic Model Environment

Geometry, Lattice, Optics

This section describes the process of constructing a model of the optics of the accelerator suitable for applications.

The problem is get from a beamline designer's view of the accelerator (the "geometry" - the physical layout of the machine and its gross beamline elements) to the "lattice" (how the beam is constrained in terms of controllable devices, marker points etc), to the description of the transport system "optics" (R-matrices and Twiss parameters) used in online model based applications.

Use Case: A beamline physicist will provide MAD, Parmela or Elegant files (see #ref_latticefiles). Some designated person will update the Oracle DB with relevant information from these files, and run some process (presumably a SQL script) which outputs an XAL lattice input file in XML - the file containing the lattice description of the machine (See questions). Given this input lattice file, software in the XAL package library can then be used by applications to compute optics (Twiss, R-matrices, etc), which they can then use to calculate bumps, orbit corrections, beta-match settings etc.

Systematics of Optics Computation

In XAL each application which uses optical parameters (Twiss and R-matrices) must track a lattice or use a version of the XAL beamline description file which includes the Twiss parameters - from which it must compute R-matrices if it needs them. If tracking, it must acquire the energy values from klystrons and k-value of optical components, and track the lattice (which XAL calls "probe"). One of the apps, the Model Optics application, allows a user to browse optics so calculated.

For LCLS, we will pre-compute the optics from the lattice, to produce the XAL file which includes the Twiss params, so each modeling application doesn't have to track a lattice for itself before starting. The online modeling "tracker" will be that in XAL (primarily in the gov.sns.xal.model package). The output of this phase will be the Twiss parameters of each element of the beamline. A second probe can calculate the transfer matrices (R-matrices). The Twiss param outputs will be captured in a series of second XAL files. There is presently no file-handling code in XAL for statically recording the R-matrices or global parameters (path length, energy, betamax), so new output files will be designed for that (either extending the XAL syntax, or creating some other XML file). proposal: adapt probe for coupled R-matrices.

One can track an XAL beamline in either of 2 "modes", i) "design" or ii) "from EPICS PV" - which is to say using settings for each lattice element found either in the XAL input file ("design") or by accessing the EPICS Process Variables of optical components, and since XAL is energy dependent, the klystron values also, themselves at runtime. The latter produces a model of the optics of the extant machine. We should have file management capability to store a single recognized "design" lattice file, plus a "gold" lattice (the inputs "from PV" that produced good optics (see To Add), plus at least one experimental lattice file. This capability to switch easily between a lattice of the extant machine and an ad-hoc lattice was badly missed in the SLC modeling environment.

Store the optics in a well known place

The results of tracking (Twiss, R-matrices, global machine params, k-values of optical components) for each of these beamlines for each of the choices of model (design, gold, experimental) for each source of input data (design, PV), will have to be stored. Put twiss and R-matrix outputs in CVS, primarily to record change history.

The input source combinations lead to a number of such outputs, calling for careful filenaming. Additionally, some applications will require coupled R-matrices and global parameters (Orbit correction, bumps). User won't want to wait for coupled lattice to be tracked in the initialization stage of each modeling program's startup, which recall involves getting extant PV values and tracking the lattice. Proposal: put the outputs (Twiss, R-matrices) into the Oracle DB. Programs will go to the db to get those optical parameters. See figure 2.

"KMOD to BDES"

The XAL model includes energy which it acquires explicitly from klystrons (if running "from PV") at the time the lattice is tracked. Therefore there is no "BDES to KMOD" requirement for online applications as such . Is there a "KMOD to BDES" requirement? That is, is there a Lattice Diagnostics requirement for online applications, and a need to keep magnet "fudge factors" (see questions)?

Beamline Descriptions

The number of beamline description input files we need depends on the number of beamline geometries the LCLS will be run in (for instance, how many modeled extraction lines for experiments will there be)? It additionally depends on how many energy profiles will there be? Also high-level applications that deal with beamlines (bpm orbit displays, steering etc) will initialize displays based on the end-points of an input XAL beamline description file. So the XAL orbit correction app for instance, will always initialize to steer the whole LCLS unless a separate beamline file is written for smaller regions of interest - such as injection separately from main linac.

It seems reasonable to assume we'll need to deal with at least the injector, main linac and wiggler separately. So, we'll need some way to manage beamline sets, possibly in a similar way to SLC, where a given named "beamline" is composed of conjoined sections, and each can be tracked or used independently or conjoinedly.

Using SLC modelling to expediate LCLS apps development.

XAL is intended to provide much of the code-base of new applications of the LCLS. Until the XAL modeling environment is ready for use we're going to use the SLC modeling environment to model the optics of the LCLS injection. However, this need not stop us developing and deploying XAL based modeling applications (such as emittance measurement); since XAL uses an input XML file for both lattice description and to store the Twiss parameters following tracking, and it is this combined output file which is used as input by XAL applications, we can take an input XAL beamline description file, and insert the correct Twiss parameters from a corresponding model run on SLC via AIDA. See Figure 3. The resulting output file can be used by XAL applications before the XAL model system is ready.

Non-XAL unix based applications in general (eg matlab) can also be developed ahead of the XAL modeling environment using Aida to get R-matrices and Twiss parameters etc from the SLC model environment.

Figure 3

ARCHITECTURE

This section articulates the specific libraries, packages and programs that will be used to implement the LCLS high level applications.

Role of the SLC Control System.

As stated repeatedly elsewhere, the existing SLC applications, such as correlation plots, steering, bumps etc, can be used to access LCLS beamline orbit and control data. That is, SLC apps will work for LCLS. This is ensured by the "SLC-aware-IOC" project, in which VMS hosted SLC control system signals will be processed by field IOC's and data returned in a manner transparent to the VMS SLC control system.

Model based applications of the SLC control system, will require a DIMAD based model of the LCLS. Work to create that model and associated SLC DB support, has been planned and will start in the new year.

New applications however will use the architecture and tools described below.

Data Access and Control

Data access and control will be via:

1. Java Channel Access (JCA). JCA is the EPICS Channel Access client used by XAL.
2. Channel Access in Java (CAJ). CAJ is a Java native EPICS channel access client, which should be easier to use and higher performance, for applications that are not using XAL.
3. Accelerator Independent Data Access (AIDA) gives access to SLC db, model (Twiss and R-mats), and History and EPICS archiver data, plus will soon give access to collectively controlled SLC devices (BPMs and magnets - so can acquire whole beamine at a time etc).

The choice of which of these is appropriate will vary for each application.

Application Framework and GUI Framework

This section outlines how components of XAL and the Eclipse RPC can be used together to produce GUI applications with responsive user interfaces on any platform.

XAL has three largely distinct components: a modeling "code" or tracking engine (package gov.sns.xal.model); a javabean-oriented device control layer (classes extending gov.sns.xal.smf.AcceleratorNode), e.g. class Magnet and class BPM; and a set of classes to implement applications in a Graphical User Interface (GUI) based on Swing (packages gov.sns.application and gov.sns.tools). For LCLS applications we will initially use all 3 of these to create new applications.

Eclipse also has 3 distinct components: Firstly it's an integrated development environment for writing programs in java and C++. More importantly for LCLS applications, it includes a sophisticated skeleton for GUI based applications (the Rich Client Platform, or RCP). Applications integrated into the RPC use SWT/Jface as the GUI framework. XAL on the other hand extends the JFC/Swing GUI framework. So, one could not use the GUI components, esp widgets, of XAL in an Eclipse RPC application. Never-the-less one can use all of the non-GUI aspects of XAL for an RPC application, just like matlab XAL programs do, and conversely one can use the SWT/jface framework to write a graphical XAL application.
Therefore, depending on the application, a programmer can choose to write a given LCLS application that uses XAL, either using JCF/Swing, in which case they will be able to use the GUI framework of XAL, or SWT/Jface.

Eclipse Plugins, Software Development and Distribution

Eclipse includes an excellent facility for developing extensions to the basic Rich Client Platform. Eclipse is itself composed of a hierarchical system of plugins. Programmers can develop plugins adding their functionality (for instance an orbit correction plugin). The plugin self-describes the precise versioning requirements it has with the plugins on which it builds. Eclipse includes a facility for distributing plugins via an update site. This provides a framework in which we can develop and distribute new code, both internally and externally with the "EPICS Office collaboration" (plus XAL if we wrap XAL as an Eclipse plugin).

Proposal: develop new applications assuming JFC/Swing. If the first couple are not performing adequately, convert to SWT/Jface. Use Eclipse as a launching platform.

TOOL SUMMARY

Integrated Development Environments

Not a good idea to prescribe these for everyone. Three good open source choices for our technology stack are emacs+jdee, Netbeans (including matisse) and the Eclipse IDE. All three include CVS integration.

RED FLAGS

Some important choices or questions.

Windows filesystem.
If we do require native execution on Windows that decision should be made clearly and early, and resources assigned to implement the common executable and configuration file distribution so that Unix and Windows running applications are in sync and see the same configuration files. Java Web Start and Eclipse update site technology may be places to find solutions.

XAL Plane Coupled modeling.
The existing XAL probes for calculating R-matrices, run on either the X-plane or Y-plane and returns only a 2x2 matrix for each element. That is, it's uncoupled. This is reflected in the fact that only the Twiss parameters can be stored into an XAL file - it only contains uncoupled optics. So, if plane coupled orbit correction is important for us, we need to add at least 4x4 and probably 6x6 tracking.

XAL Modelled acceleration and solenoid. XAL does not model acceleration (required for ???) nor solenoid field (required for ???). These have to be added.

Archiver. Which one?

XAL file generation directly from Oracle. Is there a mechanism to flag items in the oracle db for inclusion in the lattice? Danger is people will edit the Oracle db to say move an item to a new z location or include a dummy bpm, and the lattice will change.

REQUIRED ADDITIONS TO SLC CONTROL SYSTEM

An LCLS model beamline, associated sectors and db additions in support of it, must be added.

SCP runtime problems on Linux fixed. At time of writing this includs a problem with dialog inputs, data has to be entered twice.

Pulse-id acquisition by Correlation Plots. Presently broken needs to be fixed.

REQUIRED SOLUTIONS

This section lists functional components of the LCLS modeling applications architecture which for which we need to decide on a vendor:

1. Numerical Analysis package, for matrix manipulation and linear algebra. Some classes of XAL extend JAMA 4, this may be sufficient.
2. J2EE compliant Application Server. Application Servers include technology for accessing databases (eg oracle) and displaying information to the web. This will be useful for displaying static control data, model information and other slow control configs etc, to the web and to the Integrated Control Program. Suggest Apache Jakarta, plus PHP, Xerces, Xalan, etc.

JOB LIST

Lists some jobs that might otherwise be forgotten.

1. SQL script and other munging to get from Oracle db description of geometry and devices to a lattice description in an XAL input file.
2. The XAL tracker. An XAL application which tracks the lattice and produces a 2nd XAL file containing the twiss parameters for each device, plus other per-element parameters.
3. Add acceleration and solenoid elements to XAL.
4. Check XAL's linear algebra support. Done: it uses JAMA -see ref.
5. Create a Skills Matrix for applications, eg, XML and associated technologies (DOM, XSL etc), Oracle and associated technologies (SQL, PHP, persistence).

QUESTIONS

1. Verify AFS client for Windows is not workable.
2. Find best performing Windows X11 server for JFC/Swing clients.
3. How complex is the conversion of Oracle DB to XAL lattice? Does the db contain enough to emit a lattice? Mark Woodley warns that there are probbaly many things necessary in the XAL lattice which need not be in the DB (marker points), and many things in the DB which should not be in the lattice. This latter class includes things like dummy BPMs, so not simply the case that everything of a given class should be in the lattice. People change the z of things in the DB. May be a good idea to have a "Skeleton" XAL file (maybe processed by xsl/php).
4. Which XAL applications allow one to update the EPICs magnet k-values after a successful model run?
5. Does XAL really not model acceleration? If not how is it useful for SNS?
6. Verify that XAL does not model a plane coupled system.
7. Does XAL offer matrix manipulation, linear algebra and fitting adequate for the LCLS? Ans: yes, uses JAMA.
8. How long does XAL take to parse an XAL beamline? It has to build a DOM tree, which can't be negligible. Since that is a requirement of every XAL application at initialization its an important performance bottleneck.
9. How many beamline descriptions should there be?
10. What is the canonical list of XAL physics apps?
11. Does LCLS need SVD based orbit minimization? Is there an orbit minimization in XAL?

R&D

This section outlines some constrained R&D we should do to check architecture decisions.

Matlab

Accelerator Toolkit (AT)

See what applications are available in Accelerator Toolkit (AT). Use of matlab for applications depends on desirability of the functionality offered, responsiveness of the application in the context of a large accelerator's operations, and importantly, ability to secure licensing with manageable cost/benefit.

Specifically: Do AT applications appear to offer support for configurations necessary in a large machine (e.g. choosing from available bpms and correctors in an orbit correction package). How does it handle errors? Are bpm statuses communicated to the user interface and graphics?

Matlab Plotting

Can Matlab be used as a general purpose charting engine? This would be very much more desirable than XAL's charting if having charted the data for a plot one could easily "drop into matlab" to further analyse the data just plotted.

Filesystem and distributing software releases

Should the production executables (XAL java jar files etc) and configuration files be on AFS or NFS? Operative points in the question are that both XAL and Eclipse require that the user specify a "workspace" at startup, which defines their particular configuration. This workspace must always be writable, so AFS token expiry after 25 hrs use, can be problematic. Simplest solution is makefile "install" to a mounted NFS directory - but that directory has to be accessible by all head nodes (that is, all linux desktops running the applications).

Eclipse includes two software distribution mechanisms: first requires "pull" (the user initiates the update), 2nd is Java Web Start, which checks for updates at predefined intervals.

DETAILS

This section gives details behind architectural statements made above.

Why so definite about x86 Linux RH with KDE/GTK-2?

To create a high performance user interface we should lock down the desktop hardware. We can support more than one such configuration, such as x86 Windows and x86 Linux with GTK, but for high performance UIs we need to build for specific desktops, so we have to know what they will be a-priori. Eg, when building SWT/Jface apps, one has to target a gives set of desktops so the appropriate libraries can be included in the build.

ERROR HANDLING, LOGGING AND VIEWING

A programmer spends half their time handling errors, so making that as easy as possible is a productivity priority. For LCLS we will use the following technology stack:

Err package (see http://www.slac.stanford.edu/grp/cd/soft/err/) makes the process of rigorously handling errors easy. It allows a programmer to define an error code in a structured, recorded global way; and to use that code to issue errors and associated diagnostics from or linux, solaris, windows platforms in either java or C/C++, and issue them directly to CMLOG (or some other message logger). It includes support for exception chaining ("caused-by") and exception translation, so the end user can see what systematic thing caused their functional problem (E.g. of a error logged by Err in cmlog might be "UnableToTrackBeamlineException: LCLS ; caused by FileNotFoundException: LCLS.XML when attempting to steer LCLS".

Cmlog. cmlog is a message logger that can be used for error messages. It includes components for receiving messages to log from programs running on the network (either in hosts or IOCs), logging those errors in a database, and a viewer for browsing the logged messges.

jcmlog. jcmlog is a sophisticated error log viewer for CMLOG developed at SLAC recently for LCLS. It is in general much faster and smother than the nominal CMLOG viewer from Jlab and includes facilities for handling the particular requirements of Err (above) such as horizontal scroll for long messages.

NAMES

Light relief - what are we going to call this system?

LIPS LCLS integrated physics system
SIPS SLAC integrated/interactive physics system
iSCP Integrated SLAC Control System
SOMA SLAC online modeling applications.

TECHNOLOGY GLOSSARY

1. Oracle: The enterprise db contains static data on devices and their beamline geometry.
2. XAL: Existing software which contains a lattice tracker, java bean classes for programs to deal with accelerator devices in an object oriented way, and code for building GUI applications based on Swing.
3. XML (see references): A computer language for writing files containing structured data. XML is used by XAL as the language of its input model files. Data in XML is trivially transformed via XSL into other forms, such as HTML for easy display on the web, or other XML files suitable for other programs or putting into a db.
4. AIDA (Accelerator Independent Data Access), a Java API to EPICS Channel Access, SLC database device data, BPM and magnet data and control, EPICS and SLC history data, SLC modeling data.
5. Err - a platform and language independent (java & c++) error handling framework for creating and logging error messages.
6. global message: error or warning text message typically going to some log like cmlog, and viewed by a user through a browser like cmlog browser. A global message is on generated by a front-end computer or an application that thinks it's worth telling the world about. compare to local message
7. local message: a message generated by an application that is intended only for viewing by the application's user at that time. Eg "0 is not a permitted number of iterations to average bpm readings".

REFERENCES

1. XAL http://www.sns.gov/APGroup/appProg/xal/xal.htm
2. XML entry in Wikipedia http://en.wikipedia.org/wiki/XML
3. LCLS lattice files http://www-ssrl.slac.stanford.edu/lcls/linac/optics/
4. JAMA http://math.nist.gov/javanumerics/jama/

To add to this document

Add gold lattice to optics flowchart diagram
Each physics application
History/archiving
correlation plots