Overview
For the purposes of this guide, it is assumed that the user is logged onto a host machine belonging to the PCDS network. In this case the host is psdev. For those unfamiliar with the PCDS network, psdev serves as a gateway to the network and it allows one to log onto the ATCA host machine, which is the computer connected to the ATCA crate where the carrier boards are installed.
In terms of facility, the laser locker referred to in this guide is located in the ASC timing lab. For the ASC facility, the ATCA host is ctl-hpl-srv01. What we expect to find in the laser locker top directory on the ATCA host is the compiled module itself and everything it comes with natively from the git repo.
Note that the objective is to do a static build for the laser locker binary either on the ATCA host itself (if configured adequately) or on another host with a compatible operating system, followed by porting the built code to the ATCA host.
| Laser Locker Tag | Target OS | Build On Host | Run On Host | Non-EPICS Dependencies | EPICS Dependencies |
|---|---|---|---|---|---|
| R1.5.0-1.0 | RHEL7 | aird-b50-srv01 | ctl-hpl-srv01 | cpsw, yaml-cpp, boost, hpstpr, llrf, commonATCA | iocAdmin, miscUtils, asyn, autosave, seq, caPutLog, yamlLoader, ycpswasyn, crossbarControl, tprPattern, tprTrigger, ATCACommon |
To see what tags are used for the various laser locker dependencies, check out the build guide: PCDS Laser Locker - Build Guide
How to check out and build the IOC
To perform the checkout and build, keep in mind that:
- You will need to be on a host with git accessibility to successfully check out the code from the appropriate SLAC repository.
- You will need to build on a host with AFS access so that the laser locker dependencies --such as EPICS modules and non-EPICS packages-- are accessible.
- The server used to develop and build the laser locker binary for tag R1.5.0-1.0 was aird-b50-srv01. For now, no other hosts are known to be properly configured for this application and hence no other hosts can be recommended besides aird-b50-srv01.
Use the eco tools to check out the preferred laser locker tag or branch to match your program needs/requirements and computing environment. For instance, for a PCDS RHEL7 environment, the appropriate/latest tag (as of 06/24/2022) is R1.5.0-1.0.
$ eco -m laserLocker R1.5.0-1.0
Once you check out the source code, build as shown below:
$ cd laserLocker/R1.5.0-1.0 $ make CROSS_COMPILER_TARGET_ARCHS=
Upon successful completion of the previous steps, and assuming the build took place on a non-PCDS host, simply copy over the project directory to an accessible PCDS server.
To only build the laser locker binary for the host operating system (i.e. RHEL7 in this case), we issue a make CROSS_COMPILER_TARGET_ARCHS= command as shown above, otherwise (i.e. make) the build system will compile a binary for the host operating system and cross-compile a binary for LinuxRT.
As briefly mentioned in the overview above, for more details on the laser locker dependencies and how those were built, see the link below. Note that the reference below is not necessary if you simply want to build the laser locker binary and set up an IOC somewhere on the PCDS network.
PCDS Laser Locker - Build Guide
How to create a new branch for the IOC
For any necessary changes in the source code or startup script, ensure that a new branch is created first. Follow the process below:
- With tag R1.5.0-1.0 as your starting point, create a new branch with the branch, switch or checkout commands
- Perform a git push with the –set-upstream option to set the remote repo for the new branch
- Continue to perform git commit locally on the new branch
- Simply use a git push origin command on subsequent pushes of the new branch to the remote repo
TID will be responsible for the final merge operation, once the user confirms the branch's readiness to merge.
How to load the firmware image
Obtain the .mcs firmware image from the firmware engineer. Ensure the .mcs file is copied in the firmware/ directory of the laser locker home on the ATCA host. After that, follow the process in the link below to reprogram and reboot the FPGA. Note that for the specific ATCA crate connected to ctl-hpl-srv01, the laser locker is in Slot 2 of the crate and the corresponding shelf manager is shm-hpl-atca01.
Please ensure that the firmware's .msc file is captured and added to the git repository.
HowTo - Reprogram AMC's Carrier FPGA With FirmwareLoader
How to handle the firmware YAML files
Obtain the yaml file tarball from the firmware engineer. Copy the tarball in the laser locker home directory and untar the archive in there. Issue the following commands to see the git hash for the specific firmware image. For PCDS, you will need to do that on psdev. Note that we always need to associate the firmware git hash with the specific yaml files that correspond to that build.
-bash-4.2$ source /afs/slac/g/reseng/IPMC/env.sh -bash-4.2$ amcc_dump_bsi --all shm-hpl-atca01 ================================================================================ | BSI: shm-hpl-atca01/1/CEN (shm-hpl-atca01/1/4) | ================================================================================ | BSI: shm-hpl-atca01/2/CEN (shm-hpl-atca01/2/4) | BSI Ld State: 3 (READY) BSI Ld Status: 0x00000000 (SUCCESS) BSI Version: 0x0103 = 1.3 MAC 0: 08:00:56:00:4f:c4 MAC 1: 08:00:56:00:4f:c5 MAC 2: 08:00:56:00:4f:c6 MAC 3: 08:00:56:00:4f:c7 DDR status: 0x0003: MemErr: F, MemRdy: T, Eth Link: Up Enet uptime: 187948 seconds FPGA uptime: 187949 seconds FPGA version: 0x01000000 BL start adx: 0x04000000 Crate ID: 0x0002 ATCA slot: 2 AMC 0 info: Aux: 01 Ser: be00000135448e70 Type: 07 Ver: C03 BOM: 01 Tag: 07 AMC 2 info: Aux: 01 Ser: f900000135665570 Type: 07 Ver: C03 BOM: 03 Tag: 02 GIT hash: 94011158d3d7c51239481383b6bd718b32440398 FW bld string: 'AmcCarrierMrEthV2: Vivado v2021.2, rdsrv318 (Ubuntu 20.04.4 LTS), Built Fri 03 Jun 2022 11:41:44 AM PDT by ruckman' --------------------------------------------------------------------------------
From the above, we conclude that the git hash for the firmware build is 94011158. After extracting this information, we rename the expanded yaml directory to yaml-94011158. This name will be referenced later in the laser locker IOC's startup script. Once confirmed that the YAML files are correct and function as expected, please ensure that those are added to the corresponding git repository to make them available to all other users of the specific tag.
As of 06/24/2022, the latest tarball is the one shown above (i.e. git hash 94011158) and is currently available in the laser locker git repository (tag R1.5.0-1.0).
How to configure the IOC
The primary interface available to us for setting up, configuring and booting the IOC is the startup script st.cmd. In this section, we will briefly discuss some of the settings and commands we see in st.cmd of laser locker R1.5.0-1.0.
For convenience, it is useful to define EPICS environment variables in the form of strings to represent paths, devices, facilities, IP addresses and other useful information and settings.
$ epicsEnvSet("EPICS_CA_MAX_ARRAY_BYTES", "1000000")
$ epicsEnvSet("DEVICE","OSC")
$ epicsEnvSet("SECTOR","ASC01")
$ epicsEnvSet("STATION","0")
$ epicsEnvSet("IOCPV", "SIOC:ASC01:LS01")
$ epicsEnvSet("IOC", "sioc-asc01-ls01")
$ epicsEnvSet("LOCA", "ASC01")
$ epicsEnvSet("IOC_UNIT", "ASC01")
$ epicsEnvSet("INST", "0")
$ epicsEnvSet("YAML_DIR", "yaml-94011158")
$ epicsEnvSet("FPGA_IP", "10.0.2.102")
EPICS database definition files are loaded into the EPICS shell with the following command:
## Register all support components
dbLoadDatabase("dbd/laserLocker.dbd")
Database records are loaded in this way:
# load records from the laser locker cpsw substitutions database
dbLoadRecords("db/laserLocker_cpsw.db", "P=${DEVICE}:${SECTOR}:${STATION}, PORT=ATCA2")
Load the FPGA hierarchy YAML file:
cpswLoadYamlFile("$(YAML_DIR)/000TopLevel.yaml", "NetIODev", "", "$(FPGA_IP)")
Load the FPGA register configuration and initialization YAML file:
cpswLoadConfigFile("$(YAML_DIR)/config/defaultsGenRTMV2.yaml", "mmio", "")
For more information on how to use the yamlLoader module and in particular the above two API calls, see the README file below.
$ vim /afs/slac/g/lcls/epics/R7.0.3.1-1.0/modules/yamlLoader/yamlLoader-git/README
Next, configure the tprTrigger driver. The tprTrigger is an epics module which provides a PV interface for trigger configuration and monitoring.
tprTriggerAsynDriverConfigure("trig", "mmio/AmcCarrierCore")
For more information on how to configure the tprTrigger driver, see the README file below.
$ vim /afs/slac/g/lcls/epics/R7.0.3.1-1.0/modules/tprTrigger/R1.5.0-1.0/README
A database called subR_laserLocker.db is loaded for specific PVs of interest. This EPICS record contains algorithms implemented in C. See below how to perform these calls.
dbLoadRecords("db/subR_laserLocker.db", "NAME=OSC:ASC01:0:M:ATOP:ACORE:SM:loPllAmplitude:Rd, DESC=L0Amplitude, INAM=LL16bitInit, SNAM=LL16bitAmpl, SCAN=Passive, PP=CPP")
dbLoadRecords("db/subR_laserLocker.db", "NAME=OSC:ASC01:0:M:ATOP:ACORE:SM:loPllPhase:Rd, DESC=L0PhaseErr, INAM=LL16bitInit, SNAM=LL16bitPhase, SCAN=Passive, PP=CPP")
dbLoadRecords("db/subR_laserLocker.db", "NAME=OSC:ASC01:0:M:ATOP:ACORE:SM:loPllLocked:Rd, DESC=L0PLLLocked, INAM=LL16bitInit, SNAM=LL16bitBoolean, SCAN=Passive, PP=CPP")
dbLoadRecords("db/subR_laserLocker.db", "NAME=OSC:ASC01:0:M:ATOP:ACORE:SM:clockPllPhase:Rd, DESC=ClockPhaseErr, INAM=LL16bitInit, SNAM=LL16bitPhase, SCAN=Passive, PP=CPP")
dbLoadRecords("db/subR_laserLocker.db", "NAME=OSC:ASC01:0:M:ATOP:ACORE:SM:clockPllAmplitude:Rd, DESC=ClockAmplitude, INAM=LL16bitInit, SNAM=LL16bitAmpl, SCAN=Passive, PP=CPP")
dbLoadRecords("db/subR_laserLocker.db", "NAME=OSC:ASC01:0:M:ATOP:ACORE:SM:clockPllLocked:Rd, DESC=ClockPLLLocked, INAM=LL16bitInit, SNAM=LL16bitBoolean, SCAN=Passive, PP=CPP")
dbLoadRecords("db/subR_laserLocker.db", "NAME=OSC:ASC01:0:M:ATOP:ACORE:SM:Status_13:Rd, DESC=LaserPhaseCon, INAM=LL16bitInit, SNAM=LL16bitPhase, SCAN=Passive, PP=CPP")
dbLoadRecords("db/subR_laserLocker.db", "NAME=OSC:ASC01:0:M:ATOP:ACORE:SM:Status_14:Rd, DESC=LaserPhaseErr, INAM=LL16bitInit, SNAM=LL16bitPhase, SCAN=Passive, PP=CPP")
dbLoadRecords("db/subR_laserLocker.db", "NAME=OSC:ASC01:0:M:ATOP:ACORE:SM:Status_15:Rd, DESC=LaserPhaseSet, INAM=LL16bitInit, SNAM=LL16bitPhase, SCAN=Passive, PP=CPP")
dbLoadRecords("db/subR_laserLocker.db", "NAME=OSC:ASC01:0:M:ATOP:ACORE:SM:Status_16:Rd, DESC=unlockdetect, INAM=LL16bitInit, SNAM=LL16bitBoolean, SCAN=Passive, PP=CPP")
dbLoadRecords("db/subR_laserLocker.db", "NAME=OSC:ASC01:0:M:ATOP:ACORE:SM:Status_17:Rd, DESC=ADC00_phase, INAM=LL16bitInit, SNAM=LL16bitPhase, SCAN=Passive, PP=CPP")
dbLoadRecords("db/subR_laserLocker.db", "NAME=OSC:ASC01:0:M:ATOP:ACORE:SM:Status_18:Rd, DESC=ADC00_max, INAM=LL16bitInit, SNAM=LL16bitAmpl, SCAN=Passive, PP=CPP")
dbLoadRecords("db/subR_laserLocker.db", "NAME=OSC:ASC01:0:M:ATOP:ACORE:SM:Status_19:Rd, DESC=ADC01_max, INAM=LL16bitInit, SNAM=LL16bitAmpl, SCAN=Passive, PP=CPP")
Now, run YCPSWASYN to create EPICS Records from the dictionary and substitution files. If we have selected to auto-generate the PVs, then all registers will be mapped to corresponding EPICS records. The parameters in YCPSWASYNConfig() are described below.
YCPSWASYNConfig( Port Name, Root Path, Record Name Prefix, Use DB Autogeneration, Load dictionary, Named Root )
where:
Port Name → The name given to this port driver
Root Path → OPTIONAL: Root path to start the generation. If empty, the Yaml root will be used
Record name Prefix → Record name prefix
Use DB Autogeneration → Set to 1 for autogeneration of records from the YAML definition. Set to 0 to disable it
Load dictionary → Dictionary file path with registers to load. An empty string will disable this function
Named Root → OPTIONAL: Name of the root to be used, only necessary to support multiple ATCA blades on a single IOC.
Hence, in this case, YCPSWASYNConfig() is called in the following way:
YCPSWASYNConfig("$(PORT)", "", "${PREFIX}", "${AUTO_GEN}", "$(YAML_DIR)/${DICT_FILE}", "")
Follow Section 1 in the README file below (Building the IOC application with the ATCACommon module) to set up the Asyn drivers and see how to load the DaqMux PVs:
$ vim /afs/slac/g/lcls/epics/modules/R7.0.3.1-1.0/ATCACommon/<version>/README
The drivers are configured as shown (see also README above):
crossbarControlAsynDriverConfigure("crossbar", "mmio/AmcCarrierCore/AxiSy56040")
cpswATCACommonAsynDriverConfigure("atca", "mmio", "")
cpswDebugStreamAsynDriverConfigure("atca_str0", 0x4000, "header_enabled", "Stream0", "Stream1", "Stream2", "Stream3")
cpswDebugStreamAsynDriverConfigure("atca_str1", 0x4000, "header_enabled", "Stream4", "Stream5", "Stream6", "Stream7")
Before ATCACommon was created, crossbarControl was an independent module. With ATCACommon added to the IOC, all previous references of crossbarControl (configure/RELEASE, Db/Makefile, src/Makefile) should be removed and new ones should be added according to the README file above.
To have PVs available, we need to load the EPICS database with the respective PVs in st.cmd. In general, the only exception comes with EPICS 7 pvAccess and pvData that don't use the channel access protocol. With them, you can create EPICS 7 PVs directly from the C++ code without the need for an EPICS database file. However, the Laser Locker module has no code written for pvAccess and pvData.
How to use autosave & restore
EPICS - Summary On Autosave/Restore For Modules
As of 06/24/2022, the UED autosave/restore files were leveraged to create the boot state for the PCDS laser locker.
How PV names are handled by module ycpswAsyn
EPICS - Automatic Derivation Of PV Names In Modules
EPICS - Manual Derivation Of PV Names In Modules
As of 06/24/2022, the PCDS laser locker tag R1.5.0-1.0 uses a manual derivation of PVs. The specific mechanism in place uses (1) a dictionary file that defines a mapping between the YAML paths to the registers of interest and user-selected strings representing the Asyn parameter names associated with those registers and (2) a substitutions file that defines what EPICS record templates to use for each of those registers. Once all that is in place, the compiler will create an EPICS database file containing the appropriate records to match the register types we want to access.
How to start the IOC
To start the IOC, we simply need to run the startup script we set up for the specific facility. For instance, for the ASC lab, switch to directory iocBoot/sioc-asc01-ls01/ and execute the startup script st.cmd:
$ cd iocBoot/sioc-asc01-ls01/ $ ./st.cmd
Many of the PVs are set using dbpf() calls in st.cmd. Please use the above startup script in sioc-asc01-ls01 as a reference to create your own st.cmd.
An alternative way to start the IOC is by using a screen session. This allows multiple users who work collaboratively to stop and start the IOC as needed. See instructions below.
HowTo - Using The Screen Command To Launch An IOC
How to exit the IOC
From the IOC shell, we simply type exit to quit the IOC.
How to launch the engineering GUI
See instructions below on how to invoke the laser locker GUI on the PCDS network.
PCDS - Running The EDM GUI For Laser Locker
How to launch the DaqMux GUI
DaqMux is a capability to multiplex multiple data streams that get generated in FPGA-land and send them to software for visualizing and analysis. On the ATCA host or a different host on the same subnet as the ATCA host, set up the environment for the DaqMUX PyDM GUI as shown below:
$ source /reg/g/pcds/pyps/conda/py36env.sh $ export PYDM_PVA_LIB=PVAPY
Ensure the laser locker IOC is running and then check out ATCA_PyDM_Screens:
$ git clone git@github.com:cristinasewell/ATCA_PyDM_Screens.git
Check the stream data type used:
$ caget OSC:ASC01:0:STR0:STREAM_TYPE0
If the type is not int16/SHORT, please ensure the PVs are set correctly:
$ caput OSC:ASC01:0:STR0:STREAM_TYPE0 "int16 -SHORT" $ caput OSC:ASC01:0:STR0:STREAM_TYPE1 "int16 -SHORT" $ caput OSC:ASC01:0:STR0:STREAM_TYPE2 "int16 -SHORT" $ caput OSC:ASC01:0:STR0:STREAM_TYPE3 "int16 -SHORT" $ caput OSC:ASC01:0:STR1:STREAM_TYPE0 "int16 -SHORT" $ caput OSC:ASC01:0:STR1:STREAM_TYPE1 "int16 -SHORT" $ caput OSC:ASC01:0:STR1:STREAM_TYPE2 "int16 -SHORT" $ caput OSC:ASC01:0:STR1:STREAM_TYPE3 "int16 -SHORT"
Launch the DaqMux PyDM GUI:
$ pydm -m "DEVICE=ASC01:ASC01,LOCA=ASC01,IOC_UNIT=ASC01,INST=0" ATCA_PyDM_Screens/atcaLLRF.py
The above way of invoking the GUI will provide the user access to the timing/triggering screens as well.
How to configure DaqMux
By configuring DaqMux, we mean configuring the DaqMux FPGA registers in the firmware. At a minimum, you will need to follow the steps in Section 3 of the README file below to configure the waveforms:
$ vim /afs/slac/g/lcls/epics/modules/R7.0.3.1-1.0/ATCACommon/<version>/README
It is also important to confirm that all steps in the above README file have taken place at some point to ensure that all pieces are there prior to using the GUI.
If no plots are shown in the main window after invoking the GUI, select the DaqMux Settings Setup tab at the top of main window, then select the Stream Settings tab and click the Trigger button for all 8 channels. Check to see if this resolved the issue and if plots are displayed in the main window.
Overview
Content Tools