View Source

Table of Contents

IRMIS Overview

"IRMIS is a collaborative effort between several EPICS sites to build a common Relational DataBase schema and a set of tools to populate and search an RDB that contains information about the operational EPICS IOCs installed at that site." IRMIS (the schema, crawler programs and UI) was developed by Don Dohan and Claude Saunders at APS. For general information and distributions see the IRMIS home page.

IRMIS @ SLAC

IRMIS is used at SLAC for the following purposes, from several different interfaces:

Data source for PV names for AIDA (nightly cron jobs)
Data source for element EPICS device names for LCLS_INFRASTRUCTURE (nightly cron jobs)
PV and PV Client lists and data (IRMIS gui)
IOC configuration and parameters (IOC Info APEX: https://seal.slac.stanford.edu/apex/mccqa/f?p=104:8)
IOC and application configuration data (IOC Info jsp app: https://seal.slac.stanford.edu/IRMISQueries )
EPICS camdmp application (APEX app: https://seal.slac.stanford.edu/apex/mccqa/f?p=103:4)
Archiver PV search application (APEX app: https://seal.slac.stanford.edu/apex/mccqa/f?p=259:8 )
Lists of IOCs and their PV populations (web page: http://mccas0.slac.stanford.edu/crawler/ioc_report.html)
ad hoc queries using TOAD, sqlplus, pl/sql, perl scripts, or other db query tool.
IRMIS crawler logs, with duplicate PV reports: http://www.slac.stanford.edu/grp/lcls/controls/sysGroup/report

Elements of the IRMIS database that have been adopted and modified for the controls software group at SLAC:

The Oracle PV schema
PV, ALH crawlers
the IRMIS gui
Other elements of the collaboration IRMIS installation include cabling, device and application schemas. We are not populating these now...but may in the future.

Elements modified or created at SLAC:

IOC boot syntax adaptations in the PV crawler and IOC Info crawler (adopted from SNS) and schema
Special application-oriented tables in the IRMIS schema, and their population scripts (for example for EPICS camdmp and the archiver PV viewer app)
Addition of config file tree structure crawling in the ALH crawler
New crawlers for Channel Watcher and Channel Archiver
New PV Client viewer addition to the IRMIS Desktop
New guis

UI and DATABASE QUERYING

IRMIS GUI
This is a java UI for the IRMIS Oracle database, developed by Claude Saunders of the EPICS collaboration, can be invoked in 3 ways:

from the lclshome edm display:
click the “IRMIS…” button
from a Solaris or Linux workstation:
run this script:
irmisUI

The gui paradigm is a set of “document types”; click the File/New Document menu for the list. Right now there are only 2 available for use at SLAC:

idt::pv – Search for lists of PVs and IOCs. This is the most useful interface, and it comes up upon application startup.
idt::pvClient – Search for PV Client lists (alarm handler, archiver, channel watcher)

Query results can be saved to an ascii file for further processing.

IOC Parameters APEX application
This is an APEX application showing IOC configuration data, and various operational parameter snapshots (obtained live nightly using caget).
https://seal.slac.stanford.edu/apex/mccqa/f?p=104:8

IOC Info query application
This is a jsp web application containing data about IOCs and applications, and their configurations.
https://seal.slac.stanford.edu/IRMISQueries/

EPICS camdmp APEX application
Various reports listing PVs and their module and channel connections.
https://seal.slac.stanford.edu/apex/mccqa/f?p=103:4

Archiver PV query APEX application
Lists of archived PVs by IOC application, IOC.
https://seal.slac.stanford.edu/apex/mccqa/f?p=259:8

IOC list report (run nightly)
This html report is created nightly by the LCLS PV Crawler
http://mccas0.slac.stanford.edu/crawler/ioc_report.html

PV Crawler logs (which include the duplicate PV lists)
http://www.slac.stanford.edu/grp/lcls/controls/sysGroup/report/
a subset here: http://www.slac.stanford.edu/cgi-bin/lwgate/CONTROLS-SOFTWARE-REPORTS/archives

SQL querying
A view has been created to ease sql querying for PV lists. This view combines data from the IOC_BOOT, IOC, REC and REC_TYPE tables. It selects currently loaded PVs, where IOC_BOOT.CURRENT_LOAD = 1, with the latest IOC boot date captured.

CURR_PVS view has all currently loaded PVs

OPERATIONAL and SUPPORT DETAILS

At SLAC, in a nutshell the PV and PV client crawlers update:

all IRMIS tables in MCCQA. All IRMIS UIs query data on MCCQA: PV and IOC data, and PV Client data.
3 production tables on MCCO, which are used by other system – AIDA, BSA applications

Oracle schemas and accounts
The IRMIS database schema is installed in 4 SLAC Oracle instances:

MCCQA IRMISDB schema: Production for all gui’s and applications. Contains data populated nightly by perl crawler scripts from production IOC configuration files. Data validation is done following each load.
MCCO IRMISDB schema: contains production data, currently for 3 tables only: bsa_root_names, devices_and_attributes, curr_pvs. MCCQA data is copied to MCCO once it has been validated. So MCCO is as close as possible to pristine data at all times.
SLACDEV contains the 3 schemas which are used for developing, testing and staging new features before release to production on MCCQA (see accounts below)
1. IRMISDB – sandbox for all kinds of development, data sifting, etc. Not refreshed from prod, or not very often.
2. IRMISDB_TEST – testing for application implementation. Refreshed from prod at the start of a development project.
3. IRMISDB_STAGE – staging for testing completed applications against recently-refreshed production data.
SLACPROD IRMISDB schema: this is obsolete, but will be kept around for awhile (6 months?) as a starting point in case the DB migration goes awry somehow.

Other Oracle accounts

IRMIS_RO – read-only account (not used much yet – but available)
IOC_MGMT – created for earlier IOC info project with a member of the EPICS group which is not active at the moment- new ioc info work is being done using IRMISDB

For passwords see Judy Rock or Poonam Pandey or Elie Grunhaus.

As of September 15, all crawler-related shell scripts and perl scripts use the getPwd script (Greg White) to get the latest Oracle password. Oracle passwords must be changed every 6 months; new passwords will be given to Ken Brobeck to update the secure master password files at password change time.

** The IRMIS GUI and the JSP application still use hardcoded passwords. These must be changed “manually” at every password change cycle.

Database structure: see schema diagram below. (this diagram excludes the EPICS camdmp structure, which is documented separately here: <url will be supplied>)

Crawler scripts
The PV crawler is run once for each IOC boot directory structure. The LCLS PV crawler (runLCLSPVCrawlerLx.bash) runs the crawler only once. It is separate to enable it to be run on a different schedule and different host which can see the LCLS IOC directories. Also, the crawler code has been modified to be LCLS-specific; it is a different version than the SLAC PV crawler.

The SLAC PV crawler (runSLACPVCrawler.csh) runs the crawler a couple of times to accommodate the various CD IOC directory structures.

cron jobs

LCLS side: laci on lcls-daemon2: runLCLSPVcrawlerLx.bash: crawls LCLS PVs and creates lcls-specific tables (bsa_root_names, devices_and_attributes), copies LCLS client config files to dir where CD client crawlers can see them.
LCLS side: laci on lcls-daemon2: caget4curr_ioc_device.bash: does cagets to populate curr_ioc_devices for the IOC Info APEX app. Run separately from the crawlers because cagets can hang unexpectedly – they are best done in an isolated script!
CD side: cddev on slcs2: runAllCDCrawlers.csh: runs CD PV crawler and all client crawlers, data validation, and sync to MCCO.
FACET side: flaci on facet-daemon2: runFACETPVcrawlerLx.bash: crawls FACET PVs, copies FACET client config files to dir where CD client crawlers can see them.

PV crawler operation summary
For the location of the crawler scripts, see Source code directories below.
Basic steps as called by cron scripts are:

run FACET pv crawler to populate MCCQA tables
run LCLS pv crawlers to populate MCCQA tables
run CD pv and pv client crawlers to populate MCCQA tables
run Data Validation for PV data in MCCQA
if Data Validation returns SUCCESS, run synchonization of MCCQA data to selected (3 only at the moment) MCCO tables.
run caget4curr_ioc_device to populate caget columns of curr_ioc_device

** For PV Crawlers: the crawler group for any given IOC is determined by its row in the IOC table. The system column refers to the boot group for the IOC, as shown below.

* The PV client crawlers load all client directories in their config files; currently includes both CD and LCLS.

LOGFILES, Oracle audit table

Log filenames are created by appending a timestamp to the root name shown in the tables below.

The major steps in the crawler jobs write entries into the Oracle CONTROLS_GLOBAL.DATA_VALIDATION_AUDIT table. Each entry has these attributes:

Instance
Schema
Process
Stage
Status
Message
TOD (time of day)
(see below for details on querying this table)

Descriptions of the MAIN scripts (there other subsidiary scripts as well):

these are all ultimately invoked from the cron jobs shown above; the cron scripts call the others.

BLUE script names are on the CD side

GREEN script names are on the LCLS side

PURPLE script names are on the FACET side

{table:border=1}
{tr}
{th}script name{th}
{th}descr{th}
{tr}
{td}{color:#0000ff}runAllCDCrawlers.csh{color}\\
{color:#0000ff}{_}in /afs/slac/g/cd/soft/tools/irmis/cd_script{_}{color}{td}
{td}runs
* SLAC PV crawler
* all client crawlers
* rec client cleanup
* data validation for all current crawl data (LCLS and CD)
* sync to MCCO
\\
Logfile: /nfs/slac/g/cd/log/irmis/pv/ CDCrawlerAll.log
{td}
{tr}
{td}{color:#0000ff}runSLACPVCrawler.csh{color}
\\
{color:#0000ff}{_}in /afs/slac/g/cd/soft/tools/irmis/cd_script{_}{color}{td}
{td}run by runAllCDCrawlers.csh: crawls NLCTA IOCs (previously handled PEPII IOCs)
\\
The PV crawler is run 4 times within this script to accommodate the various boot directory structures:
|| system || IOCs crawled || log files ||
| NLCTA | IOCs booted from $CD_IOC on gtw00. \\
Boot dirs mirrored to /nfs/mccfs0/u1/pepii/mirror/cd/ioc for crawling | /nfs/slac/g/cd/log/irmis/pv/pv_crawlerLOG_ALL.\* \\
/tmp/pvCrawler.log |
| TR01 | TR01 only (old boot structure) | /nfs/slac/g/cd/log/irmis/pv/pv_crawlerLOG_TR01.\* \\
/tmp/pvCrawler.log |
{td}
{tr}
{tr}
{td}{color:#008000}runLCLSPVCrawlerLx.bash{color}
\\
{color:#008000}{_}in /usr/local/lcls/tools/irmis/script/_{color}{td}
{td}crawls LCLS IOCs
|| system || IOCs crawled || log files ||
| LCLS | LCLS PRODUCTION IOCs. all IOC dirs in \\
$IOC /usr/local/lcls/epics/iocCommon | /u1/lcls/tools/crawler/pv/pv_crawlerLOG_LCLS.timestamp \\
/tmp/ LCLSPVCrawl.log |
{td}
{tr}
{tr}
{td}{color:#800080}runFACETPVCrawlerLx.bash{color}
\\
{color:#800080}{_}in /usr/local/facet/tools/irmis/script/_{color}{td}
{td}crawls FACET IOCs
|| system || IOCs crawled || log files ||
| FACET | FACET PRODUCTION IOCs. all IOC dirs in facet $IOC /usr/local/facet/epics/iocCommon | /u1/facettools/crawler/pv/pv_crawlerLOG_LCLS.timestamp\\/tmp/FACETPVCrawl.log |
{td}
{tr}
{tr}
{td}{color:#0000ff}runClientCrawlers.csh{color}
\\
{color:#0000ff}{_}in /afs/slac/g/cd/soft/tools/irmis/cd_utils{_}{color}
\\
\\
{color:#0000ff}individual client crawler scripts are here{color}
\\
{color:#0000ff}{_}in /afs/slac/g/cd/soft/tools/irmis/cd_script{_}{color}{td}
{td}
Runs PV client crawlers in sequence. *LCLS client config files are all scp-ed to /nfs/slac/g/cd/crawler/lcls \*Configs for crawling*
|| Client crawler || config file with list of directories/files crawled || log files ||
| ALH | /afs/slac/g/cd/tools/irmis/cd_config/SLACALHDirs | /nfs/slac/g/cd/log/irmis/alh/alh_crawlerLOG.\* \\
/tmp/pvCrawler.log |
| Channel Watcher (CW) | /afs/slac/g/cd/tools/irmis/cd_config/SLACCWdirs.lst | /nfs/slac/g/cd/log/irmis/cw/cw_crawlerLOG.\* \\
/tmp/pvCrawler.log |
| Channel Archiver (CAR) | /afs/slac/g/cd/tools/irmis/cd_config/SLACCARfiles.lst \\
note: all CAR files for LCLS are scp-ed over from lcls-archsrv \\
to /nfs/slac/g/cd/crawler/lclsCARConfigs for crawling | /nfs/slac/g/cd/log/irmis/car/car_crawlerLOG.\* \\
/tmp/pvCrawler.log |
*Also runs* load_vuri_rec_client_type.pl for clients that don't handle vuri_rec_client_type records (sequence crawler only, at the moment)
{td}
{tr}
{tr}
{td}{color:#0000ff}runRecClientCleanup.csh{color}
\\
{color:#0000ff}{_}in in /afs/slac/g/cd/soft/tools/irmis/cd_script{_}{color}{td}
{td}deletes all non-current rec client, rec client flag and vuri rows. Logs to /nfs/slac/g/cd/log/irmis/client_cleanupLOG.\*{td}
{tr}
{tr}
{td}{color:#008000}run_find_devices.bash{color}
\\
{color:#008000}{_}in /usr/local/lcls/tools/irmis/script/_{color}{td}
{td}for LCLS and FACET only, populate the devices_and_attributes table, a list of device names and attributes based on the LCLS PV naming convention. For PV DEV:AREA:UNIT:ATTRIBUTE, DEV:AREA:UNIT is the device, ATTRIBUTE is the attribute.{td}
{tr}

{tr}\\
{td}{color:#008000}run_load_bsa_root_names.bash{color}\\
{color:#008000}{_}in /usr/local/lcls/tools/irmis/script/_{color}{td}{td}
loads bsa_root_names table by running stored procedure LOAD_BSA_ROOT_NAMES. LCLS and FACET names.{td}
{tr}
{tr}
{td}{color:#008000}ioc_report.bash{color}
\\
{color:#008000}{_}in /usr/local/lcls/tools/irmis/script/_{color}
{color:#800080}ioc_report-facet.bash{color}
\\
{color:#800080}{_}in /usr/local/facet/tools/irmis/script/_{color}{td}
{td}runs at the end of the LCLS PV crawl, which is last, creates the web ioc report:
\\
[http://www.slac.stanford.edu/grp/cd/soft/database/reports/ioc_report.html]{td}
{tr}
{tr}
{td}{color:#0000ff}updateMaterializedViews.csh{color}
\\
{color:#0000ff}{_}in /afs/slac/g/cd/soft/tools/irmis/cd_script{_}{color}{td}
{td}refreshes materialized view from curr_pvs{td}
{tr}
{tr}
{td}{color:#008000}findDupePVs.bash{color}
\\
{color:#008000}{_}in /usr/local/lcls/tools/irmis/script/_{color}
{color:#800080}findDupePVs-all.bash{color}
\\
{color:#800080}{_}in /usr/local/facet/tools/irmis/script/_{color}{td}
{td}finds duplicate PVs for reporting to e-mail
\\
(the \--all version takes system as a parameter){td}
{tr}
{tr}
{td}{color:#008000}copyClientConfigs.bash{color}
\\
{color:#008000}{_}in /usr/local/lcls/tools/irmis/script/_{color}
{color:#800080}copyClientConfigs-facet.bash{color}
\\
{color:#800080}{_}in /usr/local/facet/tools/irmis/script/_{color}{td}
{td}copies alh, cw and car config files to /nfs for crawling by the client crawler job{td}
{tr}
{tr}
{td}{color:#008000}copyClientConfigs.bash{color}
\\
{color:#008000}{_}in /usr/local/lcls/tools/irmis/script/_{color}
{color:#800080}copyClientConfigs-facet.bash{color}
\\
{color:#800080}{_}in /usr/local/facet/tools/irmis/script/_{color}{td}
{td}copies alh, cw and car config files to /nfs for crawling by the client crawler job{td}
{tr}
{tr}
{td}{color:#008000}refresh_curr_ioc_device.bash{color}
\\
{color:#008000}{_}in /usr/local/lcls/tools/irmis/script/_{color}{td}
{td}refreshes the curr_ioc_device table with currently booted info from ioc_device (speeds up query in the Archiver PV APEX app).
\\
Also see caget4curr_ioc_device.bash below.{td}
{tr}
{tr}
{td}{color:#008000}find_LCLSpv_count_changes.bash{color}
\\
{color:#008000}{_}in /usr/local/lcls/tools/irmis/script/_{color}
{color:#800080}find_pv_count_changes-all.bash{color}
\\
{color:#800080}{_}in /usr/local/facet/tools/irmis/script/_{color}{td}
{td}finds the IOCs with PV counts that changed in the last LCLS crawler run. This is reported in the logfile, and in the daily e-mail.
\\
(the \--all version takes system as a parameter){td}
{tr}
{tr}
{td}{color:#008000}populate_dtyp_io_tab.bash
\\
populate_io_curr_pvs_and_fields.bash{color}
\\
{color:#008000}run_load_hw_dev_pvs.bash{color}
\\
{color:#008000}run_parse_camac_io.bash{color}
\\
{color:#008000}{_}in /usr/local/lcls/tools/irmis/script/_{color}{td}
{td}set of script to populate various tables for the EPICS camdmp APEX application.{td}
{tr}
{tr}
{td}{color:#008000}write_data_validation_row.bash{color}
\\
{color:#008000}{_}in /usr/local/lcls/tools/irmis/script/_{color}
{color:#0000ff}write_data_validation_row.csh{color}
\\
{color:#0000ff}{_}in /afs/slac/g/cd/soft/tools/irmis/cd_script{_}{color}
{color:#800080}write_data_validation_row-facet.bash{color}
\\
{color:#800080}{_}in /usr/local/facet/tools/irmis/script/_{color}{td}
{td}writes a row to the controls_global.data_validation_audit table (all environments){td}
{tr}
{tr}
{td}runDataValidation.csh
\\
runDataValidation-facet.csh
\\
_in /afs/slac/g/cd/soft/tools/irmis/cd_script{_}{td}
{td}runs IRMISDataValidation.pl -- CD and LCLS PV data validation{td}
{tr}
{tr}
{td}{color:#b00e00}runSync.csh{color}
\\
{color:#b00e00}{_}in /afs/slac/g/cd/soft/tools/irmis/cd_script{_}{color}{td}
{td}runs data replication to MCCO, currently these objects ONLY:
* curr_pvs
* bsa_root_names
* devices_and_attributes{td}
{tr}
{tr}
{td}{color:#008000}caget4curr_ioc_device.bash{color}
\\
{color:#008000}{_}in /usr/local/lcls/tools/irmis/script/_{color}{td}
{td}does cagets to obtain IOC parameters for the curr_ioc_device table. Run as a separate cron job from the crawlers because cagets
\\
can hang unexpectedly -- they are best done in an isolated script\!{td}
{tr}
{table}