Overview
Camera Watchdog monitors the statuses of the FACET Profile Monitors, SIOCs, and POE Hubs, and attempts to revive cameras that become disconnected.
Key Features
- The Camera Watchdog creates Camera objects in MATLAB for each FACET Profile Monitor. Each Camera object stores data such as the Camera's name, location, and the number of times a reboot has been attempted.
- The Camera Watchdog also creates SIOC objects and POEHub objects for the relevant SIOCs and POE Hubs.
- Each Camera assigns itself a State based on its Connection status, Acquisition status (Acquiring or Idle), Trigger Mode (Sync in 1 or Free Run) and Array Rate (zero or nonzero). The Camera object is listening for updates from EPICS, so if one of these PVs changes, the Camera updates the State.
- If the Camera is in a disconnected State, it attempts to revive itself.
- If an SIOC or POE Hub is down, the Camera Watchdog can be used to revive it.
- Once a day, the Camera Watchdog creates a report and saves it to a MAT file.
States
The possible States for the Cameras are:
0 - The Camera is connected, acquiring, the array rate is nonzero, and the trigger mode is set to 'Sync in 1'.
1 - The Camera is disconnected.
2 - The Camera is connected and acquiring, but the array rate is zero.
3 - The Camera is connected, but it is idle.
4 - The Camera is connected, acquiring, and the array rate is nonzero, but the trigger mode is set to 'Free Run.'
5 - The Camera has an Alarm or its POE Hub is down.
The Camera object is periodically checking for updates from EPICS. If one of the relevant PVs changes, it updates the State. All State updates are recorded in a log file.
Rebooting
If a Camera is in State 1, it can attempt to reboot itself by calling its PowerCycle PV. It will only attempt to revive itself once. If revived, the Camera Watchdog will also jiggle the camera.
Additionally, if a Camera is in State 3, it can potentially put itself in acquiring mode by setting the Acquisition PV. If a Camera is in State 4, it can potentially set its trigger mode to 'Sync in 1' by setting the Trigger Mode PV.
Daily Report
Once a day, the Camera Watchdog creates a report of camera data for all of the FACET profile monitors. It captures the following data in a table and saves it to a MAT file:
Name, Model, PV, X Orientation, Y Orientation, Resolution, Exposure, Connection, Serial Number, and Number of Reboots
The daily reports and log files can be found here:
|
PyDM GUI
State indicators were added to the PyDM GUI in FACET Home to visually display the state of each profile monitor.
The colors for the indicators are:
State 0 - Green
State 1 - Grey
States 2, 3, or 4 - Yellow
State 5 - Purple
An additional column of indicators was also added to display the state of the POE for each camera.
The colors for these indicators are:
Green - Power on
Grey - Power off
Purple - Alarm status, POE is down
