...

• TMO: /cds/group/pcds/epics/ioc/common/andorCam/R1.01.31/children/build/iocBoot/ioc-tmo-andor1/edm-ioc-tmo-andor1.cmd
• RIX: /cds/group/pcds/epics/ioc/common/andorCam/R1.01.31/children/build/iocBoot/ioc-rix-dir-andor/edm-ioc-rix-dir-andor.cmd
         /cds/group/pcds/epics/ioc/common/andorCam/R1.01.31/children/build/iocBoot/ioc-rix-vls-andor/edm-ioc-rix-vls-andor.cmd
         /cds/group/pcds/epics/ioc/common/andorCam/R1.01.31/children/build/iocBoot/ioc-rix-norm-andor/edm-ioc-rix-norm-andor.cmd

...

Power cycle procedure for camera/USB fiber extender

NOTE: This may not be necessary if one uses Dan's ~ddamiani/reset_andor.sh script described below.

Occasionally after power cycling the camera or everything if there was a power outage the USB fiber extender will come back in a bad state where it won't communicate with the camera. One time we saw these errors in /var/log/messages when the VLS camera broke:

...

1. Power off the camera (NOTE: to do this I believe you have unplug the power supply, since the power supply just has a rocker switch which I think selects between standard-cooling/deep-cooling?).  The only LED that gives any indication of power is a green one inside the power supply, visible only if you put your eye somewhere near the deep-cooling ("||") side of the rocker switch at a specific angle.
2. Power off the camera side of the extender
3. Unplug the server-side extender USB cable from daq-rix-andor-01. The DIR andor extender has the black USB cable (top USB port) and the VLS andor extender is the silver cable (bottom USB port).
4. Re-plug the server-side extender USB cable into daq-rix-andor-01.
5. Power on the camera side of the extender. You should see the link led turn on on the extender, and the host led blink slowly.
6. Power on the camera. The host led on the camera side extender show turn on solid now, and the activity led should blink intermittently.
7. Restart the IOC (the existing IOC process my be locked up and may need to be kill -9 to die) or alternatively just reboot daq-rix-andor-01.

Debugging

Remote Power Cycling of Cameras

As of Nov. 17, 2023 rix now has a control panel to remotely power cycle andor cameras:

Image Added

Debugging

You can see log files for the IOC You can see log files for the IOC here:  /cds/data/iocData/ioc-rix-dir-andor/iocInfo/.

...

daq-rix-andor-01:~$ lsusb | grep Andor
Bus 001 Device 071: ID 136e:0005 Andor Technology Ltd. 
Bus 001 Device 063: ID 136e:0005 Andor Technology Ltd. 
daq-rix-andor-01:~$ 

Workaround for RIX ANDOR-DIR hang

Other debugging techniques:

• telnet to the ioc and restart
  • when restarting, ioc connects to all the cameras to see if it's the right one
• Sometimes have to reset all usb hubs/bridges like this:

bash-4.2# ~ddamiani/reset_andor.sh 3-2
bash-4.2# ~ddamiani/reset_andor.sh 3-2.4
bash-4.2# ~ddamiani/reset_andor.sh 3-2.4.4
bash-4.2# ~ddamiani/reset_andor.sh 3-2.4.4.1
bash-4.2# 

Workaround for RIX ANDOR-DIR hang

NOTE: We have seen cases where this does not work and we are forced to power cycle daq-rix-andor-01 to recover.

This is for when the camera is visible with "lsusb This is for when the camera is visible with "lsusb | grep Andor" (note uppercase "A") but the camera isn't communicating:

• Execute the following command on daq-rix-andor-01 as superuser: ~ddamiani/reset_andor.sh 1-2
• Restart one IOC (e.g. DIR), wait for it to finish
• Restart the second IOC, wait for it to finish

At the moment there is no way to know which camera is which, except for unplugging them one at a time.

The argument to the script ("1-2") can change, in principle but should do so rarely.  The “1-2” is coming from the USB bus and port enumerations from the operating system:
 
ddamiani@daq-rix-andor-01:~$ egrep 136e /sys/bus/usb/devices/*/idVendor
/sys/bus/usb/devices/1-2.3.1/idVendor:136e
/sys/bus/usb/devices/1-2.4.1/idVendor:136e
 
The first part of the for X-Y.blah bit which is 1-2 currently.

...