Remote Visualization

Overview

When users are off-site or have slow internet connections, displaying graphics can be cumbersome.  PCDS has explored two solutions for this problem:

1. The first ("NX technology") gives the user a special remote desktop that is specifically designed to improve the performance of X11 graphics over slow connection speeds (compared to ssh).  
2. The second ("Virtual Box") allows the user to directly install the psana analysis environment on their Windows, Linux or Mac machine, so that LCLS data analysis (including graphics) can be done with no network connection (click here for information on the virtual box approach).

This page is about the usage of NoMachine for providing NX technology to users involved in LCLS experiments or operations.

NX Technology

NoMachine NX is supported on Windows, MAC and Linux computers. 

Minimum System Requirements

A single 4k (3840 x 2160) display requires ~20 Mbps of download speed to support a 1Hz refresh rate.  This may vary depending on the number of high-definition or rapidly changing UI elements being displayed.  The upload requirements are minimal for control, but may limit speed of file transfers from the client to the host.

Note: MAC Clients must install XCode and XQuartz.You can download XQuartz from https://www.xquartz.org.

Installation

You can download the latest version of the enterprise client from https://www.nomachine.com/download-enterprise#NoMachine-Enterprise-Client. Ubuntu/Mint users should download the Debian version of the NoMachine client.

Configuration

LCLS is currently making two servers available for remote connections:

1. psnxserv.slac.stanford.edu (aka psnxserv01, aka psnx01): for general purpose connections
2. psnxana.slac.stanford.edu (aka psnxserv02, aka psnx02): for interactive data analysis
3. psnxopr.slac.stanford.edu (aka psnxserv03, aka psnx03): for remote operation support.

You will be prompted to setup a login profile when launching the NoMachine for the first time. Select the following options:

The allocation of theses servers to the various activities is just to provide a low-tech load balancing and is not enforced.

Accept the default options if no graphical display is required (Application - Run the console, Options - Run the command in a floating window) by clicking Continue.

This will open a remote terminal on your local desktop from which you can type the name of the application you want to launch. For example, you can type "xterm&" or "gnome-terminal" to open additional terminals or "emacs&" to open the emacs editor.

If your session window appears briefly and then NoMachine windows close, you may need to apply this fix from

	- Edit the player.cfg on the Windows computer and change the value of this key:
	  <option key="Switch to software decoding on Windows if resolution is bigger than 1920x1080" value="false" />
	  to:
	  <option key="Switch to software decoding on Windows if resolution is bigger than 1920x1080" value="true" />

Type "ssh -Y psana" to connect to the interactive pool to do data analysis.
 

When you disconnect and reconnect to NoMachine, these terminals and applications will remain in the state you left them in. If you would like to log out completely from your session, you can select the option "terminate".

If you require a very basic display manger you can select "Run the command in a virtual desktop"

At the login prompt type the command "startx"

If you get a dialog box, select the "default configuration panel" to obtain a similar image to the one below:

To get a nice working display, you can either resize the virtual desktop to fit the size of a defined window or full-screen (Ctrl+Alt+F)

When done, you can disconnect your session (keep state) or terminate (remove session).

Please contact pcds-it-l@slac.stanford.edu if you need more information or additional assistance with NoMachine.

Troubleshooting

Bashrc

Avoid sourcing the psconda environment on the NoMachine servers.  To add the conda in your bashrc file, you can do the following:

• Set an alias to source it manually:  alias psconda='source /reg/g/psdm/etc/psconda.sh' 
• Only source the environment on the ana nodes 

export MN=$(uname -n)
if [[ "$MN" =~ ^psana ]]; then
    source /reg/g/psdm/etc/psconda.sh
fi

Connection Issues

If you are not able to connect to NoMachine, check the followings:

• For an offending ssh key error, please remove the old key by typing the following command: ssh-keygen -R "hostname" (e.g. ssh-keygen -R psnxserv01)
  
  
• MAC Clients must install XCode and XQuartz.You can download XQuartz from https://www.xquartz.org. Otherwise, you will get the following error:

• The server is setup to limit to 4 connection attempts per minute, this is to discourage brute force attempts for security purpose. If it seems to take a long time to connect after you made multiple connection attempts quickly, you can just cancel out and try to log back in after a couple of minutes.

• If you have disconnected your session without logging out and try to reconnect, once you hit the connect button, you will see the following dialog box. You can create a new session by selecting 'Create a new desktop or custom session' or you can reconnect to your existing session. If the existing session failed to reconnect, you can right-lick on the icon representing your previous session and select Terminate session. And then click on 'Create a new desktop or custom session' and select 'Create a new virtual desktop' to click on continue.
  
  

Reconnect "Black Screen" Issues

Sometimes users will find that they are unable to reconnect to an existing session, and are shown only a blank screen.  We do not yet understand this issue, so our solution is to terminate the user's existing sessions, and kill all the user's running processes on the NoMachine server.  The easiest way for the users to deal with this is to send a note to pcds-it-l@slac.stanford.edu and we will remove all old sessions and process.

It's also possible for users to fix with this themselves by ssh'ing to psnxserv.slac.stanford.edu and killing their own process.  Potentially useful commands follow:

Show all running user processes: 

ps -f -u $USERNAME

Kill a specific process using its process ID (found in the PID column of the "ps" command output):

kill <PID>

Kill a specific process that doesn't want to die (i.e. was not killed by "kill <PID>"):

kill -9 <PID>

Kill all user owned processes running on the server, including current ssh session, shell and the pkill command itself. This is a quick way to remove the majority of user owned processes:

pkill -u $USERNAME

Note that the user will have to reconnect after running pkill and may have to try a number of times (possibly using the -9 option, as shown with the kill command).

Resuming a NoMachine Connection

If you are not able to resume a NX Connection and you typically connect from multiple computers, be sure to use similar version of NoMachine client where you initiate the connection from. To check for NoMachine version, after you have launch NoMachine client, go to Preferences on top, then click on Updates, it will show you the version that you are running NoMachine on.

Broken display with "Green Screen" Issues

If your display is broken with mainly a green screen, it could be a hardware decoding issue. Click on "Display" button on NoMachine (shown below). Then click "Change settings" and turn on "Disable client side hardware decoding".

Other issues

If Ctrl-C (Ctrl-break) does not work on the Linux client, hit the Alt-Ctrl-C (Use it in proper sequence so hold down the key one at a time). WindowsKey-Ctrl-C, or turn on Caps Lock and hit Ctrl-C will also work.

Also see our how-to guide for Diagnosing Network/Connection Problems