DPS Pump Troubleshooting

Table of Contents

Summary

The most common problems with the DPS is the roughing pumps tripping off and causing a cascade of other faults. So the first thing to look for is what stage or stages are faulted, and what roughing pump is likely to blame.

There is some redundancy in the system, so depending on the type of fault, the system should be able to continue to operate well enough to keep the beamline valves open if a single stage fails, but you may not be able to operate any gas sources without the full DPS operational.

Check for your problem in the table below, then see full instructions below that for how to fix the issue.

Troubleshooting guide:

Resetting a roughing pump fault

This may be indicated by one or more of the following:

1. The roughing pump is stopped (red) or disconnected (purple), or
2. The foreline vacuum gauge is reading a value >1 Tor, or
3. A warning or alarm is present in the "More..." panel of the roughing pump panel, or
4. Something else that I have not though of.

In the case of pressure rise on gauges 3259 and 3291 due to a failure of one or more pumps in the DS-DPS system, this procedure should be used to isolate the beamline and stop the pressure from rising further.  See the attached document for the full procedure.

In most cases this will isolate the beamline and allow the other remaining operating stage to pump down the beamline and allow operations to commence.

Link to full procedure:   FACET S20 DPS Troubleshooting - 03-04-2022.pdf

How to silence the roughing pump alarms in the tunnel

The roughing pumps will make an audible alarm if they are displaying a warning or alarm state. This is how to 

Ideal method:

To check the cause of the alarm:

1. Open the "DPS full schematic" from the LI20 Vacuum Epics page.
2. For each roughing pump - click on the pump icon, then on "More..."
3. You should see a list of all the warning and alarm states. If any of these has light beside it, then that is warning/alarm that is causing the problem.
4. Make an Elog entry with the warning and alarm states that you find

If you understand the cause of the alarm and the implication of clearing it, then you can go back to the roughing pump control window (with the speedometer) and click on "Reset" to clear the warning/alarm state. You may have to hit reset a few times if it doesn't stop beeping on the the first try.

Alternative method:

If you do not understand why the particular error message is being displayed or if you are not comfortable clearing the error, but you need to stop the incessant noise from driving you crazy - then you can simply unplug the handheld controller panel that sits beside the roughing pumps under the light pipe.  If you chose this path, then make all reasonable efforts to ensure that the controller panel is eventually reconnected.

In all cases - please notify the appropriate authorities that you have done something!

Troubleshooting if pumps keep turning themselves off during startup

At the moment you cannot restart a turbopump with the DPSvacWatcher running.  It will detect that things aren't in the nominal operating state and will stop the pump (or insert the foreline valve). This will be signaled by an entry in the elog.

Stop the watcher, restart the pumps, then restart the watcher once the valves are in a good state. Alternatively, once a turbo is tripped off then you have ~10 minutes while the watcher ignores that stage.  So if the turbo is shut off by the DPS when restarting, just try to restart again a second time within the 10 minute window and you should be fine.

Power cycling using a breaker

Breaker panel for the DPS roughing pumps is: 
 
Panel 2PK102E2 on south wall of S20 gallery 

     208V 3 phase 
     Flash hazard Category 0 
     PPE: Long sleeve shirt and pants (natural fiber), safety glasses 

Circuits are tied together in sets of 3: 
13,15,17 = 3151:US2 
19,21,23 = 3171:US1 
31,33,35 = 3259:DS1 
30,32,34 = 3291:DS2 
 
If a roughing pump will not reset an error message, turn power off on breaker
 panel for 30 seconds, then power back on. More status info is available on the hand held
 controllers in the FKG racks. 
 
Turbopumps also receive power from this panel. Turbo controllers are at penetration 20-10. There
 is a small brown half rack on the floor.

Step-by-step troubleshooting guide for vacuum issues

1. Open the full schematic: Select “DPS full schematic…” from the LI20 Vacuum page on Epics.
   
   
2. If the 3259:DS1 FV-DS1 foreline valve is closed AND the 3259:DS1 turbo is off, then this stage is in the normal “off” state – go to step 4.
   
   
3. If the 3259:DS1 FV-DS1 foreline valve is open, and either the 3259 turbo or roughing pump (or both) are off, then this indicates a problem with this stage. The flow chart indicates the actions to be taken to put the stage into an “off” state:
   
   DS1 troubleshooting flow chart:
   
   
   

   If you get to the green “Everything looks good” box, then this stage appears to be in a good state.

   
   

4. Now – assess the DS2 stage in the same way: If the 3291:DS2 FV-DS2 foreline valve is closed AND the 3291:DS2 turbo is off, then this stage is in the normal “off” state – go to step 6.
   
   
5. If the 3291:DS2 FV-DS1 foreline valve is open, and either the 3291 turbo or roughing pump (or both) are off, then this indicates a problem with this stage. The flow chart indicates the actions to be taken to put the stage into an “off” state:
   
   DS2 troubleshooting flow chart:
   
   
6. If at this point either of the 3259 or 3291 turbos are running, the beamline should be pumping down to <1e-6 Torr in a very short period of time. If neither is running, but the 3250 valve is open – then the beamline should be slowly being pumped down through the dump line turbo.
   
   
7. If both turbos are off and the 3250 valve is closed, then this section of the beamline will not pump down until one of the pumps is restarted.

If you reach this state and cannot continue or experience any difficulties – please call Doug Storey at SLAC extension # 3020, or cell # 650-656-5940.