Using Globus

Overview

Globus is a tool for users to request data transfers between predefined Globus sites (endpoints) and/or user desktops/laptops running Windows/Mac/Linux. Under the hood, Globus uses GridFTP to do managed data transfers, but users do not need to know anything about GridFTP. Globus provides high performance, reliability (e.g., via restarts after transient errors), and security.

Usage

Globus is an online service that allows transferring data between different locations.  To use Globus you need to sign in. There are different options to sign in, including using a institutional identity (e.g., SLAC username), sign in with Google or creating a free Globus account (Globus ID).

Once logged in, users can request data transfers, view progress on existing transfers, manage transfer requests, and view history.

In addition to its web interface, Globus also provides a Python-based command line interface (CLI) and application programming interface (API). For installation and usage please refer to the CLI documentation and the API documentation.

A data transfer happens between two Globus endpoints. The user must pick a data source and destination endpoint, and authenticate to each with their credentials. The authentication generates tokens with limited lifetime. These tokens are stored at Globus for that user, and used to act on behalf of the user for the lifetime of those tokens. If the data transfer is stopped because a token has expired, the user needs simply to re-authenticate to renew the token and resume the transfer.

Transfers can be performed between any Globus endpoint that the user is authorized to access. There are more than 10,000 Globus endpoints active, including at most DOE laboratories, essentially all US research universities, and many international sites. Either endpoint (or both, if the user’s institution has a Globus subscription) can also be a Globus Connect Personal agent running on a user's destop/laptop. Globus Connect Personal is available for free download after login to Globus.

Globus endpoint at SLAC/LCLS

Since 2021 LCLS is using the S3DF for data storage and processing. The Globus endpoint for the S3DF is slac#s3df_globus5. Older data that are still hosted on the old file systems are accessible by the lcls_pcds endpoint.

 Access to both is granted with the users SLAC Unix username/password.

Support

For more information about Globus, including documentation, support, and troubleshooting, check out the resources at the bottom of globus.org. If you suspect that a problem is SLAC/LCLS specific, please contact pcds-datamgt-l.