Globus Data Transfer
How do I use Globus to transfer data at UH HPE DSI?
The University of Houston participates in the Globus data transfer service. Globus is a service that allows for the fast transfer of data between computer systems, using a convenient web interface. Globus data transfer speeds are typically significantly faster than data transfer speeds using other methods. Globus also contains data sharing capability, which allows users to share data with collaborators at UH or other institutions.
UH has Globus endpoints on both the Carya Cluster filesystem and Sabine Cluster filesystem. The endpoint display names are "University of Houston carya directories" and "University of Houston sabine directories" for carya and sabine reespectively. These endpoints allow UH users to transfer data to and from their home, and the group-shared directories, using Globus.
This page describes how to set up and use Globus to transfer data to and from UH HPE DSI systems. For advanced documentation, see the Globus How To webpage, the Globus FAQ webpage, and the Globus documentation site.
Transferring Data With Globus
In order to use Globus to transfer data between UH and another system, follow these simple steps:
- Navigate to https://www.globus.org/ and click the Login button in the upper right. When prompted for an organization, select the University of Houston. Complete the login using your cougarnet username and password.
- From the Globus menu, select Transfer Files. This will lead to a webpage showing two panels that can be used to select Globus endpoints.
- First search for "University of Houston". To select the 'University of Houston carya directories' or the "University of Houston sabine directories" endpoint, click in one of the Endpoint fields.
- Use the transfer panels to select files or directories, and transfer them between any two Globus endpoints using the left and right arrows in the center of the interface. Following a completed transfer, the Globus system will send a notification email message. Active and pending transfers can be viewed or canceled under the Activity link. If you would like to set up a Globus endpoint on your local computer, see the relevant section below.
Warnings:
Globus automatically retries failed transfers over a period of 24 hours following an initial transfer request. If a transfer request contains an error such that it will never succeed, it is beneficial to delete the request using the activity panel, to prevent continued failed transfer attempts.
Users are advised against using one endpoint as both source and destination for a single transfer, as this slows the endpoint, and more effective techniques for moving data within HPE-DSI are available.
Transfer Restrictions
As with other transfer methods, users may only transfer data for which they have normal filesystem read access, and may only place new data in directories for which they have normal filesystem write access. Globus is bound by filesystem permissions in the usual way.
Additionally, a user may only perform Globus transfers involving data within their home directory and other locations. Other directories such as /project/$PIs_account are available as well. If a path is unavailable and you would like it to be made available, contact us at support@hpedsi.uh.edu.
On UH HPE-DSI systems Globus will deny transfers involving home directory hidden, aka ‘dot’, files.
Setting Up a Globus Endpoint on a Local Machine
After logging in to https://www.globus.org select Endpoints, and click on the link for Add Globus Connect Personal Endpoint. This will lead to the page shown below, where it is possible to generate a Globus key, and to download the Globus Connect Personal client.
Follow the steps on the page to create a name for your local endpoint and generate a Globus key. Next, follow the directions to download the Globus Connect Personal client appropriate for your local operating system. Install the Globus Connect Personal client by running the installation program, and when prompted, paste in the Globus key which was generated for your personal endpoint.
While the Globus Connect Personal client is running on your local machine, your local machine contains a Globus endpoint. The Globus endpoint on your local machine may be selected by name from the Endpoint field in the Transfer Files panels. You may now use Globus to transfer files between your local machine and other Globus endpoints.
When the Globus Connect Personal client on your local machine is stopped, the local endpoint will stop.
Globus transfers involving a personal endpoint are likely to be slower than transfers between institutional endpoints.
Warning:
Globus write-sharing is disabled on HPE DSI cluster systems by default, and attempts to write to HPE DSI cluster directories via sharing will fail. If you believe you have a need for Globus write-sharing within your collaboration, please contact support@hpedsi.uh.edu.