The National Institute for Computational Sciences

Data Transfer

  Data Transfer

NICS provides several ways for transferring files to/from all of our HPC resources that can benefit from the high speed network connections with the XSEDE network and Internet2.

SCP, SFTP

NICS supports file transfer with OpenSSH file transfer utilities SCP and SFTP which are available on NICS resources and usually available on all Linux/Unix machines. There are command line and graphical clients available for OpenSSH. SCP and SFTP utilities are available for transferring files but will usually perform slower than GridFTP.

All NICS login nodes support OpenSSH which will allow SCP and SFTP utilities. There is also a GSI-OpenSSH capability for OpenSSH for use with GSI user certificate authentication (XSEDE MyProxy certificate) available at gsissh.beacon.nics.utk.edu. gsissh.beacon.nics.utk.edu is currently unavailable as this node is being used for ACF testing. When it is back online this documentation will be updated.

When the ACF comes online in the fall 2017, the Data Transfer Nodes (DTN) DNS names scp.nics.utk.edu can be used to access SCP/SFTP functionality from the NICS DTN nodes.

BBCP

This is another tool to securely and quickly copy data from source to target. The BBCP utility is capable of breaking up your transfer into multiple simultaneously transferring streams, thereby transferring data faster than single-streaming utilities such as SCP and SFTP. Check for local availability, but if it is not available the source code can be obtained from its homepage. Several examples about how to use it, can be found at its dedicated BBCP page.

GridFTP

GridFTP, a fundamental server component of Globus Online, is the highest performance method NICS supports to transfer files. NICS provides data transfer nodes (DTNs) with GridFTP services installed and login nodes have the Globus toolkit tools with the GridFTP client commands. Many other computing centers support GridFTP for file transfer. This method can be used using a web interface or at the command line. For more detailed information about how to use GridFTP see our dedicated GridFTP page.

Access GridFTP from the NICS DTN nodes using DNS entry gridftp.nics.utk.edu. This DNS name will stay the same for ACF.

Globus Online

NICS users can use the web Globus Online interface to perform data transfers to/from NICS supported resources. The visual interface makes it quite easy to move, back up or restore relevant data. To get you started, visit the Globus website and consult the Quick Start guide.

Globus endpoints to access NICS resources

XSEDE endpoints (i.e. using XSEDE authentication):

  • XSEDE beacon
  • nics#beacon-myproxy

One of the latest features of Globus is Globus Connect Personal. Globus Connect Personal turns your personal computer into a Globus endpoint so you can share and transfer files to/from a local machine—campus server, desktop computer or laptop.


Transfer files between local computer and NICS resources