1 / 8

Data Transfers in the ALCF

Learn how to transfer data to and from ALCF using various methods such as SFTP, SCP, Globus Online, and GridFTP. Find out which systems are best for small and large transfers, and get started with Globus Online.

rdeloach
Download Presentation

Data Transfers in the ALCF

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Transfers in the ALCF Robert Scott Technical Support Analyst Argonne Leadership Computing Facility

  2. Transferring Data To/From ALCF • sftp and scp (for “small” transfers) • If you must use scp, eureka, and in the future tukey are better systems to scp to/from. All paths will be the same as they are on our other systems. • Eureka is also a better host for compressing and uncompressing large file archives • GridFTP (for large transfers) • Other site must accept our CA • ssh / cryptocard access available • For more information on Grid FTP please go here: http://www.alcf.anl.gov/resource-guides/using-gridftp-intrepid • Globus Online (for large transfers) Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy

  3. Getting Started with Globus Online • You’ll need a Globus Online account • Go here to signup: https://www.globusonline.org/ • Globus Connect: to transfer files to and from your local machine (laptop) • SSH key or crypto card to access an ALCF endpoint • Go here to initiate your transfer: https://www.globusonline.org/xfer/StartTransfer Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy

  4. What is Globus Online? • Reliable File Transfer • Easy: Simple signup, login and use – and minimized intervention • Fast: GO can move large filesets in hours • Secure: No Need to worry about security configs or certs, and one-time passwords work • Research-focused: Globus Online is the only file transfer service with the scientific researcher in mind • No IT required • No client software installation • Works with existing GridFTP servers • Globus Connect solves “last mile problem” for local machines • Consolidated support and troubleshooting Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy

  5. Transferring Data To/From ALCF using GO • Globus Online (for large transfers) • Globus Onlineaddresses the challenges faced by researchers in moving, sharing, and archiving large volumes of data among distributed sites. • ALCF BG/P endpoints: alcf#dtn_intrepid, alcf#dtn_surveyor, alcf#dtn • Check if your laboratory, university or research center has already an endpoint. • Globus Connectto transfer files to and from your local machine. • http://www.alcf.anl.gov/resource-guides/data-transfer Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy

  6. Globus Online Demo • Let’s start the Demo…. Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy

  7. Table of BG/Q File Systems on ALCF Resources NOTE: all “filesystems” are also available via the /gpfs/$filesystem_name path on these systems. Argonne Leadership Computing Facility – www.alcf.anl.gov

  8. Thank You! • Questions? Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy

More Related