1 / 28

Collaboration on Large Datasets using Globus

Collaboration on Large Datasets using Globus. Rachana Ananthakrishnan University of Chicago. Data sharing in collaborations. Registry. Registry. Staging Store. Ingest Store. Ingest Store. Community Store. Community Store. Analysis Store. Analysis Store. Archive. Mirror. Archive.

edan
Download Presentation

Collaboration on Large Datasets using Globus

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Collaboration on Large Datasets using Globus Rachana Ananthakrishnan University of Chicago

  2. Data sharing in collaborations Registry Registry Staging Store Ingest Store Ingest Store Community Store Community Store Analysis Store Analysis Store Archive Mirror Archive Mirror

  3. Data Management User Stories • “I need a good place to store / backup / archive my (big) research data” • “I need to easily, quickly, and reliably move or mirror portions of my data to other places.” • “I need a way to easily and securely share my data with my colleagues at other institutions.” • “I want to publish my data.” • “I want to discover published data.” • …

  4. Exemplar: ISI-MIP • Inter-Sectoral Impact Model Intercomparison Project • Framework to collate climate impact data across scales and sectors • World-wide collaboration with data assets managed by the collaboration • Inputs from various climate models & output forms basis for model evaluation and improvement Credits: Dr. Joshua Elliot, University of Chicago

  5. ISI-MIP Use Cases • Share data with researchers across institutions world-wide • Restricted sharing • Multiple institutions • Accept data submissions • Restricted writing to archive • Publish results • Move selected results to other locations • Track metadata • Discover data

  6. What is Globus? Big data publish*, transfer and sharing… …with Dropbox-like simplicity… …directly from your own storage systems * In pilot phase

  7. Publish walk-through IIT Univ. of Chicago UIUC Argonne 1. Publish Data 4. Curate Dataset 2. Describe Submission Scientist 3. Assemble Dataset (Transfer Data) Collaboration Archive Curator

  8. Login with Campus Identity

  9. New submission

  10. Assemble the Dataset

  11. Move data to publish archive

  12. Grant Submission License

  13. Submission Complete

  14. Curator Logs in

  15. Curation Workflow Options

  16. Verify Metadata & Files

  17. Approve the Submission

  18. Submission is now Published with DOI

  19. Discover walk-through 5. Search IIT Univ. of Chicago UIUC Argonne 6. Download 1. Publish Data 4. Curate Dataset 2. Describe Submission Scientist 3. Assemble Dataset (Transfer Data) Collaboration Archive Curator

  20. Search Published Datasets

  21. Discovering a Published Dataset

  22. Download the Published Dataset

  23. Select Download Destination

  24. Globus Under the Covers … Globus APIs Globus Connect Sharing Service Transfer Service Identity, Group, Profile Management Services Globus Toolkit

  25. Reliable, secure, high-performance filetransferandsynchronization 2 Globus moves and syncs files • “Fire-and-forget” transfers • Automatic fault recovery • Seamless security integration • Powerful GUIand APIs Data Source Data Destination User initiates transfer request 1 3 Globus notifies user

  26. Simple, secure sharing off existing storage systems 2 Globus tracks shared files; no need to move files to cloud storage! • Easily share large data with any user or group • No cloud storage required 3 1 User A selects file(s) to share, selects user or group, and sets permissions Data Source User B logs in to Globus and accesses shared file

  27. Thank you • Signup and useGlobus to transfer and share • globus.org/signup • Signup as early adopters of publish • globus.org/data-publication • Support • support@globus.org

  28. Thank you to our sponsors! U.S. DEPARTMENT OF ENERGY

More Related