4CeeD: Private Cloud and Data Cyber-Infrastructure for Scientific Instruments

1. 4CeeD: Private Cloud and Data Cyber-Infrastructure for Scientific Instruments Steve Konstanty, Senior Research Programmer, CSL

2. Overview • What is 4CeeD Storage Facility and 4CeeD Services • Why are 4CeeD Services important to MRL/MNTL researchers • How does MRL/MNTL researcher work with 4CeeD Services • How does MRL/MNTL researcher sign for 4CeeD Services

3. 4CeeD Storage Facility and Services Goals Address Scientific Digital Data Acquisition, Curation and Sharing prior to Scientific Publication of Results via Private Cloud Storage Facility

4. Data come of various types and multimodal formats • Each type requires a different dataprocessing workflow Scientific Digital Data Acquisition and Workflow Challenges Sample output data from SEM microscopy • ProfilometryEllipsometry • ProfilometrySEM • ProfilometryEllipsometry • SIMSSEM • Optical microscopy • SiO2 MaskDeposition • SiNxDeposition • PlasmaEtching • Lithography • Diffusion • SiNxRemoval • Device Characterization • Oxidation • Lithography • Metallization • ProfilometrySEMOptical microscopy • Optical microscopy • SEM • SPA • SEM

5. Scientific Data Input/Output 4CeeD System Architecture User Office Scientific Devices CuratorClient Uploader LAN/WAN Uploader/Curator Server Task/Resource Coordinator Service Private Cloud Compute/Storage Services • github.com/4ceed

6. 4CeeD Private Cloud Compute/Storage Service What we looked for • Redundancy • Availability • Scalability Storage Layer • 40 TB (20 TB per investor) • Replicated for redundancy Compute Layer • Docker container orchestration (Kubernetes) • Single master (High Available masters in future) 4CeeD Cloud Storage 4CeeD Cloud Compute

7. 4CeeD Uploader Service (Simple and Speed-Up Usage at Microscopes) Simple steps, with support for advanced usage 1. Choose or select a collection. 2. Load template and enter user defined metadata to create a dataset. 3. Upload files to cloud coordinator.

8. 4CeeD Curator Service (Speed-Up Curation) File View Dashboard View [Preview, annotate, download, extracted metadata] [Dashboard management]

9. 4CeeD Smart Data Management 4CeeD Data Model organizes projects into collections, datasets, and files. These can then be shared in spaces.

10. Why 4CeeD? Easy to use tool for collecting data and metadata for storage and sharing with research group

11. Current Situation in MRL Facilities Current situation for experimental data involves manual processes for data capture and storage leading to poor documentation of results Data transfer is often done via “sneaker-net” techniques using flash-drives or email No data file conversion is available “Best” results and images are kept, but what is “best” is determined by a narrow, specific scientific objective. “Imperfect” data is often discarded or not available for others to review.

12. Current State of Data Capture • Fabricate experimental sample • Prepare analytical sample • Bring sample to instrument for analysis • Extract data (File conversion) • Transport data to office computer • Analyze data • Repeat per iteration • Extract data (File conversion) • Transport data to office computer Limits of home analysis -Only what you have -Connect notes to data (after time delay) -Retroactive not reactive ‘Sneakernet’ -Security risk at both ends -Very limited transport space -Lost or forgotten flash drives • Analyze data Metadata Loss -Excessive file name schemes -Manual notes -What metadata is important? Instrument (MRL/MNTL) Flash Drives Office

13. 4CeeD Data Capture • Fabricate experimental sample • Prepare analytical sample • Bring sample to instrument for analysis • Extract data • Transport data to office computer • Analyze data • Repeat per iteration • Extract data (NO FILE CONVERSION) • Transport data to office computer (DIRECT) • Analyze data (REAL TIME) Direct web interface -Highly secure data storage -Large transport volume -Easy to use/ annotate Real time -Metadata paired with data -Reactive interpretation -Easy interpretation/searching Lossless data -More simple file names -Automatic note taking -All metadata included Instrument (MRL/MNTL) Laptop Collaborators Campus PC Office

14. Access to 4CeeD in MRL • Access to 4CeeD through web-browser (https:/4CeeD.illinois.edu) • Extraction of meta-data automatically performed • Storage of data on RAID storage cloud maintained by Engr-IT • All researchers of MRL Facilities (and their collaborators) will be allowed access to the MRL space

15. 4CeeD advantages • One stop location for all of your research data • Sharable between you, your advisor, and your research partners • Visual interpretation of data without having to open specialty software • Cloud based  you can access your data anywhere • Provides machine specific meta-data on measurement for future replication of measurements • Provides templates which allow for the entry of metadata specific to YOUR project

16. How does it work? Web based application for uploading, sharing, downloading, and working with data. Extractors for: TEM/SEM/AFM/SIMS/XRAY

26. Advanced Materials Characterization Workshop June 6 & 7, 2017

27. Partner with Engineering IT for Maintenance 4CeeD private cloud for MRL/MNTL • Production cloud with Engineering IT staff maintenance • 40 TB cloud for Material Research and Semiconductor Research at UIUC • Purchase and Installation Started in Spring 2017 • Extensive Testing during Spring/Summer 2017 • Educational Material to Use the 4CeeD facility are available on 4CeeD website • https://4ceed.github.io/ • http://t2c2.csl.illinois.edu/ • Workshops to Use the 4CeeD facility will be offered in Summer and Fall 2017 • 4CeeD Services are available now • 4CeeD Services will be in full production mode in August 2017 (beginning of Fall semester)