1 / 19

JAINIS ( J CU a nd In diana I nstrument S ervices):

JAINIS ( J CU a nd In diana I nstrument S ervices): A Grid Enabled Remote Access and Data Management System for X-Ray Diffraction. I. M. Atkinson 1 , M. D. Wyatt 1 , T. King 1 , N. G. Sim 1 , P. Turner, R. Quilici 2 , D. F. McMullen 3 A collaboration between:

tomai
Download Presentation

JAINIS ( J CU a nd In diana I nstrument S ervices):

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. JAINIS (JCU and Indiana Instrument Services): A Grid Enabled Remote Access and Data Management System for X-Ray Diffraction I. M. Atkinson1, M. D. Wyatt1, T. King1, N. G. Sim1, P. Turner, R. Quilici2, D. F. McMullen3 A collaboration between: James Cook University, the University of Sydney and Indiana University.

  2. Goal:Develop a Grid enabled and Web services based portal system for collaborative remote access to scientific instruments and their data. Why? Maximise resource utilization and return on investment/s Outreach, diffusion, training and education Grid? Access to compute cycles for processing and modeling and flexible distributed storage management that may further facilitate remote access Multi-Site Collaboration Goal for production level service: Significant collaboration between USyd, JCU, IU (and many others: UAdel, Monash, UQ, DART…) Deployment and user focus issues (this is hard)

  3. Common Instrument Middleware Architecture (CIMA) • Integrate instruments and sensors (e.g. real-time data sources) into a grid environment with Web Services interfaces • Abstract instrument capabilities and functions to reduce data acquisition and analysis applications dependence on specialized knowledge about particular instruments • Move production of metadata as close to instruments as possible and facilitate the automatic production of metadata • Develop a standard, reusable methodology for grid enabling instruments • Rick McMullen, • Knowledge Acquisition and Projection Laboratory, • Indiana University

  4. IU CIMA X-Ray system 4 Componentsi. Instrument Representative, ii. My Manager, iii. Data Manager, iv. Portal Interface • Instrument Rep, My Manager, Data Manager Service implemented in c++, gSoap, Perl • Storage – ext3 file-system, MySQL database • Portal – jrs168 portlets, gridsphere Issues – My Manager & Portal implementations domain specific i.e. Crystallography vs oceanographic monitoring

  5. CIMA Data Manager Workflow & comp pipelines. Automatic metadata generation. Instrument Representative CIMA SOAPInterface SOAP Instruments Live data feeds Video feeds GridSphere Portal SRB MCAT PGL: SRB access & schemas Data Processing Applications, Annontation and visualisation SRB data repository (data and metadata) HSM Storage CIMA core code from IU DART/JCU/UQ/Usyd/MU Implementation

  6. Data Management • NFS and MySQL replaced with Storage Resource Broker (SRB) • Kepler workflow used for MyManager • Ability to customise data storage via a workflow system • Highly extensible, not restricted to SRB as repository

  7. Storage Resource Broker (SRB) • Data Grid middlewarefrom SDSC (San Diego Supercomputing Centre) • Uniform interface to heterogeneous resources over a network. • Logical View of data • User defined metadata structures • Works in conjunction with MCAT. http://www.sdsc.edu/srb

  8. User Facing Portal • Use of Personal Grid Library (PGL) for SRB data manipulation / metadata display • Metadata schema definitions applied to experimental data • Stored experimental data is able to be easily retrieved, secured and annotated • Visual diagnostics for sensors (e.g. Graphs, Visual Indicators) • Live CCD image analysis (experimental status)

  9. A. Beitz, et al. Monash

  10. J. Hunter, et al. UQ

  11. Data Federations • Initial Instrument Representatives deployed at JCU, USyd, Monash & UQ Goal • Each site having their own Data Manager and SRB storage facility which is federated across sites • Shibbolith based AAA and Virtual Organisations • Federate/Replicate data into aa national SRB store…

  12. Future Developments • Data federations (SRB) • Security and Rights Management • Shibboleth • Analysis tools (Portals/portlets) • Workflows • Goal is an end-to-end workbench • Extensions to larger and smaller instruments (e.g. sensors)

  13. Acknowledgements: JCU Sydney Uni Indiana Uni Adelaide Uni S. Ness D. du BoulayK. Chiu (SUNNY) A. Wendelborn M. Morgan C. CheeP. Coddington F. Eilert D. Zhang Department of Education Science and Training:Dataset Acquisition, Accessibility, and Annotation e-Research Technologies (DART) project Australian Research Council:e-Research Seed Funding Programme and the Research Networks Programme (MMSN: Molecular and Materials Structure Network).

More Related