1 / 18

Scientific Data Grid Updates

Scientific Data Grid Updates. Kai Nan Computer Network Information Center, CAS PRAGMA 3rd Workshop 23 January 2003, Fukuoka. Agenda. SDG Testbed SDG Middleware SDG Security System SDG Information Service Universal Metadata Tool SDG pilot Application Virtual Observatory

tait
Download Presentation

Scientific Data Grid Updates

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Scientific Data Grid Updates Kai Nan Computer Network Information Center, CAS PRAGMA 3rd Workshop 23 January 2003, Fukuoka

  2. Agenda • SDG Testbed • SDG Middleware • SDG Security System • SDG Information Service • Universal Metadata Tool • SDG pilot Application • Virtual Observatory • Snapshot of Grid Activities in China

  3. SDG Testbed • Purchase of new clusters deferred  • Current Status • Sharable resource for PRAGMA: 8 nodes (AIX) • 2 PowerPC CPU, 1GB Memory per node • Gatekeeper (Linux) • because Globus didn’t work on our AIX 4.3, until now • gatekeeper.sdg.org.cn • Linux 7.2, Globus 2.2, PBS Server/Scheduler, MPICH • service name: jobmanager-pbs • Node • AIX 4.3, PBS mom, MPICH • CA cert • http://www.sdg.org.cn/download/sdgca.cert

  4. SDB Status • By Dec. 2002 • 31 member institutions • 217 databases • 3.2 TB • Classification • numerical: 46% • text: 18% • spatial: 22% • multimedia: 14%

  5. SDG Middleware Application applications SecuritySystem Info. Service app-oriented, unified program interface Grid API coordinated access to multiple data resources Data Res. Broker uniform access interface to single data resource Uniform Access Int. local data management system, could be DBMS or file system Local Data System databases

  6. SDG Security System • Requirements • Single Sign-On • Delegation • Universal credentials • Integration with local policies • Policy management • Data-oriented access control • User-based trust/trusteeship • Logging • Open architecture & Interoperability with other Grids

  7. SDG Security System (cont’d) • Services • Authentication (Based on Globus GSI) • secure connection • user proxy management • Authorization • mapping global certificates to local roles • role-based access control • local role management • Accounting

  8. CAPP/KAPP Authen. CUP1 VS CAPP Step2 Client App. CU KU CUP1 KUP1 GAPI Step3 Authen. CUP2 VS CIS CUP2/KUP2 Step4 Authen. CUP2 VS CDRB Step1 Create user proxy CDRB/KDRB Authen. CUP3 VS CIS Step5 SDG-IS DRB CUP3/KUP3 CMDS/KMDS Step6 Authen. CUP3 VS CUAI Step7 Authen. CUAI VS CIS CUAI/KUAI UAI Step10 Access data Step8 Map global cert to local role Step9 Role-based access control DBMS LACL Map SDG Security System (cont’d) Full Process of security-related operations under SDG Security System

  9. SDG Information Service • Requirements • resource discovery • answer to “What, How”– intrinsic properties of data • relatively static metadata, generated by man • location & monitoring • answer to “Where, When”– extrinsic properties of data • dynamic information, generated by program • API for other modules of SDG • collect information about data resource automatically • metadata tools

  10. SDG Information Service (cont’d) • SDG Info. Service • DCIS: Data Container Info. Service • built on Globus MDS • design DIT for SDG (schema, OID, namespace) • develop a program which collects information and returns it as LDIF, called info. provider • configure a new MDS • MDIS: MetaData Info. Service • actually a normal LDAP • add ldbm-backend to MDS in order to store static metadata • develop the metadata tool to manage MDIS • Compatible with Globus MDS 2.1

  11. SDG Applications Query GRIP GRRP MDRP SDG GIIS C-MDW SDG Sub-GIIS DCIS MDIS I-MDIS C-MDIS DCIS MDIS I-MDIS C-MDIS SDG Information Service (cont’d)

  12. SDG Universal Metadata Tool • Requirements • why universal • many disciplines in SDG  similarly many or more metadata standards • it’s not good for us to develop a tool for every metadata schema individually • input metadata for existing databases is more bothersome, so a ease-to-use tool might be must-have in practice • input: a metadata schema (xml DTD) • output: • Web-based, customizable UI • LDAP-based Storage • Management functions (add, delete, modify and query) • back-end is MDIS

  13. MD schema install & configure Process(Java bean) MDIS(LDAP) interim XML User page XML engine universal, extensible customizable SDG Universal Metadata Tool • metadata is tree-like and more flexible than fix-column tables, difficult to deal with on web UI • use xml files to store interim results

  14. Applications • Virtual Observatory • Astronomical data is huge & well documented • Most is online & sharable • So, Internet is becoming the world’s best telescope • Data integration and federation(by many different instruments from many different places and many different times) • Ease-to-use for astronomers • LAMOST (aperture: 4m, by 2004) • Collaboration betweenNAO and CNIC

  15. Applications (cont’d) • Some fields we’re trying • Biology • Prof. Ma, IM/CAS • Chemistry • Prof. Chen, ISOC/CAS

  16. Projects on SDG • Supported by MOST (863 Program) • From Oct. 2002 to Oct. 2005 • SDG is an Application Grid of the China National Grid (CNGrid) • Emphasis on Grid-enabled applications in science and research • Supported by CAS • From 2001 to 2005 • Cover testbed, middleware and application

  17. Snapshot of Grid Activities in China • Scientific Data Grid – CAS • CNGrid – 863/MOST • Grid-enabled Cluster (>4 Tflop/s) • Grid Nodes (Total 6-10 Tflop/s) • Grid Software (Grid OS, Developer and User Environment) • Grid Applications in Science, Manufacturing, Service industry, and Environment/Natural Resource, etc. • China Grid – MOE (proposed) • To connect 100 universities • ? – NSFC

  18. Thank you!

More Related