310 likes | 433 Views
Computer Architecture & Grid Research Group and Grid-Ireland OpsCentre. Brian Coghlan Trinity College Dublin. Pro-Active Healthcare. Intelligent Transportation Systems. e-Science. Person to Person. Computer Systems Education. Innovative Pedagogies. Learning and Instruction.
E N D
Computer Architecture & Grid Research Groupand Grid-Ireland OpsCentre Brian Coghlan Trinity College Dublin
Pro-Active Healthcare Intelligent Transportation Systems e-Science Person to Person Computer Systems Education Innovative Pedagogies Learning and Instruction Programming Models Computer Architecture Software Adaptation Domain Specific Languages Clusters and Metacomputing Execution Models, Computistics Aspect Oriented Architectures Service Oriented Architectures Stream Processing & Multimedia Parallel & Virtual Architectures Ubiquitous Computing Sensor Enabled Artifacts Sensor Networks Intelligent Mobile Systems Global Smart Spaces Self Organising Systems Security and Trust Autonomic Computing Peer to Peer CVTR Lero Wireless Communications Delay Tolerant Networking Ad-hoc Networking Software Radio Channel Modelling Grid Computing Meta/Virtual/OnDemand Grids Grid Heterogeneity, eLearning Grid Trust, Security & Intrusion Grid Info, Viz, Policy, Ctrl, Data Grid-Ireland Computer Systems Research Lab
Grid Research & Operations Grid Research • Meta/Virtual/OnDemand Grids • Grid Heterogeneity, eLearning • Grid Info, Viz, Policy, Ctrl, Data • Grid Trust, Security & Intrusion Grid-Ireland OpsCentre • ROC for Ireland • OpsCentre manages Grid-Ireland • Remotely deploys & maintains core infrastructure • Grid component of national e-Infrastructure (e-INIS)
e-INIS Storage Databases Metadata Grid-Enabled Data/Metadata Services Federated ID
AMGA FTS LFC Database SRM Datastores Compute Data Grid Lambda-switched Networking OpsCentre OGSA-DAI LFC DPM Gateways Proposed DataManagement
Tightly Coupled Loosely Coupled Tightly Coupled Loosely Coupled Tightly Coupled Loosely Coupled Central Visualization Distributed Centralised Capability Distributed Capacity Algorithms Central Data Distributed Single User Viz Engine TCHPC Multi User Viz Engines e.g.TCD CSc Desktop Viz Engines Multicore Desktops Supercomputers ICHEC capability NUIG SMP Numa TCHPC Clusters UCD Rowan TCD CSc UCC Boole Cycle Harvesting TCD CSc Teaching Labs Central Query Engines ICHEC Regional Query Engines UCC NUIG Repository Query Engines Repository Sites e-INIS
Grid-Ireland Infrastructure Grid-Ireland
European Grid Infrastructures 2-OCT-07: ~250 sites >45,000 CPUs
UPS Network switch Gridfw (firewall) Gridinstall (Quattor) UPS Network switch Gridfw (firewall) Gridgate (CE) Gridinstall (Quattor) Gridstore (SE) Gridgate (CE) Gridmon (test WN) Gridstore (SE) Gridui (UI) Gridmon (test WN) Gridnm (NM) Gridui (UI) Gridnm (NM) Site Architecture Grid Gateway: All Virtual machines All run on 1 physical machine Remotely managed by OpsCentre Cluster(s): Managed by local admins OpsCentre supports integration Various config & install options
OpsCentre Central Services • OpsCentre: • Management of Grid-Ireland • Testing, porting, customisation • Deployment to remote sites • Remote management of sites • Monitoring of Grid services • National services (e.g. CA) • Dissemination and training • Grid courses and e-Learning • Links to EU Grids: • EGEE-II • int.eu.grid
gLite Services Typical Grid service software stack Pending layers: NGS GT4 WebCom-G SGA
Proposed Distributed Build Infrastructure Autobuild Current TCD Local Build Lifecycle
TestGrid site replicas cpDIASie Grid software (EGEE) OS updates csQUBuk Repository csTCDie Integration csUCCie Ops Centre hierarchical profiles scgNUIGie Certification
Transactional Deployment Geoff Quigley, TCD
gridinstall CE Repository SE Caching proxy UI Quattor profiles SW packages WN Transactional Deployment GUI Ops Centre Site Transactional Deployment Geoff Quigley, TCD
Operational Status - 100% available
Usage • 87% European- 13% National • 64% European- 36% National
Grid Research & Operations Grid Research • Meta/Virtual/OnDemand Grids • Grid Heterogeneity, eLearning • Grid Info, Viz, Policy, Ctrl, Data • Grid Trust, Security & Intrusion Grid-Ireland OpsCentre • ROC for Ireland • OpsCentre manages Grid-Ireland • Remotely deploys & maintains core infrastructure • Grid component of national e-Infrastructure (e-INIS)
ALTERNATIVES ALTERNATIVES Client SP GridShibUS WAYF ??? VOMS SAML ShibGridUK attributes SARONGUS VOMS Classic IdP VASH SLCS Attribute Cert Signing SAML Staff/StudentdB voms-proxy-init THEFT ? attributed cert AFTER USR/PW iS ISSUED usr/pw usr/pw usr/pw ADMIN Staff/StudentRegistration External ID Extra info Extra info proxy cert Grid cert THEFT ? RA Grid-Ireland CA VOMS Admin 10b Grid-Ireland RA VOMS Registration PhotoID PhotoID USER ABSENT ? 10a AFTER PHOTOID IS ISSUED 23 24 22 21 19 7 4 2 3 1 20 5 13 8 6 15 14 12 17 18 11 9 16 26 25 27 28 30 VO Admin Federated ID ALTERNATIVES ABSENT ? THEFT ?
Social Grid Agents - Economic Markets- Interoperability
Discovery Engine Consumer gridfs_discover Node 1 Discovery Engine R-GMA Node 4 Discovery Engine Producer gridfs_publish Node 6 export info query query publish export info export info export info export info Directory Engine Producer gridfs_publish_namespace Node 2 Directory Engine R-GMA Node 5 = Existing software publish = New software server config namespace info namespace info query Data Movement Engine Server GridSite Node 7 Directory Engine Consumer server config server IN namespace info server OUT namespace info query VFS VFS FUSE kernel module Data Movement Engine Client FUSE User Space Daemon Node 3 CURL User’s Job client OUT Physical Storage client IN Grid Filesystems - Standard file I/O calls- Traverses firewalls- Location-transparent- Grid secured
Infogrid - Relational interface to the Grid- Stream-oriented (uses R-GMA)
Submission System Submission System Submission System WebCom-G Infrastructure NUIG User TCD User Metadata carries job security token, which includes VOMS attributes UCC User Static SSL network connections, protected with KeyNote credentials and grid host certs Grid proxy cert GSI/SSL network connections
closed Secure WebCom world Border node WebCom execute nodes E WebCom portal MJX WebCom Server (entry node) CG + jobID + metadata SUBMIT + CG + proxy SUBMIT + CG + proxy + Dtoken REQ_TOKEN + proxy REQ_TOKEN + proxy Dtoken Dtoken MSX These actions could be done (if invoking Grid) directly from the WebCom portal Border node WebCom instance (grid submit node) Grid jobID + metadata GRID_SUBMIT + proxy S Grid response WebCom + Secure Connection Manager + Grid-Ireland host certs + KeyNote Security Manager + KeyNote certs + Map jobID to per-job filesystem + Execution in per-job context + TCD Secure Engine Module + TCD VOMS Job Security Module Host authN via Connection Mgr Host authZ via Security Mgr REQ_PROXY + Dtoken proxy Grid proxy cert Delegation token (NOT job security token) WebCom-G/Grid Security