1 / 38

GOSC and NGS Status

GOSC and NGS Status. http://www.ngs.ac.uk. http://www.grid-support.ac.uk. Neil Geddes Director, GOSC Thanks to Stephen Pickles and Andy Richards. Outline. Users registrations, usage, helpdesk queries Services & Middleware what we offer today managing change Expansion “joining” the NGS.

indra
Download Presentation

GOSC and NGS Status

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GOSC and NGS Status http://www.ngs.ac.uk http://www.grid-support.ac.uk Neil Geddes Director, GOSC Thanks to Stephen Pickles and Andy Richards

  2. Outline • Users • registrations, usage, helpdesk queries • Services & Middleware • what we offer today • managing change • Expansion • “joining” the NGS

  3. Users

  4. NGS core nodes data nodes at RAL and Manchester compute nodes at Oxford and Leeds free at point of use apply through NGS web site accept terms and conditions of use light-weight peer review 1-2 weeks to do: project or VO-based application and registration all access is through digital X.509 certificates from UK e-Science CA or recognized peer National HPC services HPCx CSAR Must apply separately to research councils Digital certificate and Conventional (username/ password) access supported Gaining Access

  5. User registrations so far

  6. User Survey • Query all users who have had accounts > 3 months • 16 responses (out of ~100 users) • 3 papers • AHM04 (2), Phys. Rev. Lett. • 6 conference presentations • AHM04, SC2005, Systems Biology, IPEM(2), MC2004

  7. Helpdesk http://www.grid-support.ac.uk support@grid-support.ac.uk UKGSC: Queue Totals For Month Prior To (10:11 AM, 26/11/2004) • Notes • Certification still dominant query type • General contains GGUS related queries from EGEE/GGUS helpdesk • (A separate project for this has now been created) • LCG savannah generated queries are increasing through this helpdesk

  8. NGS Core Services Globus, SRB, Oracle, OGSA-DAI, and others

  9. NGS Core Services: Globus • Globus Toolkit version 2 • GT 2.4.3 from VDT 1.2 • Job submission (GRAM) • File transfer (GridFTP) • Shell (GSI-SSH) • Information Services (MDS/GIIS/GRIS) • Information providers from GLUE schema • Use BDII implementation of MDS2 (as does EGEE)

  10. NGS Core Services: SRB • Storage Resource Broker from SDSC • Location transparent access to storage • Metadata catalog • Replica management • Clients on compute nodes • Servers on data nodes • Issues/to do: • licensing • MCAT replication and failover

  11. NGS Core Services: Oracle • Oracle 9i database • Only on data nodes • Populated by users/data providers • Infrastructure maintained by NGS database administrators • Accessed directly or via OGSA-DAI

  12. NGS Services: OGSA-DAI • Developed by UK e-Science projects OGSA-DAI and DAIT • OGSA-DQP (Distributed Query Processing) • Experimental service based on OGSI/GT3 on Manchester data node only • containment: 2 cluster nodes reserved for development and production • will consider WS-I and WSRF flavours when in final release • Uses Oracle underneath • User-provided Grid Data Services validated on test system, then transferred to production during scheduled maintenance • Early users from e-Social Science (ConvertGrid) • Established liaisons with OGSA-DAI team

  13. NGS Core Services: other Operated by GOSC for NGS and UK e-Science programme In production: • Certificate Authority • Information Services (MDS/GIIS) • MyProxy server • Integration tests and database • Cluster monitoring • LCG-VO In testing: • VOMS • EDG Resource Broker In development: • Accounting • using GGF Usage Record standard for interchange, software from MCS project and TeraGrid • Portal

  14. Expansion Adding new resource providers

  15. Expansion Resource providers join the NGS by • Defining level of service commitments through SLDs • Adopting NGS acceptable use and security policies • Run compatible middleware • as defined by NGS Minimum Software Stack • and verified by compliance test suite • Support monitoring and accounting Two levels of membership • Affiliation • a.k.a. connect to NGS • Partnership

  16. Affiliation Affiliates commit to: • running NGS-compatible middleware • as defined in NGS Minimum Software Stack • this means users of affiliate’s resources can access these using same client tools they use to access NGS • a well-defined level of service and problem referral mechanisms • SLD approved by NGS Management Board and published on NGS web-site • providing technical, administrative, and security (CERT) contacts • providing an account and mapping for daily compliance tests (GITS++) • accepting UK e-Science certificates • maintaining baseline of logs to assist problem resolution

  17. Partnership Partners commit to same as affiliates, plus: • making “significant” resources available to NGS users • creation of accounts/mappings • in future, VO support, pool accounts, etc • recognise additional CAs with which UK e-Science programme has reciprocal agreements • publish/provide additional information to support resource discovery, brokering • ability to compile code for computational resources Bristol and Cardiff going through certification process now. • supported by “buddies” and NGS-Rollout list • useful feedback on viability of NGS Minimum Software Stack

  18. Benefits Affiliation: • NGS brand • certified NGS-compatible • better integrated support for local users who also access NGS facilities • assistance/advice in maintaining NGS-compatibility over time Partnership: • higher brand quality • representation on NGS management board • either direct, or through regional or functional consortia NGS brand must be valuable to make this work.

  19. User Support

  20. Provide first point of contact support Contact point between other helpdesks Or provide helpdesk facilities for other sites Users input queries at a range on places behind the scenes collaboration user gets answer back from where they asked it Develop Support relationship with technical expertise at sites

  21. Possible ticket status and order for Helpdesk Tickets

  22. Query History • An audit trail of all actions performed on a query can be viewed from the History button on the Details screen. • View Contact History on the Edit screen shows all queries associated with a user

  23. Inter-Helpdesk Development e.g. GGUS

  24. Helpdesk Plans • Upgrade server to FootPrints v6.5, includes new user portal interface to access knowledge base without logging in. Make server SSL-enabled i.e. https • Explore use for other areas of e-Science eg GridPP, OMII • Populate UKGSC knowledge base with additional solutions and FAQs. • Publicise web interface to users and include web link in email notifications to users. Note: Users logon to FootPrints with their email address and register a password. • Create additional business rules to automate procedure and add reports to satisfy SLAs etc. • Review best practice for interactions with other e-Science helpdesk systems, internally and externally (e.g. GGUS relationship)

  25. Recent Developments and Summary

  26. Recent Developments • NGS newsletter • OMII helpdesk • Resource Broker (LCG) available for early adopters • TeraGrid • UK certificates accepted on TeraGrid • Looking at INCA monitoring with Jenny Schopf • Trial of accounting software from MCS project • using GGF Usage Record draft standard (as EGEE) • Meeting with Open Science Grid’s iGOC • Collecting user exemplars • Agreed form of acknowledgement • Workshop towards an NGS user induction course. Need: • Easier access (e.g. user interface machines and portals) • T-infrastructure

  27. Summary TODAY • 4 core nodes operational • 150 Users registered (50 since 1 September ’04) • Grid enabled – Globus v2 (VDT distribution v1.2) at present • BDII information service (GLUE + MDS Schemas) • Data Services – SRB, Oracle, OGSA-DAI • Growing base of user applications • MyProxy and CA services • VO Management Software – LCG-VO • User support: Helpdesk Next… • Other Middleware [gLite/OMII etc…] • NGS Portal • Resource Broker • SRB production service • Accounting • Continued expansion

  28. GOSC and NGS Process decision/testing/transition/deployment of Web Service Grid Middleware

  29. Maintaining Compatibility • Operating a production grid means valuing robustness and reliability over fashion. • NGS cares about: • alignment/compatibility with leading international Grid efforts • special requirements of UK e-Science community • easy migration/upgrade paths • proven robustness/reliability • based on standards or standards-track specifications • NGS cannot support everything • Everyone wants service-oriented grids • but still settling out: WS-I, WS-I+, OGSI, WSRF, GT3, GT4, gLite • Caution over OGSI/WSRF has led to wide convergence on GT2 for production grids and hence some inter-Grid compatibility • but there are potentially divergent forces at work • Significant changes to NGS Minimum Software Stack will require approval by NGS Management Board on conservative time scales

  30. Strategic Framework • GOSC/NGS  UK e-Science project • support other UK (e-)science projects • International Compatibility • EGEE • European infrastructure (and possible funding) • LHC at most UK universities • only user group who want to build the grid • GridPP committed to common w/s plan in 2005 • GEANT • Others • TeraGrid – US cyberinfrastructure $$$ (unlikely to pay us) • Open Science Grid – will develop compatibility with LCG • RoW e.g. China • Want use other software, but must be EGEE compatible • Also driven by user requirements • Sets framework for relationship with OMII and others

  31. Process for Moving Forward • New developments evaluated by ETF • must have some longer term support likely • User requests treated on case by case basis • NGS Technical Board consider against needs • user demand • new functionality • improved functionality • improved security/performace/managability • Proposal brought to GOSC Board • Prepared by GOSC “executive” • N.Geddes, S.Pickles, A.Richards, S.Newhouse

  32. “User requests treated on case by case basis” • Already see users running web services in user space • Exactly what we want … but … • Potential security risks • Change Conditions of Use to reflect user responsibilities • Require secured web services (X509) • Encourage auditing and audit trails • With time limits • Services run “at risk” • Services lead to significant system load • run on head node (or other specialised node) • Full support only when “approved”

  33. The GOSC Board Director, GOSC (Chair) Neil Geddes Technical Director, GOSC Stephen Pickles Collaborating Institutions CCLRC Prof. Ken Peach Leeds Prof. Peter Dew Oxford Prof. Paul Jeffreys Manchester Mr. Terry Hewitt Edinburgh/NeSC Prof. Malcolm Atkinson UKERNA Dr. Bob Day London College tbd ETF Chair Dr. Stephen Newhouse GridPP Project Leader Prof. Tony Doyle OMII Director Dr. Alistair Dunlop EGEE UK+I Federation Leader Dr. Robin Middleton HEC Liaison Mr. Hugh Pilcher-Clayton Also invited e-Science User Board Chair. Prof. Jeremy Frey Director, e-Science Core Programme Dr. Anne Trefethen 

  34. Expectations • Little demand for GT4 • expect usability much better than GT3 • Watching brief, OGSA-DAI may drive this • Indications of Glite improvements in • VOMS, RB, shell, File i/o, data catalog • unlikely to have full ETF assessment by End March • unlikely to all be robust before Q4 • OMII job submission • expect good usability, limited functionality • run as user services • problems integrating into Resource Broker and Accounting ? • Net result likely to be a vague w/s plan • Hopefully able to focus on some key components

More Related