1 / 27

gLite Information System

Valeria Ardizzone INFN-Catania 1° EELA Grid School Itacuruçà Island, State of Rio De Janeiro, Brazil. gLite Information System. Outline. Grid Information Systems Overview Information Systems in gLite GLUE Schema Grid elements LCG Information System Hierarchy LDAP model

vernon-stafford
Download Presentation

gLite Information System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Valeria Ardizzone INFN-Catania 1° EELA Grid School Itacuruçà Island, State of Rio De Janeiro, Brazil gLite Information System

  2. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 Outline • Grid Information Systems Overview • Information Systems in gLite • GLUE Schema • Grid elements • LCG Information System • Hierarchy • LDAP model • R-GMA in depth • Overview • Server Architecture • Producer and Consumer model • Query and Storage Types • R-GMA Browser • References

  3. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 Grid Information Systems Overview • Collect information of grid resources • Discovering new added resources • Monitoring load and health status • Publish these information • Periodically updated • Well know data model • Used by • Users searching a concrete resource • WMS allocating and managing jobs • Other monitoring services

  4. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 gLite Information Systems • Basic data model. • Grid Laboratory Uniform Environment (GLUE) Schema. • Two architectures in glite3 • LCG Information System • BDII over Globus MDS. • OpenLDAP interface. • Preserves LCG-2 compatibility. • Relational Grid Monitoring Architecture (R-GMA) • Based on the GMA standard from the Grid Global Forum • Information in SQL relational databases • Web Services. • Future replacement of LCG IS.

  5. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 GLUE Schema

  6. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 GLUE Schema • A schema of objects and attributes describing Grid resources and its relationships. • Originally a EU-DataTAG and US-iVDGL coordinated effort. • Current participants: EGEE, OSG, Globus and NorduGrid. • A way to describe Grid info • Statically and dynamically supplied • Hierarchically represented • Independently of the framework (LDAP, XML, SQL…) • Actual release (1.2) is mapped into • LDAP • XML • Old ClassAd (used by Condor Matchmaking)

  7. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 Site Element

  8. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 Cluster Element

  9. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 Computing Element

  10. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 LCG Information System

  11. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 LCG Information System Levels • Resource level: Grid Resource Information Server (GRIS) • One GRIS on top of each CE, SE, RB, MyProxy (no WNs). • Sensors and scripts get status of concrete resources statically (e.g. GlueCEUniqueID) or dynamically (e.g. GlueCEStateWaitingJobs) • Site level: Grid Index Information Server (GIIS) • Compiles all the information of the different GRISes in a site. • LCG-2 recommends using a BDII instead of a GIIS • Improves robustness and stability. • Called the site BDII. • Top level: Berkeley DB Information Index (BDII) • Keeps all Grid information about the VOs (generally only one). • Stores information from local BDIIs or GIISes in its database. • Only queries sites that are included in a configuration file.

  12. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 LCG Information System Hierarchy

  13. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 A LDAP Model • Way of collecting info • Pull model (higher level servers periodically query lower level servers) • All servers are based on LDAP • Inherit hierarchical structure (tree-like) • LDAP Data Information Format (LDIF) • Users get info with • Generic applications • ldapsearch (MDS: 2135 and BDII:2170 ports) • Graphical (BDII web; LDAP GUIs) • Always can get information about specific resources (maybe more up-to-date) by querying directly the site BDIIs, GIISes or GRISes. • Querying VO info with lcg-infosites or lcg-info

  14. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 Relational-GMA

  15. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 Grid Monitoring Architecture(GMA) PRODUCER REGISTRY CONSUMER • The Producer stores its location (URL) in the Registry. Store location • The Consumer looks up producer URLs in the Registry. • The Consumer contacts the Producer to get all the data or the Consumer can listen to the Producer for new data. Transfer Data Lookup location

  16. R-GMA within Testbed 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006

  17. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 R-GMA: Schema-Registry-Mediator TABLE 1,Producer P1 details TABLE 1, Colum defs TABLE 2,Producer P1 details TABLE 2, Colum defs TABLE 2,Producer P2 details TABLE 3, Colum defs TABLE 2,Producer P3 details TABLE 4, Colum defs TABLE 3,Producer P2 details SCHEMA TABLE 3,Producer P1 details TABLE 3,Producer P3 details MEDIATOR REGISTRY SCHEMA : it holds the names and definitions of all of the tables in the virtual database, and their authorization rules. R-GMA Server VIRTUAL DATABASE REGISTRY: It holds the details of all producers that are publishing to tables in the virtual database and it also holds the details of “continuous” consumers. MEDIATOR: a set of rules for deciding which data providers to contact for any given query.

  18. R-GMA: Producer-Consumer P1 TABLE 1,Producer P1 details TABLE 1, Colum defs TABLE 2,Producer P1 details TABLE 2, Colum defs TABLE 2,Producer P2 details P2 C1 TABLE 3, Colum defs TABLE 2,Producer P3 details TABLE 4, Colum defs TABLE 3,Producer P2 details SCHEMA TABLE 3,Producer P1 details C2 TABLE 3,Producer P3 details MEDIATOR P3 REGISTRY Producers: are the data providers for the virtual database. Writing data into the virtual database is known as publishing, and data is always published in complete rows, known as tuples. There are three types of producer: Primary, Secondary and On-demand. R-GMA Server VIRTUAL DATABASE SQL “INSERT” SQL “SELECT” Consumer: represents a single SQL SELECT query on the virtual database. The query is matched against the list of available producers in the Registry. The consumer service then selects the best set of producers to contact and sends the query directly to each of them, to obtain the answer tuples. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006

  19. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 Producer Types Queries User Code User Code User Code Producer Service Producer Service Producer Service Producer API Producer API Producer API C C C P Control and inserted tuples Tuples Tuple Storage Tuple Storage Tuples SELECT * Queries Control only Tuples Tuples Queries Queries Control only Tuples User Code • Primary Producer • Secondary Producer • On-Demand Producer

  20. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 Query and Storage Types P1 TABLE 1,Producer P1 details TABLE 2,Producer P1 details TABLE 2,Producer P2 details TABLE 2,Producer P3 details TABLE 3,Producer P2 details TABLE 3,Producer P1 details TABLE 3,Producer P3 details REGISTRY • Continuous: as soon as new data becomes available it is broadcast to all interested parties. • Latest: correspond to intuitive idea of current information. • History: return time sequenced data. Tuple-store can be in Memory or Database P1 LATEST RETENTION PERIOD (LRP) and HISTORY RETENTION PERIOD (RTP) allow producers to periodically purge old tuples, and to give a precise meaning to the “current state”. Latest-store Continuous&History-store

  21. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 Continuous Producer API Insert Producer Servlet Schema SQL “CREATE TABLE” SQL “INSERT” Store location Store table description Registry Continuous Lookup location Consumer API Query Consumer Servlet SQL “SELECT” Result Set

  22. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 History or Latest Producer API Insert Producer Servlet Schema SQL “CREATE TABLE” SQL “INSERT” Store location Store table description Registry Query Lookup location Consumer API Query Consumer Servlet SQL “SELECT” Result Set

  23. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 R-GMA APIs • APIs exist in Java, C, C++. • For clients (servlets contacted behind the scenes) • They include methods for… • Creating consumers • Creating primary and secondary producers • Setting type of queries, type of produces, retention periods, time outs… • Retrieving tuples, inserting data • … • You can create your own Producer or Consumer.

  24. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 The R-GMA Browser • The easiest way to try out R-GMA. • It is installed on the machine running the Registry and Schema: https://egris198.eela.ufrj.br:8443/R-GMA • Using the Browser you can do the following. • Browse the tables in the schema. • Look at the table definitions. • See all the available producers for a table. • Query a table. • Query only selected producers.

  25. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 https://egris198.eela.ufrj.br:8443/R-GMA

  26. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 References • BDII exercises on EGRIS Wiki: http://egris200.eela.ufrj.br/twiki/bin/view • GLUE Schema • http://glueschema.forge.cnaf.infn.it/ • LCG-2 User Guide • https://edms.cern.ch/file/454439//LCG-2-UserGuide.html • gLite 3.0 User Guide • https://edms.cern.ch/file/722398/1.1/gLite-3-UserGuide.pdf • R-GMA home page • http://www.r-gma.org/ • R-GMA in EGEE • http://hepunx.rl.ac.uk/egee/jra1-uk/

  27. 1° EELA Grid School, Itacuruçà Island, State of Rio De Janeiro (Brazil), 04-15.12.2006 Questions…

More Related