Performance of the Relational Grid Monitoring Architecture (R-GMA)

1. Performance of the Relational Grid Monitoring Architecture (R-GMA) • CMS data challenges. The nature of the problem. • What is GMA ? • And what is R-GMA ? • Performance test description • Performance test results • Conclusions IEEE NSS 2003

2. The Nature of the problem • As part of the preparations for data taking CMS is performing DATA CHALLENGES. • Large number of simulated events to optimise detectors and prepare software • Enormous processing requirements BUT each event is independent of all the others each event can be generated on a machine without any interaction with any other IEEE NSS 2003

3. The local solution Work split between farms. How to handle the book-keeping ? a data-base automatically updated Implemented via a job wrapper BOSS Output to <stdout> and <stderr> is intercepted and the information is recorded in a mySQL production database. Event generation and job accounting decoupled IEEE NSS 2003

4. The local solution (schematic) Worker Node (WN) Database Machine WN WN WN WN WN Submission Machine UI WN WN WN IEEE NSS 2003

5. The grid solution (schematic) Database Machine Submission Machine UI IEEE NSS 2003

6. register producer Ask for data locate producer address of producer Grid Monitoring Architecture (GMA) of the GGF Producer data data data data Registry (Directory services) data data data data Consumer IEEE NSS 2003

7. Developed for E(uropean) D(ata) G(rid) • Extends the GMA in two important ways • Introduces a time stamp on the data. • A relational implementation • Hides the registry behind the API R-GMA (Relational GMA) Can be used for information and monitoring Each Virtual Organisation appears to have one RDBMS IEEE NSS 2003

8. The syntax of R-GMA The user interface to R-GMA is via SQL statements (not all SQL statements and structures are supported) Information is advertised via a table create Information is published via insert Information is read via select … from table The first read request registers the consumer as interested in this data. Relational queries are supported NOTE : sql is the interface – it should not be supposed an actual database lies behind it. IEEE NSS 2003

9. Fit between R-GMA and BOSS R-GMA can be dropped into the framework with very little disruption • Set up calls for mySQL are replaced by those for R-GMA producers • An archiver (joint consumer/producer) runs on a single machine which collects the data from all the running jobs and writes it to a local database (and possible republishes it). The data can then be queried either by direct mySQL calls or via R-GMA consumer (a distributed database has been created) IEEE NSS 2003

10. R-GMA R-GMA R-GMA R-GMA R-GMA R-GMA R-GMA R-GMA Fit between R-GMA and BOSS (i) LAN Connection WAN Connection Database BOSS IEEE NSS 2003

11. R-GMA Measurements • The architecture of GMA clearly provides a putative solution to the wide area monitoring problem. BUT Does a specific implementation provide a practical solution Before entrusting CMS production to R-GMA, we must be confident that it will perform. What load will it fail at and why ? IEEE NSS 2003

12. Message time distribution from 44 jobs <Message length> 35 chars. IEEE NSS 2003

13. R-GMA servlet R-GMA consumer Simulation of a CMS job Multi-threaded job each thread produces messages. Length 35 chars, suitable distribution. Threads starting time distribution can be altered. One machine delivers the R-GMA load of a farm. IEEE NSS 2003

14. R-GMA servlet R-GMA servlet R-GMA servlet R-GMA servlet R-GMA consumer Simulation of the CMS Grid One machine per grid cluster providing loads of greater than the cluster IEEE NSS 2003

15. Current status R-GMA can survive loads of around 20% of the current CMS requirements and does provides a grid method for monitoring. An overload of a factor 2 jobs causes problems after about five minutes running. We believe these instabilities are soluble. When production starts in earnest we will compare reality with our model. IEEE NSS 2003