Architecture Overview

1. ATLAS Grid Information Systemstatus and plansAlexey AnisenkovBudker Institute of Nuclear Physics(BINP, Novosibirsk) ADC Development Meeting, June 2, 2010

2. Architecture Overview • AGIS is database based Information System AGIS stores as external data as own data managed through API/WEB I/F API SERVER WEB Server http/https http/https AGIS DB XML/JSON Clients + Database Content Viewer Clients Clients Clients (XML / JSON) Clients Clients client API (python) External data sources External sources External Python independent API Data Providers ToA, GOCDB, OIM, etc python2.5 API

3. Current Status current release in production: version 0.4.0 • AGIS core client-server architecture is implemented and deployed on production server (data models, web server, api server, client api architectures, authentications, data providers, serialization, cron jobs, etc ) • AGIS API/CLI package is developed and documented • Following (python) API implemented: • Topology API (list sites, clouds, services, storage elements, etc) • Downtime API (list downtimes) • Blacklisting API (list blacklists, get/set blacklist status, list activities) • ToA topology can be exported in XML format proposed by CERN IT • Downtime Calendar represents information collected from GOCDB, OIM sources with special highlighting of downtimes without risk (NO_RISK) for ATLAS • API is in production • Clouds data distribution program uses sites status, availability, downtime info taken from AGIS • API documentation and quick tutorial can be found at AGIS TWiki page

4. Current Status. Documentation. Links • Downtime Calendar:http://lxvm0350.cern.ch:12409/agis/calendar/ • Web server (web client i/f): production: http://lxvm0350.cern.ch:12409 development: http://lxvm0350.cern.ch:1080 • API Servers:production, stable: lxvm0350.cern.ch:12410development, new features introduced: lxvm0350.cern.ch:1081 • TWiki page:https://twiki.cern.ch/twiki/bin/view/Atlas/AtlasGridInformationSystem • Savannah project:https://savannah.cern.ch/projects/atlas-adc-agis • Mailing list:atlas-adc-agis@cern.ch

5. ToDo List from global point of view • ToA migration to AGISImplementation of DQ2.info wrapper functions, development of required APITime estimation: 1-3 months, Priority: Highest • Python independent API (requests from Yu.Andreeva, Laura Sargsyan)Extending server API to be able to response with JSON/XML reply. Update server API to provide description of ATLAS topology in JSON/XML format.Time estimation: 1-2 weeks, Priority: Normal, can be started in parallel • Panda queues and site atlnames(request from Lorenzo Rinaldi) Incorporate queue names into AGIS. Extend API to provide information about site alternative names (panda queues, OIM, GOCDB) stored in AGISTime estimation: 1-2.5 week, Priority: Normal, can be stated in parallel • TiersInfo migration to AGISProvide required API (sharing, pairing) and Web I/F for sites to modify sharesTime estimation: 2 weeks – 1 month, Priority: Normal, can be stated in parallel • AGIS and VOMS integration migration to AGISThe implementation of access policy based on VOMS certificates both for AGIS api and web services. Time estimation: 2-2.5 weeks, Priority: Normal, can be stated in parallel

6. ToDo List from global point of view • AGIS downtime for Central Services Machines (request from I Ueda) Extending AGIS functionality to store and to manage and to show on Calendar site downtimes registered directly by AGIS. Provide required API/CLI to manage them Time estimation: 1-1.5 week, Priority: High • Site Blacklisting in Downtime CalendarThe Add site blacklisting representation to the Calendar Time estimation: 1 week, Priority: Low • Database Content ViewerProvide web based content viewer (web interface) for all data stored in AGIS.Time estimation: 2.5 weeks - 1.5 month, Priority: Low, can be stated in parallel with VOMS integration • ATLAS Squid Cache Information supportExtending AGIS functionality to be able to store, to manage and to expose Squid Information from AGIS Time estimation: 2 weeks – 1.5 month, Priority: Low • Panda ShedConfig and AGIS Need to be discussed, to collect requirements first. Time estimation: > 1.5 month, Priority: Low

7. Migration to new production machines • Following services have been registered for AGIS • production WEB server: atlas-agis.cern.ch • production API server: atlas-agis-api.cern.ch • development WEB server: atlas-agis-dev.cern.ch • development API server: atlas-agis-api-dev.cern.ch • AGIS should be migrated to new serversAlso, the packages restructuring, writing setup scripts should be done to help with easy installation and package rpm building • Time estimation: 1-1.5 week • Priority: low, can be started in parallel

8. ToA & AGIS migration • The final migration can be done in several steps • We release AGIS based DQ2.info package, test it • Then test it with clients not previously used DQ2.info (f.e. TiersInfo application) • Test it again • We periodically synchronize AGIS DB with ToA content until final migration is not finished • We provide tested version of DQ2.info package, DDM developers can test our implementation with the test-suites (I suppose you have something like that to test) • Test it again • When all tests will be successfully passed and all ToA content will be migrated to AGIS, we can switch AGIS based DQ2.info module to primary one, so ToA file become obsolete

9. Ongoing work • Migration from TiersOfATLAS file to AGISThe implementation of DQ2.info wrapper functions based on AGIS API instead of underlying ToA file. The development of required client API. ToA content migration to AGIS DB Time estimation: 1 - 3 months Priority: Highest • Downtime Calendar updates and fixes • AGIS should properly manage canceled downtimes • AGIS should properly mark downtimes with LFC/FTS services affected Time estimation: 1 week Priority: High • API updates and fixes • Several cosmetic fixes should be applied (error reporting, configuration issues, etc) Time estimation: < 1 week Priority: Low, can be started in parallel Nearest checkpoint: • 1st July: 1st release of DQ2.info module implementation (AGIS based)