Configuration for gLite

1. Configuration for gLite Joachim Flammer on behalf of JRA1 integration team

2. Content Idea of this this meeting • Continue discussion where we left off in Padova • Present status & ideas of clusters that are working on configuration • Input from user community • SA1, ARDA, Biomedical Community, PO … • Brainstorming Result of this meeting • Discussions • Find a common path to continue This presentation • Introduction to • Important aspects for configuration • Different parts of configuration • Present status of Integration team work • Future ideas of integration team Configuration

3. gLite challenges Some of the gLite challenges … • Multi platform system • Linux, Windows, … • Multi language system • Java, C++, … • Development done in different places by different groups • IT, CZ, UK, CH, … • Distributed system • Production system • 24/7 system • Multi user environment • Different background (hackers, users, …) • Security • … Configuration of gLite MUST be adapted to these challenges Configuration

4. General issues Corporate approach for configuration management • Keep system homogenous • Common configuration file format • Non duplicated configuration parameters • Common place (file system, DB, …) • Common service instrumentation • Keep system manageable • User friendliness: keep the end user in mind • We are running in an distributed environment • Configuration must have a grid approach • System must be running around the clock • Use modular approach • Separate areas • Allow for changes later – new technologies etc. • Different plug-ins • Look around what industry is doing …. Configuration

5. Configuration areas • Installation • Download of gLite • Storing the configuration • Post installation • Running gLite • Accessing the configuration • from services • from human/system administrator • Managing gLite • Changing configuration • Checking configuration • Backing up of configuration Areas of configuration Configuration

6. Installation • Support of a heterogeneous environment • Multi-platform environment (Linux, MS Windows) • Multiple packaging systems (RPM, tgz, MSI, …) • Compatible with standard installation tools (apt, yum, MS Installer, etc.) and installation/configuration systems (Quattor, SMS, etc.) • Support of automatic installation (e.g. for large sites) and manual installation • Support of user in installation process • Automatic download of necessary components and dependencies • Common handling of service startup/stop • Support update of installation Key aspects Configuration

7. Installation • Post-install configuration • Environment setup • Create the working environment for the service • Copy configuration parameters to service specific files, formats • Creation of Databases … • Service configuration • Configuration of the service at startup and at run-time • Each service is doing configuration differently (… however we are putting a layer on top to hide that …) • Start/stop • global gLite init.d script • standard interface for managing start/stop of gLite services Present status Our post installation script are hiding the configuration … • clearly no long term solution …  minimize post installation implement proper service instrumentation to read the configuration directly Configuration

8. Storing configuration • Storage format • use unique format for configuration • Human readability • enforce structure (key value pair is not enough) • Storage location • Common location for configuration (no scattered around different directories …) • Enable local/centralized configuration • File system, DB, web • Don’t duplicate values • Allow common configuration for whole site • User support • Enforce configuration verification (e.g. via schema) • Guide user in what needs to be changed • Group configuration values according to • User / optimization / system parameters • Support user in choosing the correct configuration • Documentation, sample values, default values, units • Provide proper tools (GUI) • Backup of (last good) configuration Key aspects Services and users should not need to worry how configuration is stored ! Configuration

9. Storing configuration • xml configuration file format • common set of configuration files in $GLITE_LOCATION/etc/config/ • one global file • in general one per service • configuration files support • documentation • arrays • multiple instances • Possibility to combine configuration information for a site Present status Configuration

10. Running gLite – Accessing configuration • Two generic access points • Human/Management side • Application side • Provide easy access of the configuration values by providing APIs • Manager/Human • Manager should have one common place (management console) to look and change the configuration • Manager should be informed of impact of changes – what needs to/will be reconfigured, restarted, updated after a change of configuration • Application • one common interface • Implementations are language and platform dependent • But all services should have the same implementation for a given language/platform Key aspects Configuration

11. Service requirements • Control • LoadConfiguration • Change (configuration of) service dynamically • Lifecycle-specific requirements • start & stop • Status of service (pinging) – alive/dead … • Produce load statistics • Request of service information • describe how many messages it is processing at a given time • display its identification, its current version number • display its current set of dependencies • Manage the performance of services ( … goes together with testing) • response time, uptime • Metering the usage of web services: number of messages from different users • Debugging of services • Make internals visible for debugging, finding of bottlenecks • request that the service sends any error message to a named target or file • SOAP message monitoring/logging …. Requirements for “production quality” (web) services There is a big requirement for a proper service instrumentation !!! Configuration

12. Service Instrumentation Service side • a common approach to allow service to • retrieve/update configuration • provide information about internal state: configuration, resources …  Service Instrumentation User side • a common approach to • get information about the service (status, uptime, load, etc.) • get information about the configuration of the service • change configuration/manipulate service at runtime  Flexible management console Implementation needs to • incorporate the different containers of gLite (Tomcat, gSOAP, …) • have a modular approach using common interfaces • hide implementation details from services • allow to connect to industry standard interfaces (e.g. WBEM) Configuration

13. Changing configuration • Controlled change of configuration • Manager changes configuration via a common management console • Changing the configuration should automatically trigger the necessary actions • Reconfigure applications dynamically for dynamic configuration values (e.g. logging, debug levels) • Restart applications for static configuration values (e.g. port number) • Take into account the dependencies between applications • e.g. to restart tomcat the web services have to be restarted as well • Take into account the state of the application • an application wants to finish a transaction first • Track changes • What is the actual configuration of the application? • What is the last good configuration?  Monitoring Key aspects Configuration

14. Integration Team proposal ServiceConfigurator • sits inside the service and is the unique interface to • access configuration • get configuration values & information about service • give common functionalities (e.g. start/stop) if not implemented by the container • reuse container capabilities gLiteConfigurator • global tool to • manipulate configuration (management console) • start/stop services • use standard interfaces … Possible implementations • Java: • Java Management Extensions (JMX) • Outside access via SOAP / HTML/ …? • Adoption of JNDI ? • Standard WDL interface for service information • C++: • gLite Configurator Configuration

15. Web services - JMX • A possible solution: JMX – Java Management extensions • Standard designed for enabling resource management of/with Java applications • Extension of Java following standard specification • Specification within the Java Community Process (JCP) as a Java Submission Request (JSR3, JSR77, JSR xx Remote) • Several active implementations – commercial and open source • Each implementation follows the standard and gives some extras • Integrated in SUN Java 1.5 • Accepted standard in industry - used in several commercial products • Enables you to do dynamic configuration • Enables you to retrieve configuration information remotely • Enables you to read configuration from different places • …. much more like monitoring etc. … • In addition: • Common set of web service interfaces (e.g. for status request) • Enable dynamic configuration Configuration

16. Web services - JMX Management application Client connector Java VM Adapter Adapter Server Connector Agent Service Agent Service Managed resource Managed resource MBean MBean MBean Distributed ServiceLayer AgentLayer MBean Server InstrumentationLayer Configuration

17. C++ services • C++: gLiteConfigurator • Configuration based on the Service Configurator approach • Service Configurator is designed as an internal part of the service • Role of the Service Configurator : • Hide OS dependent issues from the service (messaging, …) • Create an unique interface for the management of the services • Load service configuration and forward it to the service logic • Enable addition of the additional common functionality (instru-mentation) to the services, by providing an unique interface to it Configuration

18. C++ services Service logic Service Configurator OS Unique OS independent interface OS dependent signals Linux signals Windows messages … start stop init reconfigure info status start stop init reconfigure info status Reading of configuration parameters Service Configuration

19. Possible steps … • Agree on requirements and use cases • from inside gLite point of view • from future users point of view • Agree on a common set of interfaces • What does a service has to provide? • Status, configuration information • How do we want to contact the service • Obtaining information • Sending information • How do we make sure we take different aspects into account? • Security • How to make sure that we can contact a service with high(er) priority? • Backup of configuration • Try out different approaches and decide on implementations • JMX (Java) • gLite Configurator (C++) • Investigate incorporation with existing management solutions … • Start implementing framework and adapt services this meeting this meeting ? Configuration

20. Discussion time … Configuration

21. Backup slides Configuration

22. Example configuration file <!-- Default configuration parameters for the gLite RGMA server --> <config> <parameters> <!-- User-defined parameters - Please change them --> <rgma.server.mysql_root_password description="MySQL root password.” value="changeme"/> <rgma.server.run_schema_service description="Run a schema service for the rgma server on your machine (yes|no). If you want to run it on your machine set 'yes' and set 'rgma.schema.hostname' to the hostname of your machine otherwise set 'no' and set 'rgma.schema.hostname' to the hostname of the schema server you want to use. “ value="changeme"/> <!-- Advanced parameters - Change them if you know what you're doing --> <rgma.server.httpconnector_maxthread description="Maximum number of threads that are created for the tomcat http connector to process requests. This, in turn specifies the maximum number of concurrent requests that the Connector can handle. “ value="1000"/> <!-- System parameters - You should leave these alone --> Configuration

23. Storing configuration • Next (possible) steps: • cleanup of variable naming • Improvement of xml structure • Proper documentation • Sample values … • Schema • verification of configuration files and values via schema • graphical configuration tool • work in progress: • How to store “secret parameters” – e.g. passwords • How to treat values that should be put back to default values • Saving of old configurations • ….. Configuration

24. Accessing configuration • Present status • Copying of configuration • deployment scripts copy configuration to service specific configuration files • Services read their specific configuration files • Scattered configuration files • Individual formats • Steps toward a common solution • Proposal for common configuration/control approach • Web services: JMX (see next slides) • C++: gLite Configurator (see next slides) Configuration

25. Web services - JMX Tomcat Container Service specificWSDL Service service logic Service MBean gLite Manager gLiteService Configuration MBean connector commonWSDL MBeanServer commonWSDL Service adaptor gLiteService Configuration MBean Management Application Tomcat manager service logic Configuration

26. Extra slide: Services using container Management Console GliteConfigurator Container read Service Configurator Service Manager Service Configurator read write read Service logic Service logic command Service description file Container logic Configuration file Configuration

27. Old slides Configuration