SIG Container and grid job middleware

1. SIG Container and grid job middleware by ZhenChun Huang @ WGISS-21, Budapest 8 May, 2006

2. What is an SIG container? • SIG container, or named SIG service and application container, is an integrated software package for SIG service node providers and application node providers. • It packs the software components frequently used by SIG into a single file for downloading and installing easily. • SIG container provides the basic running environment for the services and applications of SIG.

3. Hard to choose the components and install them for SIG service node Difficult to monitor and manage service nodes Often not suit for spatial information services and applications Integrated package simplifies the deploy of an SIG service node Embedded node monitor service and version service help us monitor and manage service SIG nodes by a universal interface Spatial information application oriented extensions help the SIG containers fulfill the requirements of SIG users Why an SIG Container is needed? ………… Without SIG container With SIG container

4. What does SIG container pack • A Tomcat servlet container (ver 5.0.30 now) and some java class libraries needed, such as Axis, UDDI4J, OSWorkflow, BeanShell, etc. • Version Service • Embedded user authentication and authorization framework, and a handler based on OSUser library • Node Monitor Service • SIG Resource Registry Service and jUDDI, a UDDI service provided by Apache Group • Grid Job framework and middleware ps: The red items are designed and developed by us (SIG team)

5. What does SIG container pack (cont.) • The packed java class libraries: • Axisfrom Apache Group (ver 1.2 now and will be updated to ver 1.3 soon), to implement SOAP protocol and support WSDL • UDDI4J from IBM(ver 2.0.3 now), to provide a UDDI client class library • OSCore, OSUser and OSWorkflow from OpenSymphony, to provide the basic user authentication and authorization support and workflow support (GWFL, a OSWorkflow based Grid workflow was introduced in WGISS 18, 2004) • BeanShell from http://www.beanshell.org, to support the BeanShell Script Language, a Lightweight Scripting for Java • Groovy from Codehaus, to support the Groovy Script Language, an agile dynamic language for the Java Platform • a pure java SQL DB Engine named HSQLDB (ver 1.7.0 now)from http://www.hsqldb.org, to provide a embedded database support • Some JDBC drivers, to connect DBs, includes mySQL and hSQLDB • Service clients of embedded SIG system services • Other class libraries needed such as common-logging, xerces and jdom, wsdl4j, etc. • Java core of Globus Toolkit 4 (GT4) will be packed soon.

6. Download and install • The container package packed by RAR can be download freely from: http://159.226.224.52/login1/weblogin.asp with a guest user named “user” • A JDK (1.3 or later) is needed to install and run the container, it can be download from: http://java.sun.com/. Notice: not JRE, but JDK. • A environment named “JAVA_HOME” should be set to the home directory of JDK (such as C:\jdk1.5.0_01). • Extract the container package to a local directory by rar or winrar. • Extract the packed storefiles (storefile.zip in the root directory container package extracted to) to C:\storefiles (in windows system) or other directory (in linux system) • In M$ Windows system, copy the file “win32probe.dll” in the root directory to the windows system directory (often C:\windows\system32). It is used by the monitor service • In Linux system, modify the file “sig.xml” in “conf” directory, set the <conf-root> to the directory which “storefiles” is extract to. • (for meta-edition only) Set up the data source for registry service • Start the container by running “startup.bat” or “startup.sh” in the “bin” directory • Enjoy it!!

7. Version Service • A very simple web service to announce the version information of the container which is embedded in the container when it is released. • There are 2 editions of SIG container, their version numbers are different each other • Lite edition • For the SIG nodes which provide the application service only. • Packs the necessary class libraries and grid job framework • Meta edition • For the SIG nodes which provide both the application service and the system service • Besides all the components in Lite version, packs UDDI centre, SIG resource registry service and other management services additonally • Version information includes edition information, a major version number, a minor version number and a build number. • Current Version • Lite Edition: 0.4.005 • Meta Edition: 0.4.005

8. User A&A interface for services and applications User A&A framework …… OSUser Handler Other Handler Other Handler User authentication and authorization framework • Products a simple user authentication and authorization interface for services and applications • Connect with different user A&A handlers in one framework by a XML based configuration file • A handler based on OpenSymphony OSUser library is implemented

9. public class OSUserAuthHandler implements IAuthHandler { public boolean isRole(String userReference, String[] role) { try { Document doc = new SAXBuilder().build(new StringReader(userReference)); String name; if ("OSUserAuth".equals(doc.getRootElement().getChildText("type"))) { name = doc.getRootElement().getChildText("ref"); UserManager um = UserManager.getInstance(); try { for (int i = 0; i < role.length; i++) { if (um.getUser(name).inGroup(role[i]) || role[i].equals("*")) { return true; } } } catch (EntityNotFoundException e) { System.out.println("No Such User: " + name); } return false; } else { return false; } } catch (Exception ex) { ex.printStackTrace(); } return false; } public String authenticateAndGetReference(String authInfo) { if (authInfo != null) { UserManager um = UserManager.getInstance(); try { String username = getUsername(authInfo); String password = getPassword(authInfo); if (um.getUser(username).authenticate(password)) { Element e = new Element("userReference"); Element ref = new Element("ref"); …… } } String token = service.getToken(adminInfo); System.out.println("AuthToken got: "); System.out.println(token); String id = service.createJob(jobReference, token); System.out.println(id); service.setParameter(id, "a", new Integer(1).toString(), token); service.setParameter(id, "b", new Integer(2).toString(), token); service.setParameter(id, "wait", new Integer(3).toString(), token); service.startJob(id, token); System.out.println("Job started at: " + Calendar.getInstance().getTime().toString()); while (!(JobStatus.finished.equals(service.getJobStatus(id, token))) || (JobStatus.failed.equals(service.getJobStatus(id, token)))) { Thread.sleep(1000); System.out.print("."); } System.out.println(); if (JobStatus.finished.equals(service.getJobStatus(id, token))) { System.out.println("Job finished at: " + Calendar.getInstance().getTime().toString()); String ret = service.getResult(id, "result", token); System.out.println("Get result: " + ret); } else { System.out.println("Job failed at: " + Calendar.getInstance().getTime().toString()); } service.destroyJob(id, token); System.out.println("Job destroyed."); System.out.println(service.getJobStatus(id, token)); service.discardToken(token); User authentication and authorization framework (cont.) • Usage (in the coming new version of SIG container) • Invoke AuthTool.getToken(…) to get the authorization token • Operate with the authorization token, check the token by AuthTool.checkToken(…) and AuthTool.isRole(…) • Discard the token by AuthTool.discardToken(…) when the operation accomplished • Handler development (in the coming new version of SIG container) • A user A&A handler should implement a java interface named IAuthHandler with 2 methods: authenticateAndGetReference(…) and isRole(…) • There can be more than 1 handler in a user A&A framework, they organize a handler chain to handler all user A&A request

10. Config file Monitor service interface client Monitor implementation Node Monitor Service • Monitor all the SIG nodes by a universal interface based on Web Service • In the node monitor service, a node has a series of properties, some of them are read-only, and the others are writable • Different nodes may have different series of properties, but some key properties such as “Architecture”, “CPUStatus” and “MemoryStatus” must be supported by all nodes in SIG • By read the properties, the node can be monitored • By write the writable properties, the node can be managed

11. package org.thgrid.monitor.gui; import javax.xml.rpc.ServiceException; import java.rmi.RemoteException; class MonitorServiceThread extends Thread { public NodeData data = new NodeData(); public PropertyItem pi; public class MonitorService { public static String setProperty(String url, String name, String value) throws ServiceException, MalformedURLException, RemoteException { return getBinding(url).setProperty(name, value); } public static NodeData[] getAllData(Property prop) { int size = prop.getNodes().size(); MonitorServiceThread[] threads = new MonitorServiceThread[size]; for (int i = 0; i < size; i++) { threads[i] = new MonitorServiceThread(); threads[i].pi = (PropertyItem) prop.getNodes().get(i); threads[i].start(); } NodeData[] ret = new NodeData[size]; for (int i = 0; i < size; i++) { try { threads[i].join(); } catch (InterruptedException ex) { ex.printStackTrace(); } ret[i] = threads[i].data; } return ret; } } package org.thgrid.monitor; import java.net.*; import java.util.*; public class DefaultMonitor implements IMonitor { private static final String[] availableProperties = { "Architecture", "ProcessorType", "ProcessorNumber", "MemorySize", "DiskSize", }; public String SetProperty(String name, String value) throws MonitorException { String oldvalue = staticProps.getProperty(name, "nostaticprop"); if ("DiskSize".equals(name)) { String diskStatus = csip.probe.winprobe.status.GetStorageStatus. getStorage(); java.util.StringTokenizer st = new StringTokenizer(diskStatus, ","); int size = 0; for (int count = 0; st.hasMoreTokens(); count++) { String s = st.nextToken(); if (count % 5 == 1) { s = s.substring(0, s.length() - 1); size += Integer.parseInt(s); } } return String.valueOf(size) + "M"; } if ("nostaticprop".equals(value)) { throw new MonitorException("Can't set static property's value."); } else { throw new MonitorException("Set property is not supported now."); } } } Node Monitor Service (cont.) • Usage: • Get the properties list by method getAvailableProperties(…) • Get the properties by method getProperty(…), getProperties(…) and getAllProperties(…) • Set the value of property by method setProperty(…) • These methods should be invoked in “pull” mode (the client launch it) • Monitor implementation development • Nodes with different architecture should have different monitor implementation, M$ Windows and Linux is supported now • To develop a new monitor implementation, just implement the java interface IMonitor with the following methods: getName(), getAvailableProperties(), getProperty(…) and setProperty(…) • There should be a unique monitor implementation in a monitor service, it is configured by a config file

12. package org.thgrid.monitor.gui; import javax.xml.rpc.ServiceException; import java.rmi.RemoteException; class MonitorServiceThread extends Thread { public NodeData data = new NodeData(); public PropertyItem pi; public class MonitorService { public static String setProperty(String url, String name, String value) throws ServiceException, MalformedURLException, RemoteException { return getBinding(url).setProperty(name, value); } public static NodeData[] getAllData(Property prop) { int size = prop.getNodes().size(); MonitorServiceThread[] threads = new MonitorServiceThread[size]; for (int i = 0; i < size; i++) { threads[i] = new MonitorServiceThread(); threads[i].pi = (PropertyItem) prop.getNodes().get(i); threads[i].start(); } NodeData[] ret = new NodeData[size]; for (int i = 0; i < size; i++) { try { threads[i].join(); } catch (InterruptedException ex) { ex.printStackTrace(); } ret[i] = threads[i].data; } return ret; } } package org.thgrid.monitor; import java.net.*; import java.util.*; public class DefaultMonitor implements IMonitor { private static final String[] availableProperties = { "Architecture", "ProcessorType", "ProcessorNumber", "MemorySize", "DiskSize", }; public String SetProperty(String name, String value) throws MonitorException { String oldvalue = staticProps.getProperty(name, "nostaticprop"); if ("DiskSize".equals(name)) { String diskStatus = csip.probe.winprobe.status.GetStorageStatus. getStorage(); java.util.StringTokenizer st = new StringTokenizer(diskStatus, ","); int size = 0; for (int count = 0; st.hasMoreTokens(); count++) { String s = st.nextToken(); if (count % 5 == 1) { s = s.substring(0, s.length() - 1); size += Integer.parseInt(s); } } return String.valueOf(size) + "M"; } if ("nostaticprop".equals(value)) { throw new MonitorException("Can't set static property's value."); } else { throw new MonitorException("Set property is not supported now."); } } } Node Monitor Service (cont.) • The Web-based Real-time Monitor • Based on node monitor service, a real-time monitor is developed in Java Applet. It can monitor the CPU status and memory status of several nodes pre-configured at the same time • The following is the user interface

13. SIG Resource Registry Service • The SIG container packs a UDDI server named jUDDI by Apache Group, but UDDI does not suit for SIG applications very well • So, a light weight distributed resource registry service is developed and packed in the Meta edition of SIG Container • The SIG Resource Registry Service can be accessed by standard SOAP protocol • And by the support of client library, user can search resources in multi registry services simultaneously • Passes tests on the following DB engines: SQLServer, HSQLDB, mySQL and all ODBC databases

14. <?xml version="1.0"?> <gridService xmlns ="http://grid.cs.tsinghua.edu.cn/GRSL/schema"> <serviceKey>1342667F432D7CCA</serviceKey> <name>Resource Registry Service</name> <description>the grid resource registry meta-service. It can register, manager and search resources in grid.</description> <support>the service is managed by Grid Research Lab. Department of Computer Science and Engineering, Tsinghua University. Mail to support@grid.cs.tsinghua.edu.cn for technology support.</support> <accessPoint>http://grid.cs.Tsinghua.edu.cn/GRSL/services /registry<accessPoint> <userName>root</userName> <categoryBag> <keyReference> <keyName>class</keyName> <keyValue>grid.meta.registry</keyValue> </keyReference> <keyReference> <keyName>access</keyName> <keyValue>soap</keyValue> </keyReference> <keyReference> <keyName>supportFunctions</keyName> <keyValue>GetService,RegisterService, ModifyService,UnregisterService,FindService </keyValue> </keyReference> </categoryBag> </gridService> SIG Resource Registry Service (cont.) • When an SIG resource is registered, the following information should be given: • Resource name • Resource description (for read manually) • Resource support information (such as manager’s name, email for bug reporting) • Access point of the resource (it should be a string, how to use it depends on the type and class of the resource) • Resource category bag which includes a set of features (key/value pair) to describe the resource • When register succeed, a unique ID is assigned • The registered resources can be referenced by the ID and searched by its name or features in category bag • The features in category bag can be different for different types of resources. The category bag makes it possible to define some necessary or choosable key/value pairs for application-oriented service domains, just like it is done in SIG

15. Resource extensions for SIG • In SIG, 5 types of resources are defined currently, they are: • Node (node): the computers who serve for the SIG with installed SIG container • System service resources (system): the SIG meta-services for system functions in SIG • such as resource finding, status monitor, and version control, includes sub-types: system.registry, system.bank, system.version, system.monitor, etc. • Visual service resources (visual): the services for the advanced visualization in SIG • includes sub-types: visual.ogc-wms, visual.ogc-cps, visual.vis, etc. • Data service resources (data): the resources for producing data in SIG • includes sub-types data.datasource, data.discovery, data.catalogue, etc. • Process resources (process): the resources for processing the SIG data • includes sub-types: process.simple (for the simple service mode processing resource), and process.job (for the job mode processing resource)

16. Resource extensions for SIG (cont.) • The resource with different type will have different category bags which contains different key/value pairs. • For example, the resource with “data” type should register list of data formats supported as a key named “datasource.format”, and the resource with “process” should register its service protocol as a key named “standard”. • The type of resource is specified by the feature named “class” in the category bag. • By the resource extensions, SIG registry service can register and find SIG resources by more features, it will be more powerful and flexible, and serve better for SIG. • Now, the resource extensions support the SIG registry service only, UDDI have not resource extensions for SIG

17. <?xml version="1.0"?> <gridService xmlns ="http://grid.cs.tsinghua.edu.cn/GRSL/schema"> <serviceKey>1342667F432D7CCA</serviceKey> <name>Resource Registry Service</name> <description>the grid resource registry meta-service. It can register, manager and search resources in grid.</description> <support>the service is managed by Grid Research Lab. Department of Computer Science and Engineering, Tsinghua University. Mail to support@grid.cs.tsinghua.edu.cn for technology support.</support> <accessPoint>http://grid.cs.Tsinghua.edu.cn/GRSL/services /registry<accessPoint> <userName>root</userName> <categoryBag> <keyReference> <keyName>class</keyName> <keyValue>grid.meta.registry</keyValue> </keyReference> <keyReference> <keyName>access</keyName> <keyValue>soap</keyValue> </keyReference> <keyReference> <keyName>supportFunctions</keyName> <keyValue>GetService,RegisterService, ModifyService,UnregisterService,FindService </keyValue> </keyReference> </categoryBag> </gridService> Resource extensions in SIG (cont.)

18. Access.* (description, type, jobdesc, monitor…) • quantity.* (refcount, architecture, storage, bandwidth, …) • parameter.* (description, type…) • datasource.* (region, format, …) • security.* (policy…) • Trade.* (url, …) • support.* (name, version, lastupdate…) • Class • keywords • Name • Description • Support • Accesspoint • CatagoryBag • standard For some types of SIG resources For all SIG resources For general grid resources Resource extensions in SIG (cont.)

19. Grid Job framework and middleware • In SIG, most processing task need a long time, such as a few days, weeks, even months. The classic web service can not support these processing tasks. So, an SIG job framework and its support middleware is designed and developed • It takes the same usage with 5 main steps as the jobs in high performance computing domain • Create the job • Setup parameters of the job • Start the job and wait for the job finish by monitoring its status • Get result of the job • Destroy the job • The user authorization token got by user A&A framework is required in all the 5 steps

20. user Job service Job service Descriptor Reference manages manages locates Job descriptor Job instance instantiation Grid Job framework and middleware (cont.) • Some definitions in the grid job framework • Job instance • An instance of job with its own contexts, including parameter values, result values, job status, etc. • A job instance is born when it is created and die when it is destroyed • A job instance is identified by a universal ID given by the job manager service when the job instance is created. • Job descriptor • A segment of XML string which can describe the interface and implementation of a job • The blueprint of job instances. The relationship between job instance and job descriptor likes it between java object and java class • Descriptor Reference: a XML string to locate the job descriptor • Job manager service: or “job service” shortly, the only entrance of the job framework. All job methods are invoked through this service

21. user Job service Job service Descriptor Reference manages manages locates Job descriptor Job instance instantiation Grid Job framework and middleware (cont.) • Relationship among the definitions

22. Grid Job framework and middleware (cont.) String token = service.getToken(adminInfo); String id = service.createJob(jobReference, token); service.setParameter(id, "a", new Integer(1).toString(), token); service.setParameter(id, "b", new Integer(2).toString(), token); service.setParameter(id, "wait", new Integer(3).toString(), token); service.startJob(id, token); while (!(JobStatus.finished.equals(service.getJobStatus(id, token))) || (JobStatus.failed.equals(service.getJobStatus(id, token)))) { Thread.sleep(1000); } if (JobStatus.finished.equals(service.getJobStatus(id, token))) { String ret = service.getResult(id, "result", token); System.out.println("Get result: " + ret); } else { System.out.println("Job failed at: " + Calendar.getInstance().getTime().toString()); } service.destroyJob(id, token); service.discardToken(token); • Usage: • Get user’s authorization token by getToken(…) • Create a job instance with given description reference and get its ID by createJob(…) • Set parameters of the job instance by setParameter(…) • Start the job instance by startJob(…) and monitor its status by getStatus(…) • When the job accomplishes successfully, get the results of job by getResult(…) • Destroy the job instance by destroyJob(…) • At last, discard the authorization token by discardToken(…) • Descriptor reference can be a job name (for local job descriptor file) or a URL (for a downloadable job descriptor file) • Methods such as listInstance(), listDescriptor() are also supported

23. Job status • The status of a job instance may be: • Ready: a created but not initialized job • Initialized: a job initialized and can be started • Running: a running job • Finished: an accomplished job • Failed: a job running failed • Terminated: a job being terminated by its creator • Notexisted: a non-existed job • Unknown: an unknown status

24. <?xml version="1.0" encoding="UTF-8"?> <JobDescriptor> <interface> <description> A demo job for thGridJob framework. It adds two given integers into one integer as a result, and wait for 10 secondes </description> <parameters> <parameter> <name>a</name> <type>integer</type> <description>the 1st integer to add</description> </parameter> <parameter> <name>b</name> <type>integer</type> <description>the 2nd integer to add</description> </parameter> </parameters> <results> <result> <name>result</name> <type>integer</type> <description>the result of addition</description> </result> </results> </interface> <implementation type="class"> <impl>org.thgrid.job2.impl.ThGridJobImplDemo</impl> <arguments> <argument> <name>wait</name> <type>integer</type> <value>10</value> </argument> </arguments> </implementation> </JobDescriptor> Development of job • Add a new job descriptor (develop a new job) • Create the job implementation by one of these ways: • A java class implements java interface IJobImpl • A BeanShell script or a Groovy script • A executable OSWorkflow with action “1” • Create a job descriptor file to describe the interface and implementation of this job descriptor • Remove a job • Just remove the job descriptor file, the job descriptor can not be accessed again

25. The job descriptor file <?xml version="1.0" encoding="UTF-8"?> <JobDescriptor> <interface> <description> A demo job for thGridJob framework. It adds two given integers into one integer as a result, and wait for 10 secondes </description> <parameters> <parameter> <name>a</name> <type>integer</type> <description>the 1st integer to add</description> </parameter> <parameter> <name>b</name> <type>integer</type> <description>the 2nd integer to add</description> </parameter> </parameters> <results> <result> <name>result</name> <type>integer</type> <description>the result of addition</description> </result> </results> </interface> <implementation type="class"> <impl>org.thgrid.job2.impl.ThGridJobImplDemo</impl> <arguments> <argument> <name>wait</name> <type>integer</type> <value>10</value> </argument> </arguments> </implementation> </JobDescriptor>

26. Why Grid job instead of WSRF? • The job framework is more understandable than WSRF resource, especially for the HPC users • It’s much more complex to develop a WSRF resource than SIG job • To develop a WSRF resource, we must create a WSRF WSDL first. Then, generate the java stubs by wsdl2java toolkit, program at least 2 java classes, and deploy the resource manually to accomplish the development. • To develop a Grid job, we need only create a java class first, and write a descriptor file to deploy the new job descriptor. • The Grid Job framework is light weight and high efficient • The Grid Job framework is enough in function for data processing in SIG.

27. Future Works • Packs more… • e.g. Choose a WSRF implementation such as java core of GT4 and pack it into the container • Serves more… • e.g. Make the container more platform-independent and easy-to-use • Supports more… • e.g. GridJob2: a more powerful and flexible job framework which supports algorithm integration and on-demand job • e.g. SIG Application framework and its support middleware: to make the application development easier and faster

28. Thanks very much!