1 / 17

VLab: Collaborative Grid Services and Portals to Support Computational Material Science

This paper presents VLab, a Grid and Web Service-based system that enables distributed and collaborative computational material science applications. It focuses on job preparation and submission, job monitoring, data storage and analysis, as well as distributed collaboration. The paper discusses research issues and solutions related to managing user inputs and job submissions, integrating applications with Grid messaging infrastructure, and managing plane wave self-consistent field calculations. It also describes the use of PWSCF Input Forms and JSF Grid Beans for task management and constructing task graphs.

ghurley
Download Presentation

VLab: Collaborative Grid Services and Portals to Support Computational Material Science

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. VLab: Collaborative Grid Services and Portals to Support Computational Material Science Mehmet Nacar, Mehmet Aktas, Marlon Pierce, Zhenyu Lu, Gordon Erlebacher, Dan Kigelman, Evan F. Bollig, Cesar De Silva, Benny Sowell, and David A. Yuen GCE’ 05 18 November 2005 Seattle, WA

  2. Introduction • Grid and Web Service-based system for enabling distributed and collaborative computational chemistry and material science applications for the study of planetary materials. • The requirements of The Virtual Laboratory for Earth and Planetary Materials (VLab) include • job preparation and submission, • job monitoring • data storage and analysis • distributed collaboration. • These components are divided into client entry (input file creation, visualization of data, task requests) and backend services (storage, analysis, computation). • Clients and services communicate through NaradaBrokering • We describe two aspects of VLab in this paper: • Data entry and submission, • Visualization web client/service.

  3. Research Issues • VLab presents several interesting problems. Driving Grid research issues include the following: • Persistently managing user inputs as archived, hierarchical project metadata. • Simplifying complicated, multi-staged job submissions and monitoring using Grid portal technology. • Integrating VLab applications with Grid messaging infrastructure to virtualize resource usage, provide fault tolerance, and enable collaboration.

  4. Managing Plane Wave Self Consistent Field (PWSCF) Calculations User session archive management, since VLab submissions may involve dozens or hundreds of simulation runs per user. We also anticipate the addition of more codes from the Quantum Espresso suite and the need to couple these into workflows.

  5. PWSCF Input Forms

  6. JSF Grid Beans • Using one bean that is a bean factory for GenericGridTask beans. • We create and manage multiple beans for each task. • That is, I submit the job four times in one session. • Similarly, we can create multiple task graph clones. • Beans have listeners and maintain state. • Unsubmitted, submitted, active, suspended, resumed are “live” • Stored in live repository • Failed, canceled, completed, unknown are “dead” • Stored in archive (WS-Context or other) • JSF grid beans can be easily serialized with XML. • Castor, XML Beans • Marshal and un-marshal user input for persistent storage in XML storage services • OGSA-DAI, GPIR, WS-Context • Data model classes handles monitoring of submitted jobs. These classes handled by JSF Data Table element.

  7. Task Management Class Structure

  8. Constructing Task Graphs

  9. Managing Grid Tasks and TaskGraphs • Task Manager handles independent user requests, or tasks, from the portlet client in Grid services. • The user request-generating objects are simply Java Bean class instances that wrap common Grid actions (launching remote commands, transferring data, performing remote file operations) using Java COG classes. • TaskGraph Manager handles multiple-step task submission • The TaskGraph Manager coordinates user requests with TaskGraph backing beans. • Each TaskGraph bean is itself composed of GenericGridBean implementation instances (for file transfer, job submission, etc.). • Express the dependencies using JSF tag library extensions, so that the JSF application developer can encode the composite task graph workflow out of reusable tags. • The TaskGraph Manager submits and monitors TaskGraphs through an action method when the user launches the job.

  10. <%@taglib uri="http://java.sun.com/jsf/core" prefix="f"%> <%@taglib uri="http://java.sun.com/jsf/html" prefix="h"%> <%@taglib uri="http://www.ogce.org/gsf/task" prefix="o"%> <html> <body> <f:view> <h:form id="form" > <h2>Fill out the form below</h2> <h4>(*) fields required</h4> <o:taskGraph id="myGraph" method="#{taskgraph.test}" > <o:task id="task1" method="task.create" type="FileTransfer" /> <o:task id="task2" method="task.create" type="JobSubmit" /> <o:task id="task3" method="task.create" type="FileTransfer" /> <o:taskAdd name="task1" method="taskgraph.add" /> <o:taskAdd name="task2" depends="task1" method="taskgraph.add" /> <o:taskAdd name="task3" depends="task2" method="taskgraph.add" /> </o:taskGraph> <h:panelGrid columns="3" > <h:outputText value="Hostname (*) "/> <h:inputText value="#{task.hostname}"/> </h:panelGrid> <h:panelGrid columns="3" > <h:outputText value="Provider (*) "/> <h:inputText value="#{task.provider}"/> </h:panelGrid> <h:panelGrid columns="3" > <h:outputText value="Task Name (*) "/> <h:inputText id="taskname" value="#{task.taskname}" /> </h:panelGrid> <h:panelGrid columns="3" > <h:outputText value="Executable (*) "/> <h:inputText value="#{task.executable}"/> </h:panelGrid> <h:panelGrid columns="3" > <h:outputText value="Arguments "/> <h:inputText value="#{task.arguments}"/> </h:panelGrid> <h:panelGrid columns="3" > <h:outputText value="Std Output "/> <h:inputText value="#{task.stdout}"/> </h:panelGrid> <h:panelGrid columns="3" > <h:outputText value="Std Error "/> <h:inputText value="#{task.stderr}"/> </h:panelGrid> <h:panelGrid columns="2"> <h:commandButton id="submit" value="Submit" action="#{taskgraph.submitAction}"/> <h:commandButton value="Clear" type="Reset"/> </h:panelGrid> </h:form> </f:view> </body> </html> TaskGraph Submission Form Corresponding JSF snippets <o:taskGraph id="myGraph" method="#{taskgraph.test}" > <o:task id="task1" method="task.create" type="FileTransfer" /> <o:task id="task2" method="task.create" type="JobSubmit" /> <o:task id="task3" method="task.create" type="FileTransfer" /> <o:taskAdd name="task1" method="taskgraph.add" /> <o:taskAdd name="task2" depends="task1" method="taskgraph.add" /> <o:taskAdd name="task3" depends="task2" method="taskgraph.add" /> </o:taskGraph> gdfgfd <h:panelGrid columns="3" > <h:outputText value="Hostname (*) "/> <h:inputText value="#{task.hostname}"/> </h:panelGrid> <h:panelGrid columns="3" > <h:outputText value="Provider (*) "/> <h:inputText value="#{task.provider}"/> </h:panelGrid> <h:panelGrid columns="2"> <h:commandButton id="submit" value="Submit" action="#{taskgraph.submitAction}"/> <h:commandButton value="Clear" type="Reset"/> </h:panelGrid>

  11. Task Monitoring with JSF Data Model <h:dataTable value="#{jobData.jobs}" var="job"> <h:column> <f:facet name="header"> <h:outputText style="font-weight: bold" value="Job ID" /> </f:facet> <h:outputText value="#{job.jobId}"/> </h:column> <h:column> <f:facet name="header"> <h:outputText style="font-weight: bold" value="Submit Date" /> </f:facet> <h:outputText value="#{job.submitDate}"/> </h:column> <h:column> <f:facet name="header"> <h:outputText style="font-weight: bold" value="Finish Date" /> </f:facet> <h:outputText value="#{job.finishDate}"/> </h:column> <h:column> <f:facet name="header"> <h:outputText style="font-weight: bold" value="Status" /> </f:facet> <h:outputText value="#{job.status}"/> </h:column> </h:dataTable> Corresponding Java class. public class Job { private String jobId; private String status; private String submitDate; private String finishDate; }

  12. NaradaBrokering • Publish/subscribe paradigm, • Reliable/robust flexible messaging. • Middleware infrastructure is designed around a scalable distributed network of cooperating message routers and processors. • NaradaBrokering supports • - High-performance Collaborative environments. • - Core Web and Grid capabilities • Current Pub/Sub Capabilities • Multiple transport support • Subscription Formats • Messaging Related Compliance • Current Grid/Web Application Support • Reliable delivery • Ordered delivery • Recovery and Replay • Security • Message Payload options • Grid Application Support • Web Services

  13. Collaborative Web Services (1)

  14. Collaborative Web Services (2) • A network of three NaradaBrokering brokers is deployed across Indiana University, Florida State University and University of Minnesota. Attached to the network are two wavelet services, with their service adaptors, two schedulers, and several client adaptors. Clients can access the wavelet service via a wavelet applet using standard browsers.

  15. Visualization • In our wavelet application, the applet displays the coefficients as spheres centered at the location occupied by the center of the 3D wavelet. High performance graphics are obtained through the use of JOGL, an OpenGL API for Java.

  16. Conclusion and Future Work • We have addressed three important components of this framework: data entry, job submission, and backend services. Our work is guided by the principle of ease of use, fault tolerance, collaboration, and persistent records. • Our future work will focus on improving the flexibility of collaborative environment • Utilizing data flow model to formulate the task execution and providing the directory service to search the available services. • The Web Service Resource Framework and particularly the WS-Notification specification family may be used to replace our pre-Web Service topic system. • Support for WS-Notification is currently being developed in NaradaBrokering. When this is available we will evaluate its use in our system.

  17. Conclusion • Although we include at two entities of each type in our distributed system (2+ schedulers, 2+ wavelet services, etc.), there is as yet no attempt to take network and server load into account when choosing which units will perform the actual work. It is currently a first come first chosen approach. We will evaluate existing work and enhance our schedulers and services to activate themselves based on a more realistic measure of instantaneous or extended load. • Collaboration is a natural attribute of our system. Two user tasks that subscribe to identical topics automatically receive the same information. We will investigate approaches to achieve this collaboration both at the visual level (shared user interfaces and displays), with the possibility of multiple users controlling the input. Much work has been done in this area, albeit (to the authors’ knowledge) not within the context of publish/subscribe middleware. • Complex workflows are important within VLab. Recent research has shown how to implement strategies for specifying workflows across multiple services. This work will e integrated within our system to properly link input, job submission, analysis, feedback to the user, and finally, automatic (or semi-automatic) decisions regarding the next set of simulations to submit.

More Related