GridChem Refactoring: Workflows in GridChem

1. GridChem Refactoring: Workflows in GridChem Sudhakar Pamidighantam spamidig@ncsa.edu

2. Acknowledgements • Rion Dooley, TACC • Suresh Marru, IU • Yang Liu, NCSA • Raman Sandhu, IU • Nikhil Singh, NCSA • Gaurang Mehta, ISI NSF and TeraGrid

3. GridChem RefactoringOutline • GridChem functioning • New needs and requirements • Refactoring • ParamChem • Future usage

4. GridChem Functioning • CCG Virtual Organization - Allocations, Consulting, User services • Three Tier Client, Middleware and Resources Architecture • Job Based interactions with services • Data archived on Storage resources Previous talk has some details http://teragridforum.org/mediawiki/images/e/ed/Sudhakar.NCSA12.4.08.ppt

5. GridChem Usage

6. Scince from GridChem Gateway

7. GridChemNew Needs • New projects and Collaborations Multisclae modeling for Material Science Prof. Duane Johnson, UIUC QM-QMC Coupled applications ParamChem, Profs. MacKerrel(UMB), Roitberg(UF), Connolly(Uky), Pamidighantam(UIUC/NCSA) QM-MM-MD Coupled applications Advanced Users requiring Potential Energy Hypercurface Computations, Ab initio MD, and Parameter Sweeps. Workflows

8. GridChem Refactoring • Code refactoring is the process of changing a computer program's source code without modifying its external functional behavior in order to improve some of the nonfunctional attributes of the software. Advantages include improved code readability and reduced complexity to improve the maintainability of the source code, as well as a more expressive internal architecture or object model to improve extensibility. • “By continuously improving the design of code, we make it easier and easier to work with. This is in sharp contrast to what typically happens: little refactoring and a great deal of attention paid to expediently adding new features. If you get into the hygienic habit of refactoring continuously, you'll find that it is easier to extend and maintain code.”—- Joshua Kerievsky, Refactoring to Patterns [1] http://en.wikipedia.org/wiki/Code_refactoring

9. Client New Features • New client has no server dependencies. • The client library containing the new service stubs which need axis2 dependencies. • Axis2 handles fast communication with the service and streaming data. • DTO (Data Transfer Object) classes now implemented as the service beans that hold information sent from the service. • They’re a little less redundant. • For example in the in the new software bean, the client no longer has to parse through abstract data formats to figure out the multitude of application/hpc resource combinations.

11. GridChem Client Nanocad 3D

12. GridChem RefactoringPreprocessing Tools GDIS MolGen Tubegen JMol

13. GridChem-Xbaya Composer

14. Server New Features • Axis2 implementation makes it also available via HTTP, SOAP, and RPC. • The database underwent a cleanup and schema changes. This leads to performancein terms of quicker services.

15. Server New Features • Single service distribution into Tomcat 5.5. • Uses JNDI and Tomcat’s connection pooling for much better stability. JNDI provides standardised naming for Directory Services • Removed GAT, now uses Java cogkit based GridFTP client directly for performance increase on remote directory listings, file transfers, and i/o operations.

16. Server New Features • Multiple file upload support with service-side user cache. -Several application and workflows require multiple files for execution and • Users can upload input files, browse previously uploaded files, and retrieve previously uploaded files. • Job queue prediction via qbets is available on the client side. QBets provides a way to select resources automatically. • Resource monitoring plugs into the TeraGridgpir and iis instances for accurate, effort-free resource monitoring, discovery, and access GPIR and IIS provide system and service information for various resources ( HPC systems).

17. Server New Features • Job updates are done via a RESTful trigger service. The batch scripts callout to the service with a secret service key and the service updates the job status immediately. This provides State and Session preservation • Database access times on all queries has been improved to sub-second performance. • The overall service memory footprint has been cut down 2 orders of magnitude from the current production version. • Support for ingesting TeraGrid users, their profile, project, and resource information, and allowing them to use the CCG infrastructure with their current allocations has been added. This will provide these services across TeraGrid user communities.

18. Server New Features • CGI scripts are now bundled with the service and deployed into tomcat rather than an apache server. • Software access control has been implemented. The BlackList table holds a list of users who are denied access to a Software record.

19. Workflow Selection/Execution • XBaya is a graphical client program for workflow composition, monitoring, and more. • Different Web services can be invoked at different steps. • Data from intermediate steps stored in databases for future reuse. • Each step on the workflow can be monitored.

20. Integration of Xbaya and GridChem Data

21. Paramchemmiddleware Requirements • Broad Goals Parameterization User (community) management Paramteterization process management Workflows Data management Archival and Retrieval requirements

22. Cyberenvironments for ParameterizationComputational Reference Data Generation

23. Job Manager • Paramater definitions • Optimization Monitor • Model/Reference Data Definition • Optimization • Incomplete? • Optimization Job Completed? • Merit Function Specification • Paramater testing Model • Expert Interface • Optimization Methods Choice • Successful Testing • Paramater Sensitivity Analysis • Consistency Checker • Optmization Job Launcher • Update Parameter Database with new set • Notification of End of Workflow Workflow For Empirical Parameter Optimization Molecular Force Field CyberenvironmentsParameter Initialization and optimization Workflow

24. Parameterization Menus and Data structures

25. Charmm-Gaussian Workflow in Xbaya-workflow management system

26. Client Business Model DTO DTO (Data Transfer Object) Serialize transfer through XML DAO (Data Access Object) How to get the DB objects hb.xml (Hibernate Data Map) describes obj/column data mapping DAO WS Resources Objects Hibernate hb.xml Database ParamChem Web ServicesClient Objects  Database Interaction

27. Users ParamProjects Resources UserProjectResource Users Resources SoftwareResources NetworkResources ComputeResources StorageResources ParamChem Data Models Workflows userID paramprojectID resourceID loginName SUsLocalUserUsed WFID WFName userID projID RegWFID cost WF Node/Job JobID JobName userID projID softID cost DataResources Resources resoruceID Type hostName IPAddress siteID

28. ParamChem Resourcefollowing CCG Class Dependencies

29. ParamChem Middleware Services(PMS) Use Cases • Authentication • Workflow Selection/Creation • Workflow Configuration • Workflow Submission • Workflow Resource Monitoring • Workflow Monitoring • Data Resource Monitoring/Organization • Workflow Results Retrieval/Organization • …

30. Contact PMS Creates Session, Session RP and EPR, Ind.User.Comm Sends EPR ( Like a Cookie, but more than that) Login Request (username:passwd) Validates, Loads UserProjects,Data,WFRegistries Sends acknowledgement Retrieve UserProjects (GetResourceProperty Port Type [PT]) PMS Authentication(follows GridChem Middleware Services) http://www.gridchem.org:8668/space/GMS/usecase ParamChemClient PMS • WSDL (Web Service Definition Language) is a language for describing how to interface with XML-based services. It describes network services as a pair of endpoints operating on messages with either document-oriented or procedure-oriented information. • The service interface is called the port type • WSDL FILE: <?xml version="1.0" encoding="UTF-8"?> <definitionsname="MathService"targetNamespace="http://www.globus.org/namespaces/examples/core/MathService_instance" xmlns="http://schemas.xmlsoap.org/wsdl/" …

31. Selects projectLoadVO port type(w. MAC address) Verifies user/project/MACaddr Load UserResources RP Sends acknowledgement Validates, Loads UserProjects,Data.WFs Sends acknowledgement Retrieve UserResources[as userVO/ Profile] (GetResourceProperty port Type PT) PMS Authenticationfollows GMS_WS http://www.gridchem.org:8668/space/GMS/usecase ParamChemClient PMS

32. Create WF object PredictWFStartTimePT + WF DTO ----> Node/Job DTOs Node/JobStartPrediction RP If decision OK, Submit Workflow PT + WFDTO and JobDTOs Create WF objectAPI—Submit Store WF Object Submission Xbaya GFAC CoGKit GAT “gsi-ssh” Send Acknowledgement Completion: Email from batch system to PMS server cron@PMSDB PMS Workflow Submission ParamChemClient PMS PT = portType RP = Resource Properties DTO = Data Transfer Object Need to check to make sure allocation-time is available The workflow is sane and executable.

33. ParamChemClient PMS Resources/Kits/DB Request for Job, Resource Status Alloc. Balance UserResource RP Updated from DB Send info ParamChem Middleware Services Monitoring cron@PMSserver Xbaya Monitoring Server cron@HPC Servers Job Launcher Notifications VO Admin email parses email  DB (status + cost) Workflow Status Updates (automated) Node/Job Status Display Parse XML, Display Discover Applications (Software Resources) Discover Data (Data Resources) Monitor Workflow Schedulers (Primary) Monitor System (Secondary) Monitor Queues (Tertiary) PT = portType RP = Resource Properties DTO = Data Transfer Object DB = Data Base

34. ParamChemClient PMS Resources/Kits/DB PMS Workflow Status Workflow Status WFDTO.statusWFXBaya Launcher Status Update Estimate Start time Scheduler emails/ notifications Notifications: Client, Management, email, IM

35. Retrieve Root Dir. Listing on MSS with CoGKit or GAT or “gsi-ssh” MSS query UserFiles RP + FileDTO object LoadFile PT GetResourceProperty PT FileDTO(?) LoadFile PT (project folder+job) Validates project folder owned by user. Send new listing Retrieve file: CoGKit or GAT or “gsi-ssh” API file request Store locally Create FileDTO Load into UserData RP RetrieveFiles PT (+file rel.path) PMS DATA Organization Retrieval (MSS) ParamChemClient PMS Resources/Kits/DB Job Completion, Workflow Completion Send Output toMSS GetResourceProperty PT PT = portType RP = Resource Properties DTO = Data Transfer Object MSS = Mass Storage System

36. RetrieveJobOutput PT(+JobDTO) Job Record fromDB. Running: from Resource Complete: from MSS Retrieve file: CoGKit or GAT or “gsiftp” ParamChem File Retrieval ParamChemClient PMS Resources/Kits/DB Create FileDTO (?) Load into UserData RP GetResourceProperty PT PT = portType RP = Resource Properties DTO = Data Transfer Object MSS = Mass Storage System

37. Logging Configuration Where to find Globus Where to get random seed for encryption key generation Classpath (required jars) • WSRF (Web Services Resource Framework) Compliant • WSRF Specifications: • WS-ResourceProperties (WSRF-RP) • WS-ResourceLifetime (WSRF-RL) • WS-ServiceGroup (WSRF-SG) • WS-BaseFaults (WSRF-BF) • %ps -aux | grepws • /usr/java/jdk1.5.0_05/bin/java \ • -Dlog4j.configuration=container-log4j.properties \ • -DGLOBUS_LOCATION=/usr/local/globus \ • -Djava.endorsed.dirs=/usr/local/globus/endorsed \ • -DGLOBUS_HOSTNAME=derrick.tacc.utexas.edu \ • -DGLOBUS_TCP_PORT_RANGE=62500,64500 \ • -Djava.security.egd=/dev/urandom \ • -classpath /usr/local/globus/lib/bootstrap.jar: • /usr/local/globus/lib/cog-url.jar: • /usr/local/globus/lib/axis-url.jar • org.globus.bootstrap.Bootstraporg.globus.wsrf.container.ServiceContainer -nosec ParamChem Web Services

38. pms Classes for WSRF service implementation (PT) GMS_WS client Cmd line tests to mimic client requests dao Data Access Obj – queries DB via persistent classes (hibernate) Data Transfer Obj – (job,File,Hardware,Software,User) XML dto How to handle errors (exceptions) exceptions CCG Service business mode (how to interact) model Contains user’s credentials for job sub. file browsing,… credential “Oversees correct” handling of user data (get/putfile). file file.task Define Job & util & enumerations (SubmitTask, KillTask,…) job job.task Autonomous notification via email, IM, textmesg. notification CCGResource&Util, Synched by GPIR, abstract classes NetworkRes., ComputeRes., SoftwareRes., StorageRes., VisualizationRes. resource user User (has attributes – Preference/Address) DB operations (CRUD), OR Maps, pool mgmt,DB session, persistence audit Classes that communicate with other web services query synch gpir Periodically update DB with GPIR info (GPIR calls) test JUnit service test (gms.properties): authen. VO retrieval, Res.Query,Synch, Job Mgmt, File Mgmt, Notification Contains utility and singleton classes for the service. util crypt Encryption of login password Mapping from GMS_WS enumeration classes DB enumerators gat GAT util classes: GATContext & GAT Preferences generation proxy Classes deal with CoGKit configuration.