A Service Agreement Framework for Grid based Distributed Rendering

1. John O’Brien Supervisor: Roy Kalawsky Department of Electronic and Electrical Engineering A Service Agreement Framework for Grid based Distributed Rendering

2. Outline • Why Render Agreements • Managing Render Resources • Service Level Objectives • WS-Agreement primer • The Render Agreement Framework • Automatic Agreement generation • Browser based XML GUI’s for Visualisation

3. Why Render Agreements? • Emerging grid visualisation technology • Moved from homogeneous clusters to variable heterogeneous environments • Examples include RAVE, GVK, StreamAR • Users and providers must be able to form service level agreements about rendering taking place on these systems: • Frame rate, response times, compression methods, level-of-detail. • Different application domains have differing requirements: medical vs. fly-through models

4. Managing Heterogeneous render resources • Allocating a suitable set of resources for distributed rendering is a challenging problem. • Visualisation community equally quick to exploit new extensions so taking snapshot of API is not viable • OpenGL is not pixel exact: Vendors can target performance over quality, and are quick to develop new extensions.

5. Service level objectives • Service agreements are only effective when renegotiation is minimised. • Predominantly requires some form of human interaction - Slow • Renegotiation may enact penalty clauses on provider • Correct service level objectives key to minimising renegotiation • Service level objectives are often a compromise: • Service provider (responder) is better able to offer objectives in resource terms: memory, cpus, bandwidth • Objectives of an initiator are task orientated: completion time, frame rate • Uncertainty in predicting resource objectives from task objectives is risk! Either for provider or initiator

6. Rendering Service Level Objectives • Benchmarking • Good results when visualisation application is fixed • Approach taken by viewperf standard • RAVE uses benchmarking to assign resources • Difficult to map API based benchmarks to application performance • Our solution is to use tiered objectives in the form of agreement templates: • resource orientated templates for unknown applications • Task orientate guaranteed templates for known applications

7. A Standards based approach • Emerging and existing web service standards are available to help develop service management frameworks: • WS-Agreement – provides a protocol for establishing (render) agreements • JSDL – An extensible job submission description language –Used to describe the render resource requirements of a visualisation. • WS-Inspection – Provides a simple mechanism for tracking agreement providers

8. The WS-Agreement Specification <wsag:Agreement> <wsag:Name>Sample1</wsag:Name> <wsag:Context>…..</wsag:Context> <wsag:Terms> <wsag:ServiceDescriptionTerm> … <wsag:ServiceDescriptionTerm> <wsag:ServiceReference>…<wsag:ServiceReference> <wsag:ServiceProperties>…</wsag:ServiceProperties> <wsag:GuaranteeTerm> …</wsag:GuaranteeTerm> </wsag:Terms> </wsag:Agreement>

9. Render Resource Schema • A new resource schema is needed to manage render resources: <jsdl:JobDescription> <jsdl:Resources> <jsdl-render:Api> OpenGL_1_2 </jsdl-render:Api> <jsdl-render:Visual> <jsdl-render:Attribute> doublebuffer </jsdl-render:Attribute> </jsdl-render:Visual> </jsdl:Resources> </jsdl:JobDescription> <wsag:ServiceDescriptionTerm wsag:Name="API" wsag:ServiceName="RenderJob1"> <jsdl-render:RenderResource> <jsdl-render:API> OpenGL_1_4 </jsdl-render:API> </jsdl-render:RenderResource> </wsag:ServiceDescriptionTerm> Agreement Document Job Definition Document

10. The Agreement Protocol • Agreements are unilateral made by the service provider (responder). • Agreement Templates are exposed as a WS-ResourceProperty to help initiator create an agreement document

11. Agreement Templates • Agreement Templates are exactly the same as an agreement document with additional creation constraints: <wsag:CreationConstraints> <wsag:Item wsag:Name="CompressionMethod"> <wsag:Location> //wsag:ServiceDescriptionTerm[@wsag:Name="CompressionMethod"]/jsdl-render:RenderResource/jsdl-render:CompressionMethod </wsag:Location> <wsag:ItemConstraint> <xsd:enumeration value="jpeg"/> <xsd:enumeration value="jpegls"/> <xsd:enumeration value="jpeg2k"/> <xsd:enumeration value="adaptive"/> </wsag:ItemConstraint> </wsag:Item> </wsag:CreationConstraints> Agreement Template Name Context Terms CreationConstraints

12. Agreement Monitoring • WS-Agreement defines an AgreementState service type for exposing the current state of an agreement through WS-ResourceProperties: • AgreementState – Observed / Terminating / Complete • ServiceTermStateList – NotReady, Ready (idle / processing) • GuaranteeTermStateList – Fulfilled / Violated

13. WSAG::Lite • Light weight generic implementation of WS-Agreement for WSRF::Lite • Provides an Agreement Factory for creating Agreement services from an agreement offer. • Offers are validated against the set of templates stored by an Agreement responder. • Automatically updates agreement state through WS-Notification Consumer port

14. The Render Agreement Framework WSRF::Lite with WSAG::Lite (Perl) Tomcat Servlets + Globus WSRF for Java

15. Render Pools • A Render Pool represents a collection of hardware rendering resources or pipes. • Each resource has identical graphics hardware, OS and driver implementation / config. • Guaranteed to render exactly the same on each resource in the pool. • Collecting render services into homogeneous pools is a performance optimisation inspired by Condor-G

16. The Ageement Process • A Render Pool is an Agreement Responder and as such exposes a set of Agreement Templates • Render Brokers match Job description with appropriate templates

17. Agreement Template Hierarchy • Templates are given a hierarchy on the Render Pool: • New Job templates are generated automatically when a generic agreement completes Generic Template Job ID 3 Template Job ID 1 Template Job ID 2 Template

18. Automatic Agreement Generation • Generated from agreement state information Terminating Processing New Agreements contain new Guarantee values. Also characteristics of a job description. Render Service Render Service Render Pool Notify Notify Set Resource Properties Allocation The scope of a guarantee is limited to monitored application characteristics. Allocation Get Resource Properties Client Create new template

19. User orientated agreement creation • A simple 4 step process for agreement creation is exposed to the user

20. Providing a cross platform client • Goal: Minimise installation effort for end user whilst also being specific enough to closely match native application functionality • XML User interface language (XUL) provides this flexibility • Mozilla’s plugin support allows a small platform specific core to handle visualisation.

21. XUL cross-platform client

22. DicomViz Agreement status Updates StreamARMozilla plugin Business value of quality / speed trade off

23. Questions