1 / 21

JSDL Introduction

JSDL Introduction. Andreas Savva, Fujitsu Laboratories Michel Drescher, Fujitsu Laboratories of Europe Stephen McGough, LeSC Session Name, JSDL-WG #1 28-30 June, 2005 (GGF14 in Chicago). GGF Intellectual Property Policy.

sylvie
Download Presentation

JSDL Introduction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. JSDL Introduction Andreas Savva, Fujitsu Laboratories Michel Drescher, Fujitsu Laboratories of Europe Stephen McGough, LeSC Session Name, JSDL-WG #1 28-30 June, 2005 (GGF14 in Chicago)

  2. GGF Intellectual Property Policy All statements related to the activities of the GGF and addressed to the GGF are subject to all provisions of Appendix B of GFD-C.1, which grants to the GGF and its participants certain licenses and rights in such statements. Such statements include verbal statements in GGF meetings, as well as written and electronic communications made at any time or place, which are addressed to any GGF working group or portion thereof, Where the GFSG knows of rights, or claimed rights, the GGF secretariat shall attempt to obtain from the claimant of such rights, a written assurance that upon approval by the GFSG of the relevant GGF document(s), any party will be able to obtain the right to implement, use and distribute the technology or works when implementing, using or distributing technology based upon the specific specification(s) under openly specified, reasonable, non-discriminatory terms. The working group or research group proposing the use of the technology with respect to which the proprietary rights are claimed may assist the GGF secretariat in this effort. The results of this procedure shall not affect advancement of document, except that the GFSG may defer approval where a delay may facilitate the obtaining of such assurances. The results will, however, be recorded by the GGF Secretariat, and made available. The GFSG may also direct that a summary of the results be included in any GFD published containing the specification.

  3. JSDL Introduction • JSDL stands for Job Submission Description Language • A language for describing the requirements of computational jobs forsubmission to Grids and other systems. • A JSDL document describes the job requirements • What, not how • JSDL does not define a submission interface or what the results of a submission look like • Or how resources are selected, or …

  4. Primary deliverables and status • A specification for an abstract standard JSDL independent of language bindings, including: • JSDL attributes; attribute relationships and ranges of attribute values. • Currently draft version 1.0 – submitted to the editor – in 15 day comment • A normative XML Schema • Currently draft version 1.0 • In reality the specification and schema are one document • Status: Heading towards public review • Current draft: https://forge.gridforum.org/projects/jsdl-wg/document/draft-ggf-jsdl-spec/en/21

  5. JSDL Document (1) • A JSDL document may contain • Generic (job) identification information • User information • Application description • Resource requirements (main focus is computational jobs) • Description of required data files • Out of scope (at least for version 1.0) • Scheduling • Workflow • Security • …

  6. JSDL Document (2) • A JSDL document is a template … • It can be submitted multiple times and can be used to create multiple job instances • … so JSDL does not define attributes to describe the state of a running job • No start time, end time, submission status, or even JobID • A JSDL document can be composed with other languages (open content model) • For example to express scheduling, security, etc, requirements in more detail

  7. Workflow Job Job Job Job JSDL JLM … JSDL JSDL JSDL JLM JLM JLM … … … RRL JPL RRL RRL RRL JPL JPL JPL WS-A SDL … WS-A WS-A WS-A SDL SDL SDL … … … JSDL: Conceptual relation with other standards RRL - Resource Requirements Language SDL – Scheduling Description Language WS-A – WS-Agreement JLM – Job Lifetime Management JPL – Job Policy Language

  8. JSDL Document Life Cycle • A JSDL document may be • Abstract • Only the minimum information necessary • For example, application name and input files • Runnable at sites that understand this level of description • Refined • More detail provided • Target site, number of CPUs, which data source • May be refined several times • Tied to a specific site/system • Incarnated (Unicore speak); or • Grounded (Globus speak) • This model is supported/allowed but not required by JSDL

  9. And Here Super Scheduler, or Broker, or … JSDL Here A Grid Information Service JSDL Here Existing DRM Job Manager WS Gateway WS Clients Local resource (e.g., Supercomputer) Another JSDL system Local Information Service And Here JSDL Usage

  10. DataStaging • JSDL does not define workflow • But … data staging is a fundamental requirement for any meaningful job submission • Assume simple model • Stage-in – Execute – Stage-Out • Files required for execution • Files are staged-in before the job can start executing • Files to preserve • Files are staged-out after the job finishes execution Stage-In Execute Stage-Out

  11. <JobDefinition> <JobDescription> <JobIdentification ... />? <Application ... />? <Resources... />* <DataStaging ... />* </JobDescription> </JobDefinition> Note: None [1..1] ? [0..1] * [0..n] + [1..n] JSDL Structure Overview

  12. Job Identification Example: <jsdl:JobIdentification> <jsdl:JobName> My Gnuplot invocation </jsdl:JobName> <jsdl:Description> Simple application … </jsdl:Description> <sos:AAId>3452325707234 </sos:AAId> </jsdl:JobIdentification> <JobIdentification> <JobName ... />? <Description ... />? <JobAnnotation ... />* <JobProject ... />* <xsd:any##other>* </JobIdentification>? Extensibility point

  13. <Application> <ApplicationName ... />? <ApplicationVersion ... />? <Description ... />? <xsd:any##other>* </Application> Application Example: <jsdl:Application> <jsdl:ApplicationName> gnuplot </jsdl:ApplicationName> <jsdl:ApplicationVersion> 5.7 </jsdl:ApplicationVersion> <jsdl:Description> Use the gnuplot application v5.7 regardless where it is installed on the target system <jsdl:Description> </jsdl:Application> How do I explicitly define applications? ==> See next slide!

  14. <POSIXApplication> <Executable ... /> <Argument ... />* <Input ... />? <Output ... />? <Error ... />? <WorkingDirectory ... />? <Environment ... />* … </POSIXApplication> POSIXApplication is a normative JSDL extension Defines standard POSIX elements stdin, stdout, stderr Working directory Command line arguments Environment variables POSIX limits (not shown here) Application: POSIXApplication extension

  15. <jsdl:Application> <jsdl:ApplicationName>gnuplot</jsdl:ApplicationName> … <jsdl:posix:POSIXApplication> <jsdl-posix:Executable> /usr/local/bin/gnuplot <jsdl-posix:Executable> <jsdl-posix:Input> input.dat </jsdl-posix:Input> <jsdl-posix:Output> output1.png </jsdl-posix:Output> <jsdl-posix:Argument> control.txt </jsdl-posix:Argument> <jsdl-posix:POSIXApplication> </jsdl:Application> Application: PosixApplication example

  16. Resource <Resources><CandidateHosts ... />?<FileSystem .../>*<ExlusiveExecution .../>?<OperatingSystem .../>?<CPUArchitecture .../>?<IndividualCPUSpeed .../>?<IndividualCPUTime .../>?<IndividualCPUCount .../>?<IndividualNetworkBandwidth .../>?<IndividualPhysicalMemory .../>?<IndividualVirtualMemory .../>?<IndividualDiskSpace .../>?<TotalCPUTime .../>?<TotalCPUCount .../>?<TotalPhysicalMemory .../>?<TotalVirtualMemory .../>?<TotalDiskSpace .../>? <TotalResourceCount .../>?<xsd:any##other>* </Resources>* Example: One CPU and at least 2 Megabytes of memory <jsdl:Resources> <jsdl:CPUCount> <Exact> 1.0 <Exact> </jsdl:CPUCount> <jsdl:PhysicalMemory> <LowerBoundedRange> 2097152.0 </LowerBoundedRange> </jsdl:PhysicalMemory> </jsdl:Resources>

  17. Example: CPUArchitecture • Basic value set defined based on CIM • sparc • powerpc • x86 • parisc • mips • ia64 • arm • … • CIM defines these as numbers; JSDL provides an XML definition • Watching WS-CIM work • Ditto for values of a number of other elements: • OperatingSystemType, …

  18. Data Attribute Example: Stage in a file (from a URL) and name it “control.txt”. In case it already exists, simply overwrite it. After the job is done, delete this file. <jsdl:DataStaging> <jsdl:FileName> control.txt </jsdl:FileName> <jsdl:Source> http://foo.bar.com/~me/control.txt </jsdl:Source> <jsdl:CreationFlag> overwrite </jsdl:CreationFlag> <jsdl:DeleteOnTermination> true </jsdl:DeleteOnTermination> </jsdl:DataStaging> <DataStaging> <FileName ... /> <FileSystemID ... />? <CreationFlag ... /> <DeleteOnTermination ... />? <Source ... />? <Target ... />? </DataStaging>*

  19. <?xml version="1.0" encoding="UTF-8"?> <jsdl:JobDefinition xmlns:jsdl=“http://schemas.ggf.org/2005/06/jsdl” xmlns:jsdl-posix=http://schemas.ggf.org/jsdl/2005/06/apps/posix xmlns:sos="http://www.someotherstandard.org/schema/jsdl"> <jsdl:JobDescription> <jsdl:JobIdentification> <jsdl:JobName>My Gnuplot invocation</jsdl:JobName> <jsdl:Description>Simple application invokation: User wants to run the application 'gnuplot' to produce a plotted graphical file based on some data shipped in from elsewhere (perhaps as part of a workflow). A front-end application will then build into an animation of spinning data. Front-end application knows URL for data file which must be stage in. Front-end application wants to stage in a control file that it specifies directly which directs gnuplot to produce the output files. In case of error, messages should be produced on stderr (also to be staged on completion) and no images are to be transferred. </jsdl:Description> <sos:AAId>3452325707234</sos:AAId> </jsdl:JobIdentification> <jsdl:Application> <jsdl:ApplicationName>gnuplot</jsdl:ApplicationName> <jsdl-posix:POSIXApplication> <jsdl-posix:Executable> /usr/local/bin/gnuplot <jsdl-posix:Executable> <jsdl-posix:Input>input.dat </jsdl-posix:Input> <jsdl-posix:Output>output1.png</jsdl-posix:Output> <jsdl-posix:Argument>control.txt</jsdl-posix:Argument> </jsdl-posix:Application> </jsdl:Application> <jsdl:Resources> <jsdl:TotalCPUCount> <Exact> 1.0 </Exact> </jsdl:TotalCPUCount> <jsdl:TotalPhysicalMemory> <LowerBoundedRange> 2097152.0 </LowerBoundedRange> </jsdl:TotalPhysicalMemory> </jsdl:Resources> <jsdl:DataStaging> <jsdl:FileName>control.txt</jsdl:FileName> <jsdl:Source>http://foo.bar.com/~me/control.txt</jsdl:Source> <jsdl:CreationFlag>jsdl:overwrite</jsdl:CreationFlag> <jsdl:DeleteOnTermination>true</jsdl:DeleteOnTermination> </jsdl:DataStaging> <jsdl:DataStaging> <jsdl:FileName>input.dat</jsdl:FileName> <jsdl:Source>http://foo.bar.com/~me/input.dat</jsdl:Source> <jsdl:CreationFlag>overwrite</jsdl:CreationFlag> <jsdl:DeleteOnTermination>true</jsdl:DeleteOnTermination> </jsdl:DataStaging> <jsdl:DataStaging> <jsdl:FileName>output1.png</jsdl:FileName> <jsdl:Target>rsync://spoolmachine/userdir</jsdl:Target> <jsdl:DeleteOnTermination>true</jsdl:DeleteOnTermination> </jsdl:DataStaging> </jsdl:JobDescription> </jsdl:JobDefinition> Complete example of JSDL document

  20. Projects adopting JSDL • Project adopting JSDL • Business Grid project • NAREGI • gridMatrix (Cadence) • GridSAM (LeSC) • UniGrids (FLE) • Marketing Services (Univ. of Manchester) • JSDL/DRMAA mapping (informal) • List is incomplete; also probably • Projects in India, Poland, etc • CA • FJ

  21. JSDL-WG Next Steps • Coming into Public comment soon • Please read and comment! • Many thanks to all involved in the JSDL group

More Related