240 likes | 379 Views
AstroGrid’s Common Execution Architecture. Guy Rixon, reporting on behalf of Paul Harrison and the other AstroGrid developers SC4DEVO-1, Pasadena, July 2004. AstroGrid. UK national virtual-observatory project See cover slide for participants and funding
E N D
AstroGrid’s Common Execution Architecture Guy Rixon, reporting on behalf of Paul Harrison and the other AstroGrid developers SC4DEVO-1, Pasadena, July 2004
AstroGrid • UK national virtual-observatory project • See cover slide for participants and funding • Tasked to produce a VO toolkit to be reused by service providers • Collective author of a SOA using web services • Common Execution Architecture: • A major part of AstroGrid architecture • Part of the service-oriented architecture • A way of making and using web services • Some important applications to data mining AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 2
SOA for data mining How can we carve up the distributed application into services? And where do we put the application logic? AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 3
Distributed app: conservative Desktop Application Application logic in here Data-centre 2 Data-centre 1 Data transform Data source AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 4
Distributed app: browser-based Desktop Browser Application logic in here Data-centre 2 Data-centre 1 Data transform Data source AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 5
Distributed app: intragrid Desktop Some application logic in here; Rest still in client or in user Browser/app Node Data source Data transform Node Node Node Data-centre 2 Data-centre 1 Compute grid AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 6
Portal Distributed app: scripted App logic in here Desktop Script Browser/app Workflow engine Data source Data transform Data transform Data centre 3 Data-centre 1 Data-centre 2 AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 7
Portal Distributed app: embedded Little app logic here Desktop Script Browser Workflow engine Most app logic here Data source Data transform Application Data mine Data-centre 1 Data-centre 2 AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 8
Portal Distributed app: data grid Desktop Script Browser/app Workflow engine Data source Data transform Data transform Data centre 3 Data-centre 1 Data-centre 2 AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 9
Boring, technical: How to write apps as w/s? How to write w/s clients? How to connect apps to storage? How to register functions? Interesting, scientific: Who has useful algorithms? How to match algorithms to data? Best place to run computations? How to share code? Provenance? How to share kudos? Is more science possible? Issues to be addressed AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 10
AstroGrid architecture and products • AstroGrid bits and pieces let techies solve the tech problems so that scientists can concentrate on the science issues. • AstroGrid supports: • Browser-based app • Scripted app • Data grid • Embedded app (pre-installed exes) • Common Execution Architecture lets all parts but exe for embedded app be reusable code. AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 11
Portal Desktop Script Browser/app Workflow engine Data source Data transform Data transform Data centre 3 Data-centre 1 Data-centre 2 Common Execution Architecture (1) “CEC” = Common Execution Connector Web service (CEC) MySpace CEC configuration Application (local exe) Data mine AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 12
Portal Desktop Script Browser/app Workflow engine Data source Data transform Data transform Data centre 3 Data-centre 1 Data-centre 2 Common Execution Architecture (2) Scripts stored here, as files MySpace Page presentation Workflow definition Workflow executor (JES) Portal AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 13
Common execution architecture (3) Common language throughout system allows s/w reuse. AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 14
How to write an app using CEA? • Don’t write a web service! • Write a command-line exe • Configure a CEC to run your app: • I.e. describe parameters in the CEA language • Or: get your app configured on someone else’s CEC AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 15
How to publish an app using CEA? • Take the <ApplicationDefinition> that informs the CEC. • Wrap it in a <VOResource> of sub-type CeaApplicationType. • Publish the <VOResource> in IVO registry. AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 16
How to call an app-service using CEA? • Look it up in the resource registry. • Get the <ApplicationDefinition> from its <VOResource>. • Give the <ApplicationDefinition> to a CEA delegate => define SOAP call. • Add parameters from UI according to <ApplicationDefinition>. • Bake in pre-heated SOAP engine @ HTTP 1.1. AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 17
CEA language example <ApplicationDefinition> <Parameters> <ParameterDefinition name="DetectionImage" type="binary"> <UI_Name>Detection Image</UI_Name> <UI_Description>The image that is used to detect sources. Basic position, shape and size information is derived from this image</UI_Description> </ParameterDefinition> <ParameterDefinition name="PhotoImage" type="binary"> <UI_Name>Measurement Image<UI_Name> <UI_Description>The Image that is used to measure photometric parameters</UI_Description> </ParameterDefinition> <ParameterDefinition name="config_file" type="text"> … AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 18
CEA language schema (1) AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 19
CEA language schema (2) AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 20
JES CEA Delegate Other Delegate PAL CEC HyperZ CEC SIAP CEC CEA facilities so far Other service MySpace AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 21
Future evolution • Add facilities to CEC: • Asynchronicity • Security • More parameter types • New facility: field-programmable CEC (PCEC) AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 22
PCEC (1): local code libraries Client Select exe PCEC Load exe Code library Data mine AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 23
PCEC (2): distributed code-library VOSpace Client Code library Upload exe Select exe PCEC Download exe Sandbox Run exe Data mine AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 24