SpaceGRID and EGSO

1. SpaceGRID and EGSO Satu Keski-Jaskari Maria Vappula Parallal Computing – Seminar 01.12.2003

2. Index • SpaceGRID • overview • project • infrastructure • involved domains • in the future • EGSO • overviwe • workpackages • system architecture • roles • data model

3. SpaceGRID • ESA funded project (general study programme) • The project aims: • How GRID technology can serve requirements across a large variety of space disciplines • Sketch the design of an ESA-wide GRID infrastructure • Foster collaboration and enable shared efforts across space applications. • Proof of concept through prototyping • Keep Europe up with GRID efforts!

4. SpaceGRID - timetable • Kick off: September 26, 2001 • User Requirements and applications to be prototyped selected: April 4, 2002 • Infrastructure analysed and specified: July 4, 2003 • ESA Grid Infrastructure set-up for demonstrations: March, 2003 • Prototypes evaluated/accepted: March 20 – June 20, 2003 • Final Workshop: May 21-22, 2003 • SpaceGRID study final report: 19.09.2003

5. SpaceGRID - Goals • Assess how GRID technology can serve requirements across a large variety of space disciplines • Foster collaboration and enable shared efforts across space applications • Sketch the design of an ESA-wide (and common) infrastructure • Proof of concept through prototyping • Involve both industry and research centres • Keep Europe up with GRID efforts!!

6. SpaceGRID Project SpaceGRID Project Project Management Requirements ESA Analysis Prototyping Future Analysis Earth Observ. Earth Observ. Earth Observ. Earth Observ. Space Research Space Research Space Research Solar System R. Solar System R. Solar System R. Mechanical Eng. Mechanical Eng.

7. Study Team Earth Observation DATAMAT Spacecraft Plasma Interactions QinetiQ Space Weather CS-SI Radiation Transport QinetiQ Solar System Research RAL Mechanical Engineering Alcatel Space Study Teams (involved domains of SpaceGRID)

8. Earth Observation

9. Radiation Transport

10. Spacecraft Plasma Interactions

11. Solar System Research

12. Mechanical Engineering

13. Space Weather Simulation

14. Space Weather Simulation Scenario

15. EGSO (European Grid of Solar Observations) • EGSO is a Grid test-bed • EGSO is funded by the Europea Commission • The role of the EGSO is to provide the infrastructure, based on concepts such as grid sharing of resources and peer to peer networking

16. EGSO - Timetable • Project was started in March 2002 • A first, live demonstration of basic querying capabilities was given early in October 2003 • Implementation of major system components is under way with incremental releases that are planned for December 2003 and February 2004

17. EGSO - Goals • Construct a virtual observatory • The main products are a common metadata standard for solar physics and a common set of FITS keywords for future missions • Enable more efficient identification and utilization of solar data from a broad range of sources.

18. Why EGSO is made • Distributed data sources • Combining multiple data sources is often necessary • Large Data Volumes • Current heliosesimological datasets already contain tens of terabytes • Data Mining • Locating the combination of data from multiple instruments most suitable for addressing a given problem is increasingly problematic

19. EGSO - Workpackages • Six workpackages • WP0 • WP1 • WP2 • WP3 • WP4 • WP5 • All workpackages involve multiple project institutions

20. Workpackage 0 • responsible for overall project management, communication and dissemination. • monitors, assesses and co-ordinates the activities of the other workpackages

21. Workpackage 1 • is responsible for the definition of community and scientific requirements for EGSO, and the development of the system architecture for EGSO • plans and facilitates the integration of the outputs of other rorkpackages, ensuring interoperability

22. Workpackake 2 • produce a working version of EGSO, initially taking the form of a 'trial federation' of some of solar data archives held by EGSO partner institutions

23. Workpackage 3 • responsible for producing a set of advanced and intuitive search and visualisation tools • create a series of customisable interfaces tailored to the needs of different users accessing.

24. Workpackage 4 • create a unified catalogue of all solar observations, providing a single point of access to data from dozens of solar instruments

25. Workpackage 5 • responsible for the design of advanced image recognition techniques to identify individual features in the solar atmosphere

26. EGSO – System architecture

27. Roles • The system architecture for EGSO makes use of three high level abstractions (roles) to describe and partition system functionality • Provider • Broker • Consumer • EGSO has adopted a generic architecture for all roles that is based on the popular "n-tier" style

28. Provider • is the source of all that is handled by EGSO to satisfy user request • is the crucial interface between existing data archives and the rest of the EGSO system • may offer one or more of the following types of resources • Metadata, obtained descriptions of observations • Data, the actual data files • Services, is there being any capabilities

29. Broker • inserts itself as an intermediary between the provider and the consumer • will maintain a list of available providers, a resource registry, based on information submitted by the providers themselves, as well as information shared from others Brokers. • will also maintain a catalog, at course granularity, containing a summary of all the observation catalogs that have been incorporated into EGSO

30. Consumer • provides the means for a user of the system to actually visualize, select and manipulate the metadata and data obtained from multiple resources • will be able to store the results provided by the Broker, both in memory or, for a longer term persistence, in a disk cache managed by the Consumer • will also offer the possibility to use or define workflows, sequences of operations to be performed upon appropriate datasets of interest.

31. Data model • A Data Model has been designed so that incorporates all the relevant entities in the system • Framework will include various types of metadata that describe the objects to be included in the EGSO system. • Observational Metadata describes the contents of the stored datasets • Administrative Metadata describes the organizations and resources known to EGSO and the means of accessing them. • Derived Metadata describes additional information extracted from observational data.