Elettra Virtual Collaboratory: the evolution of a Virtual Laboratory Software from a simple web application to the GRID

1. Elettra Virtual Collaboratory: the evolution of a Virtual Laboratory Software from a simple web application to the GRIDCC Roberto Pugliese, Alessandro Busato, Alessio Curri, Enrico Mariotti, Daniele Favretto, Fulvio Billè, Roberto Borghes, Fabio Asnicar, Valentina Chenda, Laura Del Cano, Lawrence Iviani, Michele Turcinovichand the GRIDCC collaboration Sincrotrone Trieste – ELETTRA Instruments and Sensors on the Grid IEEE Conference on e-Science and Grid Computing Melbourne, Australia, on 5-8 December 2005

2. Outline • The Elettra Virtual Collaboratory (EVC) • EVC @ Work • The BIOXHIT project • Virtual Collaborative System (VCS) • The EUROTeV project • The Global Accelerator Network • The Multipurpose Virtual Laboratory (MVL) • Evolving EVC to meet VCS and MVL requirements • The GRIDCC project • The Multipurpose Collaborative Environment (MCE) • Evolving EVC with MCE

3. What is a Collaboratory? • The term “collaboratory” was coined by William Wulf by merging the words collaboration and laboratory, and defined as “... Center without walls, in which researchers can perform their research without regard to geographical location - interacting with colleagues, accessing instrumentation, sharing data and computational resource, and accessing information in digital libraries”.

4. What is a Collaboratory? • In particular, the core capabilities that constitute a collaboratory can be seen as technologies to link: • People to people (e.g., electronic mail, and tools for data conferencing, such as VRVS) • People to information (e.g., the World Wide Web and digital libraries) • People to facilities (e.g., status of remote instruments) to enhance utilization by expanding access to resources

5. What is the Elettra Virtual Collaboratory (EVC)? • EVC is an example of virtual laboratory, a system which allows a team of researchers distributed anywhere in the world to perform a complete experiment on the equipped beamlines and experimental stations of Elettra. User atELETTRA Remote Collaborator Team Member at Home Lab COLLABORATORY Equipment Control CPU Data

6. What is Elettra?

7. EVC usage scenarios:Cristallography “by mail” • EVC allows biologists to send by mail protein crystals which will be ananlized at the Xray Diffraction beamline by the beamline staff. • Collected data and resultsare accessible via EVC andresults can be downloadedas soon they are available.

8. EVC in action: a web portal • EVC is based onthe “web portal”metaphor • All you need is abrowser • EVC supports four different usercategories: • Visitors • Normal users • Project leaders • Staff

9. Collaborating to an EVC project • Scientists workingto an EVC projectcan use manyproject relatedcollaborationtools • EVC presents anadaptive interfacechanging to suitethe categoryand expertiselevel of the user

10. Collaboration Tools: EVC chat • EVC chat is“project centered”:there is a differentchannel for eachproject • Usual chat featureare extended inorder to allowexchange of • Drawings • scientific images • graphical annotations

11. Collaboration tools:scientific visualisation • Scientists canbrowse, visualiseand process remotelyscientific datain real-timeas soon asthe data is collected

12. Collaboration Tools: telepresence • The different videostreams of the equipped experimental stationscan be selected and viewed eventhrough a slowconnection • Movable camerascan be controlledvia web by theproject leader

13. Collaboration tools remote computing • Legacy software isnormally not web enabled • EVC uses VNCto web enablelegacy apps. • It is small andsimple, sharableand open • Can be tunnelledvia ssh • VNC can be used asa fast integration tool

14. Collaboration Tools: Remote Beamline Control and Supervision • Beamwatchpresents asynoptic viewof the beamlines • Autorised peoplecan thus operateremotely onthe beamlineintrumentation

15. EVC Architecture application server DB • EVC has 2 main components: • the application server and • a set of nodes • The application server is running the portal application, the user and project database; the application server activates actions implemented by agents running in the nodes or requests services to external systems nodek nodek-1 node1 node2

16. EVC facts • EVC project started on June 2001 and finished on June 2003. The first prototype was installed on the Xray Diffraction beamline of ELETTRA on June 2002 • EVC is now operating on all the beamline and experimental station of ELETTRA and acts as the web interface to the Elettra Scientific Computing Environment (instruments, computing farms, storage) • EVC was presented at SMAU2002, NOBUGS2002 and SMAU2003, HCI2003, NOBUGS2004 • EVC development staff is partecipating in manyEU founded projects under FP6(BIOXHIT, IA-SFS/JRA1, GRIDCC, EUROTeV/GAN)

17. BIOXHIT Virtual Collaboratory System • The BIOXHIT project which will develop an integrated platform for high-throughput structure determination • ELETTRA is developing the Virtual Collaboratory System a Virtual Organization (VO) connecting all the European laboratories doing research in the field of structural genomics.

18. BIOXHIT VCS scenario VCS will be used to implement a widley distributed Virtual Organisation (VO) connecting all the stations and Laboratories involved in the BIOXHIT project Processing Farm VO user beamline 1 Processing Farm Supporting Services beamline 2 Data Storage crystallisation beamline 3 VO user

19. VCS Node @ EMBL-Hamburg

20. EUROTeV/GANMVL • In the EUROTeV project the design study of the International Linear Collider ELETTRA is developing the Multipurpose Virtual Laboratory, the core tool to implement the Global Accelerator Network, a VO connecting all the international laboratories doing research in the field of Accelerators. • Remote control of an accelerator facility has the potential of revolutionizing the mode of operation and the degree of exploitation of large experimental physics facilities. • The first prototype of the system planned by April 2005 will allow the remote control of ELETTRA storage ring from DESY.

21. MVL @ Work

22. MVL @ Work

23. www.lightsources.org

24. VCS/MVL Virtual Organisation

25. Evolving EVC to implement VCS / MVL • In this architecture every institute in the VO should have a VCS Node. The node can support more stations (e.g. a data collection station, a cristallisation, a control room station, a movable station etc). Stations can share resources and tools. • The remote collaborator will use his PC equipped with a web browser and if the case with a projector. • All the communication (AS-to-AS and AS-to-LN) is done via webservices secured with X.509 certificates by mutual authentication • Legacy applications are integrated using VNC if they do not have a web interface or ssh tunnels and proxy or redirection if they already have a web interface.

26. Evolving EVC to implement VCS / MVL • All the communication between the Application Servers located in the distributed laboratories is done via webservices (Axis implementation). • All the Local Nodes run a Local Node Server. Comunication between the AS and the LN is done via webservices (gSOAP implementation). • The systems are also equipped by a Management Station which allows easy configuration and maintenance via web browser. • VCS and can be considered a sort of integration platform. Scripts stored in the AS database are transferred to the local nodes, executed and the results returned to the user via the AS.

27. The GridCC Project + Instruments Grid Computational Grid

28. GRIDCC project in 3 steps • Development of generic Grid middleware, based on existing building blocks (Grid Services) which will allow the remote control and monitoring instrumentation such as distributed systems. • Testing of the middleware on challenging applications to validate it both in terms of functionality and quality of service: • European Power Grid • Geo-hazards • Remote Operation of an Accelerator Facility • High Energy Physics Experiment • … • Dissemination of the new software technology to encourage a wide range of enterprises to evaluate and adopt our Grid-oriented approach to real-time control and monitoring of remote instrumentation.

29. The MCE and the GRIDCC landscape The MCE is a software to implement Virtual Control Rooms, i.e., multi-user, collaboration-supporting interfaces to a widely distributed control system with access to grid-enabled computing and data storage facilities Virtual Ctrl. Room Instrument 1 Computing Element Supporting Services Instrument 2 Diagnostics Storage Element Instrument 3 Virtual Ctrl. Room

30. Instrument Element Instrument Element Information Monitor Service Security Service AutS Instrument Element TGS PolR Computing Element Computing Element Computing Element Virtual Control Room Virtual Control Room Exec. Service WfMS WMS AgrS Global Problem Solver GridCC Architecture Collaborative Service Storage Elements Storage Elements Storage Element

31. The Multipurpose Collaborative Environment • a groupware providing general purpose services and interfaces to support collaboration amongresearchers and operators, to control remote instrumentation, and other tasks related with experimental activities • will be used to implement the VCR for the different GRIDCC pilot applications through customization and integration with application-specific services. • based on a core groupware application (providing, e.g., authentication, management of the VO users and instruments) and a set of plug-ins: • General purpose (e.g., chat, notebook, video conference) • Specific to the particular application (e.g., accelerator control, specific instrument control)

32. Design and Development Approach • State of the art evaluation • Collaboration support tools for scientific experimental activities • Available technologies • Derivation of general requirements • through use cases, interviews, exchanges with other related projects (e.g., EUROTeV) • Prototyping • Discussions over interface sketches • Incremental development of functional prototypes

33. Design Choices • Web-based portal interface • no installation, porting, clients available almost anywhere • Minimal system requirements: Web browser, JRE for applets • Exploit portlet technology • Why? manageable integration of application-specific functionalities within the MCE • Current prototypes are based on a modified version of the GridSphere framework, developed under the GridLab EU project • Collaboration with the GridSphere team to extend and improve the framework to our needs was established and some of our developments are already in GridSphere codebase

34. Login page

35. Personalisation

36. eLogbook

37. VO view: people browser and chat

38. Resource browser and Instrument control

39. More Instruments Control

40. Video/Phone Conference: VRVS/Skype

41. Desktop Sharing

42. Execution Services: job submission

43. Execution Services: File Access

44. Evolving EVC with MCE • EVC project started in 2001 and has now more than 4 years of operations • EVC software has been improved in these years moving from a simple single facility web application to a multi-facility integration platform based on webservices • We are currently refactoring EVC in order to migrate to the GRIDCC MCE middleware.

45. VCR VCR SecS CollabS ExeS AutS WfMS TGS WMS IMS PS PolR AgrS VIGS IE RS ACM IE IE IE CE SE LPS IM DM IMSpx GRIDCC Architecture