Problem-Solving Environments:

1. Problem-Solving Environments: The Next Level in Software Integration David W. Walker Cardiff University

2. Objectives of this talk • To review the purpose and scope of PSEs • To discuss the requirements for constructing PSEs • To identify the major technologies that will serve as the infrastructure of PSEs • To review the PSEs currently in development and in use David W. Walker, Cardiff University

3. Why PSEs? • Need: enhanced scientific insight; reduced development costs; improved product quality and industrial efficiency. • Need: transparentmeans of integrating distributed computers, instruments, sensors, and people. • Need: improved software productivity to extract maximum benefit from advances in computers, networks, and algorithms. David W. Walker, Cardiff University

4. Why Now? • Confluence of complementary technologies. • Faster networks and communications. • Network software technologies such as CORBA, Java, and XML. • “Big Science” is inherently distributed and collaborative, and needs to migrate to WAN environments to progress. David W. Walker, Cardiff University

5. What’s the Problem? • High-level, problem-specification languages, often coupled with expert system. For example, PDE solvers, numerical integration, etc. • Problem composition in form of dataflow graph using a GUI. Typically used in modelling and simulation of physical systems. David W. Walker, Cardiff University

6. PSE Requirements • Expert assistance in problem specification and input. • Transparent access to distributed heterogeneous resources. • Interactivity and computational steering. • Advanced/immersive visualisation. • Integration with other knowledge repositories and databases. David W. Walker, Cardiff University

7. Technologies for PSEs Hardware: • Increasingly powerful computers • Increasingly fast networks - gigabit ethernet, vBNS, etc. • Immersive visualisation platforms - CAVEs, ImmersaDesks, etc. David W. Walker, Cardiff University

8. Technologies for PSEs Software: • CORBA for transparent interaction between distributed resources. • Java for platform-independent programming. • XML interface specification. • MPI for message-passing in SPMD codes. David W. Walker, Cardiff University

9. An Example PSE Architecture Main PSE sub-systems are: • Visual Program Composition Environment (VPCE) for graphically composing applications. • Intelligent Resource Management System (IRMS) for scheduling applications on distributed resources. David W. Walker, Cardiff University

10. VPCE Overview • GUI is used to build an application from software components - either a java or CORBA object with its interface specified in XML. • Each component may have a performance model and help file. • An annotated dataflow graph is produced that is passed to the IRMS. David W. Walker, Cardiff University

11. IRMS Overview • IRMS locates software and hardware resources through information servers. • IRMS then schedules components on appropriate resources based on performance models and database of experience from previous runs. Genetic and neural network algorithms may be used. David W. Walker, Cardiff University

12. The PSE Research Community • European Research Conference on PSEs took place June 1999 in Spain. Next one in summer 2001. http://www.cs.cf.ac.uk/euresco99/ • EuroTools SIG on PSEs. http://www.irisa.fr/EuroTools/Sigs/ • Cardiff PSE project web site. http://www.cs.cf.ac.uk/PSEweb/ David W. Walker, Cardiff University

13. US Software Infrastructure The Grid is a computational and network infrastructure providing pervasive, uniform, and reliable access to distributed resources. • Globus: provides core services for grid-enabled computing. http://www.globus.org/ • Legion: an object-based metacomputing project. http://legion.virginia.edu/ David W. Walker, Cardiff University

14. European Software Infrastructure • UNICORE: Uniform access to Computing Resources. Aimed at providing uniform, secure, batch access to distributed resources. http://www.genias.de/unicore/unicore.html • POLDER: a more ambitiousmetacomputing project. http://www.wins.uva.nl/projects/polder/ David W. Walker, Cardiff University

15. European Software Infrastructure • CODINE: resource management system targeted at optimal use of all software and hardware resources in a heterogeneous networked environment. http://www.genias.de/products/codine/ • CCS: Computing Centre Software - resource management for networked high-performance computers. http://www.uni-paderborn.de/pc2/projects/ccs/ David W. Walker, Cardiff University

16. European Software Infrastructure • GRD: Global Resource Directorfordistributed environmentsfeaturing policy management and dynamic scheduling. http://www.genias.de/products/grd/ • NWIRE: Netwide resources - management system for WAN-based resources. http://www-ds.e-technik.uni-dortmund.de/ David W. Walker, Cardiff University

17. COVISE Visualisation Environment • The Collaborative Visualisation and Simulation Environment is a distributed software environment that seamlessly integrates simulations, post-processing, and visualisation. • COVISE supports collaborative working, and is available commercially. http://www.hlrs.de/structure/organisation/vis/covise/ David W. Walker, Cardiff University

18. Ctadel and PDE Problems • Code-generation tool for applications based on differential equations using high-level language specifications is an environment for the automatic generation of efficient Fortran or HPF programs for PDE-based problems. • Used in HIRLAM numerical weather forecast system. http://www.wi.leidenuniv.nl/CS/HPC/ctadel.html David W. Walker, Cardiff University

19. An Environment for Cellular Automata • CAMEL is a CA environment designed for message-passing parallel computers. It hides parallelism issues from a user. • User specifies only the transition function of a single cell of the system with CARPET, a high-level cellular language. http://isi-cnr.deis.unical.it:1080/~talia/CA.html David W. Walker, Cardiff University

20. A PSE for Numerical General Relativity • CACTUS is a collaborative software environment for composing applications for the solution of general relativity problems. • Has been used in distributed computing experiments using Globus. • Interactive visualisation important. http://cactus.aei-potsdam.mpg.de David W. Walker, Cardiff University

21. JACO3: Industrial Design PSE • Java and CORBA based collaborative environment for coupled simulations. • A CORBA based high performance distributed computing environment for coupling simulation codes. • Optimaldesign of complex and expensive products like airplanes, satellites, or cars. http://www.arttic.com/projects/jaco3/ David W. Walker, Cardiff University

22. A PSE for Stochastic Analysis • Promenvir:Probabilistic mechanical design environment - a metacomputing tool for stochastic analysis. • It can automatically generate a series of stochastic computational experiments, and run them on the available resources • It has been used for optimal design problems in the automobile industry. http://www.cepba.upc.es/promenvir.html David W. Walker, Cardiff University

23. PSE for Engineering Simulations • JULIUS: Joint IndustrialInterface for End-User Simulations. • Integrated HPC environment for multi-disciplinary engineering simulations. • Aimed at reducing design time for industrial products. • End-users are engineers. http://www.6s.org/ David W. Walker, Cardiff University

24. Summary • There is an active body of PSE researchers and developers in Europe. • PSEs are used in science, engineering, finance, and manufacturing. • Current emphasis is on PSE infrastructure and prototypes. David W. Walker, Cardiff University

25. Future Challenges • Maintaining good, reliable performance in distributed environments important. • Need to integrate third party software. • Need visualisation environments that scale from PC up to immersive systems. • Needs standards for interfaces and interaction between PSEs. David W. Walker, Cardiff University