Use of Java in Computational Science

1. Use of Java in Computational Science presented by Tomasz HauptNortheast Parallel Architectures Centerat Syracuse University

2. Part I Web Portals

3. Web Portals • A portal is a web entrance to a set of resources and consists of a mix of information, computer simulations and various services • For businesses portals generalize the concept of a a company Intranet and encompass domain of IBM main frames, Lotus Notes etc. • For computing, portals are called Problem Solving Environments

4. Examples of Portals • Portal to NPAC is http://www.npac.syr.edu • Portal to the world is http://www.yahoo.com/ or http://my.netscape.com/ • Portal to latest news is http://www.cnn.com • Portal to computational chemistry is http://www.osc.edu/~kenf/theGateway/ • Portal to stock trading is http://quote.yahoo.com/

5. Access to: • Search Engines • News • Weather • Stocks • Sport • Services • Bookmarks • Mail • Calendar • ... Example Portal: Netscape

6. More Portals

7. Portalsarecustomizable

8. Special Portals -- Computing • But perhaps more interestingly computing portals involve building a web based problem solving environment to link together all the capabilities needed to compute • run programs and access dynamically status of jobs and computers -- in particular allow a uniform interface to running a given job on one of many backend compute servers • compile and debug programs • link diverse data sources with computations run on multiple backend machines • visualize results • web-based help systems and collections of related scientific papers • computational steering i.e. interacting with a job (change parameters) based on dynamic results such as visualized results • See http://www.osc.edu/~kenf/theGateway/ andhttp://www-fp.mcs.anl.gov/~gregor/datorr/

9. Portals for scientific and engineering communities • Seamless access to HPC resources • Seamless access to instruments • Data storage • Application specific data base • Visualization Tools • Collaboratory • Scientific notepads

10. Front-End Desktop/Laptop FRONT-END: high-level user friendly - visual programming and authoring tools - application GUI Seamless Access RESOURCES: all hardware and software components needed to complete the user task, including, but not limited to, compute engines from workstations to supercomputers, storage, databases, instruments, codes, libraries, and licenses. Remote Resources

11. Seamless Access • Create an illusion that all resources needed to complete the user tasks are available locally. • In particular, an authorized user can allocate the resources she needs without explicit login to the host controlling the resources. • An analogy: NSF mounted disk or a network printer.

12. Gatekeeper Gatekeeper Gatekeeper Example: Globus Advantages: - platform independent mini-language (RSL) for specification of resources - can be layered on top of different schedulers- enables interoperability between resources (can allocate many resources at a time, file transfer, monitoring, etc.)Disadvantage: - a bag of low level tools GRAMClient MDS Directory Service Contact addressResource Language Specification GSS-API

13. Towards a complete solution ... PSE: problem description (physics, chemistry, ...) Task description: I need 64 nodes of SP-2 at Argonne to run my MPI-based executable “a.out” you can find in “/tmp/users/haupt” onmarylin.npac.syr.edu. In addition, I need any idle workstationwith jdk1.1 installed. Make sure that the output of my a.out is transferred to that workstation Middle-Tier: map the user’s task description onto the resource specification; this may include resource discovery, and other services Resource Specification Resource Allocation: run, transfer data, run

14. Front-End Front-End Abstract Task Specification Middle-Tier Resource Specification We need a third tier! Remote Resources

15. Abstract Task Specification Middle-Tier Resource Specification Target Architecture Problem Solving Environment CTA specificknowledgedatabases Visual Authoring Tools User and Group Profiles ResourceIdentificationand Access Visualizations Collaboration WebFlow Back-End Resources

16. Navigate and choose an existing applicationto solve the problem at hand.Import all necessary data. Example of a portal Select host Select model Set parameters Run Retrieve data Pre/post-processing Run simulations

17. PSE Example: CCM IPSE 1 .Define your problem 2. Identify resources (software and hardware) 3. Create input file 4. Run your application 5. Analyze results Ken Flurchick, http://www.osc.edu/~kenf/Gateway

18. Data-Flow Front-End QS Front End Compose interactively your application from pre-existing modules

19. Building a Portal • We can identify a set of tools that enable the construction of portals • These are roughly equivalent to the tools needed to build a general application based on “object web technologies” • There is also an architecture implying multi-tier systems with standard compliant interfaces • A common portal architecture means that portals can be conveniently linked together • e.g. the 3 portals to biology, chemistry, and physics naturally form portal to science

20. Portal Building Blocks I • So we currently have languages (Java ..) distributed object support and standards (CORBA …), interface standards (XML), transport protocols (HTTP, TCP/IP) at various levels, rendering standard (HTML). • We have web clients and servers • We need certain capabilities in common to many portals. These include • security • collaboration • visualization (for computing portals) • persistence, registration, look-up (part of most distributed object infrastructure)

21. Portal Building Blocks -- Security • So in this course, we will discuss security as it is a common service needed by many portals • It can be implemented simply as a user name / password but there are several special features • Encryption -- keeping information secret • Authentication -- identifying and authorizing individuals to access particular capabilities • Different technical approaches -- especially Kerberos and Public Key Infrastructure • And a discussion of special difficulties as seen by spate of stories about viruses, hackers and security leaks of computer information from government facilities

22. Portal Building Blocks -- Distributed Object Support • Although CORBA for instance provides (by definition) most key distributed object services such as persistence, this is not sufficient as we will inevitably mix object models and further these services are still being developed • So as one part of this course we will discuss “discuss distributed object” and “internet” (software) infrastructure • with special attention to issues of naming, registering, looking up (yellow pages) and addressing objects • Remember a web page is most common object and every Java program is an object • We need to contrast classical hierarchy of naming as in DNS and web URL’s as implemented in LDAP with much more intriguing dynamic model in Sun’s Jini and UCB’s Ninja which are suitable for mobile ephemeral objects

23. File System orDatabase holding Web Pages Web Server Basic 3 Tier Computing Model • A serveraccepts input and produces output • A Web Server accepts HTTP request and returns a web page • a Database Server accepts a SQL request and returns records selected from database • An Object Broker accepts IIOP requests to invoke methods of an “object” (e.g. run a program) • IIOP and HTTPare two common protocols (formats of control data) for inter program messages • A Web browser (Netscape or Microsoft) can access any server at “the click of a button” with data from user refining action

24. PressRunButton Object View of running a program Fortran Program is an Important Type of Object It can be built up from smaller objects e.g. Matrix library could be an object • Similar to invoking a web page • “CORBA” or “WIDL” (pure XMLCGI specification) is just CGIdone right …... Convert GenericRun Request intoSpecific Requeston Chosen Computer FortranSimulation Codeon Sequential or Parallel Machine Object Broker

25. Pragmatic Object Web Technology Model - I • Basic Vision: The current incoherent but highly creative Web will merge with distributed object technology in a multi-tier client-server-service architecture with Java based combined Web-ORB’s • Need to abstract entities (Web Pages, database entries, simulations) and services as objects with methods (interfaces) • CORBA .. XML is “just” CGI done right • COM(Microsoft) and CORBA(world) are competing cross platform and language object technologies • Every Netscape4 browser has a Visigenic ORB built in • Javabeans plus RMI and JINI is 100% pure Java distributed object technology • W3C says you should use XML which defines a better IDL and perhaps an object model -- certainly does for documents • How do we do this while technology is still changing rapidly!

26. Relational Database Object Store Multi-Tier Client Server Service Back-end Tier Services Middle Tier Servers Client Tier Object Broker IIOP HTTP Web Server Specialized Java Server RMI(IIOP)or Custom Old and New Useful Backend Systems Javabean Enterprise Javabean

27. Pragmatic Object Web Technology Model - II • Need to use mix of approaches -- choosing what is good and what will last • For example develop Web-based databases with Java objects using standard JDBC (Java Database Connectivity) interfaces • Oracle DB2 Informix Sybase, Lotus Notes, Object database confusion becomes an issue of performance/robustness NOT functionality • Use XML to record small databases in flat files • Use CORBA to wrap existing applications • Use COM to access components of major PC applications such as Microsoft Excel and Word • Use Jini and Java to implement dynamic registration of objects • Use HTML to render everything

28. Functionality of layers Database MPP Telescope File System 1)Rendering of (Multiple)Objects2)Proxy to some backend capability used to render input and output to and from service 1)Server acts as a broker and control layer 2)Same software as client but higher performance multi-user 3)Again service represented as proxy used as a token for control logic Services with specialized software and capabilities

29. Proxy -- Proxy -- Backend Capability XML XML Real Capability • The Proxies and actual instantiation are linked by messages whose semantic content is defined (best) in XML • The lower system level format can be HTTP RMI IIOP or … • The client proxy is for rendering input and output including specification of object • The middle tier proxy allows choice of backend provider and functional integration (the user can specify integration at client proxy level)

30. RenderingEngine Broker or Server XML Result XMLQuery XML Requestfor service followed byreturn of XMLresult HTML Browser RenderingEngine Universal Interfaces IDL or Templates Objects Basic Multi Tier architecture • Objects (at “logical backend”) can be on client of course • Front end can define a generic (proxy for a) object. The middle control tier brokers a particular instantiation

31. Emerging Object Web Multi-Server Model Back End Servers and their services Clients andtheir servers Middle Tier Custom Servers

32. MultidisciplinaryControl (WebFlow) Gateway Control Parallel DBProxy Database NEOS ControlOptimization OptimizationService Origin 2000Proxy MPP NetSolveLinear Alg.Server Matrix Solver Agent-basedChoice ofCompute Engine IBM SP2Proxy Data AnalysisServer MPP Multi-Server Web Computing System or Portal to Computing

33. Multi-UserMiddleware User SpecificServer Palm Top Some caveats and comments • Need version 5 browsers with good XML support to properly implement this • We draw three tier as minimum architecture but as previous diagram suggests, one is likely to build a more structured system with middle tier having more layers • Network computer breaks client tier into two with simple HTML at user and Java servlets providing heavier weight capability on a different machine • Here user is at a WebTV or Palm Pilot or similar low-end device

34. What does it take to Implement This • Well you need some hardware -- that is either an Internet or Intranet • Internet is the world running object web software • Intranet is a dedicated network (for a company, university department, PC cluster) running object web software • You need some software • You need some standards and capabilities expressed in these standards • You need some capabilities common to many applications • You need a specific system to solve a particular problem

35. More details on the implementation • Note the hardware can be as little as 1 PC on your desk • More interestingly it is your 64 PC Linux or Windows NT Cluster up to the cluster of 64 128 node SGI Origin’s at Los Alamos • i.e. a parallel computer is by definition a special case of an Intranet • Software divides into several types HTMLRendering Fortran Program PLSQL Database or ….. “Glue” with (multiple) tier servers and XML inter tier communication Style Sheetsand Page Design Java/CORBA/WIDLWrapper

36. Implementation Continued I • The backend software can be parallel or sequential and simulation or information based • It can be COBOL, Fortran, C++, Perl ... • We need to define in XML its interface needed to • run it • set its parameters -- i.e. its input mechanisms • get its output -- numbers or visualization • This backend program interface is defined as an XML file e.g.<program name=“physicssimulation1”> <run domain=“npac” machine=“maryland” type=“pc” os=“nt” exec=c:\myprogs\a.out</run> <input type=“htmlform” > <name>userinput</name> <field default=“10” >iterations</field> ………. </input> <output> …</output></program> Becomes HTML form with name userinput andtext field iterations with default value 10 on client

37. Implementation Continued II • For this example (running a physics program), we could use a specific machine as defined on previous foil (the Windows NT PC maryland) or a generic machine<run domain=“any” machine=“any” type=“pc” os=“nt” > • In this case, middle tier broker, would look in a database (XML file) to find what machines were available and either automatically or with user input choose one. • Both Software and Hardware are defined in XML • Note databases and an XML files are logically equivalent • JDBC can access either Oracle (Microsoft) database or XML ASCII file • More generally XML can be thought of as general object serialization • A database table is one type of object

38. Implementation Continued III • The front end is some document consisting of a mix of HTML or XML • The HTML is whatever you want to make a nice page • The XML is converted into some variant of HTML by • Browser default or • XSL style sheet • User Program -- logically in middle tier • Note HTML can include Java applets either directly or invoked from XSL style sheets • We will NOT discuss either how to code backend in PLSQL or Fortran or how to compose final rendered document in HTML

39. Collaboration I • This is often termed groupware support and Lotus Notes is best known corporate product • Collaboration implies sharing of electronic objects and is needed in asynchronous and synchronous modes • AOL Yahoo etc. have Internet games which illustrate one sophisticated form of collaboration • Chat rooms are perhaps most popular and are simplest synchronous tool. White boards next most popular • Asynchronous mode is • shared web pages and documents (these are be shared synchronously or asynchronously) • electronic mail, event calendars, bulletin boards • http://www.npac.syr.edu/tango/ is a collaboration system supporting synchronous sharing of events where events signify changes in objects

40. Collaboration II • Notification and linkage service can be based on object registration mechanism and allows important collaborative capabilities • one associates with each group activity a “magic ID” (barcode) • every digital object associated with this activity registers itself when it comes on line with some registry (registry can be distributed). A given object may have multiple barcodes attached to it • This is Jini model for registration but can be implement for pure Web (using JavaScript) or CORBA • Either users or Portals (PSE’s) register interest in certain barcodes • The (Portal) event service notifies registered observers when a digital object of interest becomes available

41. Collaboration III • Notification mechanism enhances collaboration as enables dynamic federation of relevant objects to be automatically maintained • Kodak would like this service to group together digital versions of photographs taken at particular events (e.g. a wedding) • Notification can be used for people so their presence on-line can be made known to those in particular collaborative users • Users decide if notification causes an active signal (send electronic mail, ring a buzzer) or passively alters a list on a web page. • Event Model unifies synchronous and asynchronous models of collaboration • Event either triggers action immediately and/or asynchronously ( sending e-mail immediately is synchronous act generating asynchronous record)

42. Collaboration and Portals I • Shared Objects need to accept data from Portal compliant applications • Portal Events need to be integrated into SPDL • Portal federates different “event domains” Collaboration ==Sharing Event in “Tango” Server PortalEvents Local Event/Message Bus Local Event/Message Bus Local Event/Message Bus

43. Collaboration and Portal II • Whiteboard and Shared Browser can be loaded with files and data from Portal compliant systems • More generally consider any client side rendering of a gateway system -- either data input or (visualization/data) output • These can be shared collaboratively • Examples from Tango and Computing Portal • Shared Biology Workbench shares client side input forms • Shared visualization (NCSA, NPAC) shares output file of a computation

44. Part II WebFlow

45. Object B (event target) Object A (event source) Method M(){ …} Fire event E WebFlow design • Object Oriented, follows JavaBeans model • everything is an object • objects interact through events Firing event E by object A causes invocation of method M of object B.The association of event E and method M is achieved by an eventregistration mechanism. An event is also an object and it carries data.

46. A few words about CORBA (a digression) more information on Java, Corba, Distributed Object: http://www.npac.syr.edu/projects/cps616spring96/index.html

47. Distributed objects • Typically WebFlow objects live in different address spaces. We use CORBA to invoke methods of the remote objects. Object B (event target) Object A (event source) Method M(){ …} Fire event E ORB

48. How is this possible? Object B (event target) Object A (event source) Method M(){ …} Fire event E IIOP ORB2 ORB1 Object Adapter serves also as a daemon -Objects A and B are CORBA objects (thus not Java objects) - Objects are defined in IDL (Interface Definition Language) - IDL definitions are compiled using (Java)IDL compiler - The IDL compiler generates new classes to be used by the Java compiler (javac) instead of the original ones, on both the client and server side - The IDL compiler generates either classes to be extended, or interfaces to be implemented

49. Example of IDL definition #include “..\BC.idl”module WebFlow { module lms{ interface runEdys:BeanContextChild { void run(); void receiveData(); void setParameter(in string p); }; interface runCasc2d:BeanContextChild{ void run(); void runAgain(); }; interface DoneEvent{ Object getSource(); }; };}; We will create 3 CORBA objects * two modules: - runEdys - runCasc2d * one event - DoneEventThey will be added to packageWebFlow.lms

50. We need more flexibility... • WebFlow objects are developed independently of each other(reusable modules): we cannot assume that the event source knows anything about the event target and vice versa