1 / 31

Components for Scientific Computing

Components for Scientific Computing. Randall Bramley Computer Science, Indiana University Center for Advanced Computation Research, California Institute of Technology. CAT Team Members. Dennis Gannon: CAT architecture design Randall Bramley: CAT/LSA design

jaeger
Download Presentation

Components for Scientific Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Components for Scientific Computing Randall Bramley Computer Science, Indiana University Center for Advanced Computation Research, California Institute of Technology

  2. CAT Team Members • Dennis Gannon: CAT architecture design • Randall Bramley: CAT/LSA design • Benjamin Temko: Globus wrangler • Fabian Breg: NexusRMI author • Prafulla Deuskar: parallel HPC++ components • Shridhar Diwan: HPC++ Manager • Madhusudhan Govindaraju: HPC++ framework, GRAM server • David Stern: HPC++ framework design, LSA components • Juan Villacis: CAT design, GUI workspace, Java Manager • Andrew Whitaker: information subsystem, GUI InfoBrowser • Esra Akman, Deepa Viswanathan, Thomas Stuckey: alums

  3. Large Sparse Linear Systems • 105-107 or more unknowns • Irregular sparsity structure • Nonbanded, non-structured • Not diagonally dominant • Nonsymmetric in values • Causes convergence problems

  4. The Good News for Linear Systems • Sparse direct solvers readily available: • Umfpack, SuperLU, Y12M • HPC banded solvers in Scalapack • Preconditioned iterative solvers: • Aztec, SPLIB, BPKIT, ITPACK, ... • Matrix manipulation packages: • Sparskit, SMMS, Metis, Chaco, ...

  5. The Bad News for Linear Systems • All methods are parameterized • enormous design space to navigate. • Black arts abound • matrix reordering, scalings, filtering • No existing theory provides a practical guide to crafting a solution strategy • experimentation required.

  6. Goals of Linear System Analyzer (LSA) • Develop “solution strategies” for linear systems • High-level rapid prototyping for combining sparse matrix manipulations and solvers • Hot-wiring capabilities: grab linear systems from running applications, return solutions/strategies dynamically • High-performance capabilities: big systems, fast network protocols. • We chose a distributedcomponent architecture approach...

  7. Classes of Components in the LSA • I/O components: getting linear system data in and out of the framework • Filter components: manipulating the system to have more favorable properties for a solver • Solve components • Information components

  8. LSA I/O Components • Read from file: H/B or Matrix Market format • Write out solution vector to a file • Take input from URL on network • Take input from active object (running program) • Feed solution back to an active object • Extract linear system from framework to file

  9. LSA Filter Components • Reorder linear system: MinDeg, RCM, ND, MaxTraversal, BBD, Weighted Diag Block • Scale system: max element, row, column, relative scalings • Filter out elements based on magnitude • Parallel partitioning reorderings: Chaco, Metis

  10. LSA Solver Components • LAPACK dense solver (extracted) • LAPACK banded solver (extracted) • SuperLU sparse direct from Xiaoye Li • SPLIB preconditioned iterative solvers • Thirteen iterative methods (only two are really useful) • Seven preconditioners (four are of use) • Heavily instrumented • Aztec parallel iterative solvers • BLOCK

  11. LSA Info Components • BasicInfo: analysis of coefficient matrix (beefed-up version of Sparskit’s infofun) • VizInfo: emily sparse matrix visualization • Wannabes: • IterativeInfo: specialized analysis for selecting iterative solvers, preconditioners • DirectInfo: estimates storage required, computational rate for sparse direct solvers • Spectral: eigenvalue/singular value decomposition

  12. Component Systems Definition Characteristics of “component” Encapsulated software object providing specific functionality or service, used with other components to build complete applications Peer-to-peer interactions vs. client-server. Applications built by connecting binaries - not via compilation/linking of sources Components interact via well-defined interfaces or ports. Connected as software IC’s in a framework Services need not be computational: visualization, object store, ... Components have swept desktop, business worlds Microsoft COM, CORBA, Java Beans, Iris Explorer, Khoros, ...

  13. Distributed Component Framework Services • Component identification and handling • LDAP component information system • Interface registry • Java RMI mechanism • Connection management; exception handling • Combination of Java, HPC++, Nexus • User interface / collaboration mechanisms • Java GUIs • Hardware resource management • Globus system

  14. CAT Framework Architecture Java Composition GUI Local Java proxies to remote objects Database Visualizer Evolver Initializer Remote objects in HPC++, Java or wrapped MPI-Fortran CAT Composition GUI Globus authentication and instantiation of components Intercomponent comm with Java RMI/HPC++ remote invocation semantics implemented over Globus/Nexus

  15. Application codes Software libraries, runtime systems Basic infrastructure software Hardware: computers, visualization equipt Standard Metacomputing Model • Application codes at top, written in C, C++, Fortran, … • HPC libraries (ScalaPack), runtimes (MPI, Tulip) • Infrastructure (Globus, Nexus, network protocols) • Hardware (Origins, SP2s, CAVES, …)

  16. Applications Component Resource Composition Tools Component Resource Management Hardware Resources Software Resources CAT Metacomputing Model • Applications … not code (physical process modeling) • Composition tools for organizing resources (CAT Java GUIs, managers) • Resource (both SW and HW) location/management (Globus, LDAP DB’s) • Actual resources: code, computers, visualization engines, databases, instruments

  17. CAT User Interfaces Composition window showing comp connects Software component finder: LDAP based Detail info about selected comp

  18. Common Component Architecture Group Existing component frameworks not suitable for HPC Participants from all national labs, few universities Key goal: develop component interface standards. Allow interchanging of components between frameworks, labs Mechanisms for a component to make its interfaces known Allow for parallel links, different network protocols. Meetings: Unfunded meetings six times last year BOF meeting at SC98 (debutante party) Last meeting has implementable goal

  19. Usage of Component Architectures Composition phase: instantiate and tie together comps Discretize Init Discretize Init Run phase: start comps executing Solve DB Solve DB Visualize Mixed phase: add comps dynamically

  20. A Possible CCA Framework System (From Rob Armstrong, SNL)

  21. Basic CCA mechanism: gPorts gPorts = generalized Ports Borrows ideas from CORBA, COM, visual programming environments like Iris Explorer, AVS, Khoros Current prototype based on CORBA-like user/provider model Uses type of COM-like IUnknown pattern Defined using linked interfaces. Source-target, start pt - end pt, source-sink, output-input, event source - listener, user-provider terms all used to describe the communicating ports

  22. Basic CCA mechanism: gPorts Output ports can be multiplexed, but not input ports gPorts directly* invoke methods on each other. Allows standard data-flow model shared-memory or “data stays, components move” models * Using underlying run-time system

  23. Basic CCA mechanism: gPorts Typed Channel Source Target inputGport Event Source “uses” remote methods Event listener “provides” methods outputGport • outputGport implements • - methods defined by typed channel • - addListener() used to connect target to outputGport • Examples: • - outputGport is list of “global pointers” in HPC++ • - logic for argument marshaling and RPC • - n x m collective communications as in MPI

  24. Basic CCA mechanism: gPorts Each Gport provides information about its types via strings for name, type: Framework responsible for mapping from strings to internal representation of types class gPortInfo { public gPortInfo(String portname, String porttype); public String getType(); public String getName(); };

  25. Basic CCA mechanism: gPorts Channel types can be defined via an IDL spec: interface MyChannelType{ int row(in float, in array<int,3>); void mat_order(in int); }; Becomes in Java or in C++ Interface MyChannelType{ int row( float, array_int_3); void mat_order(int); }; class MyChannelType{ public: virtual int row(float, array_int_3)=0; virtual void mat_order(int)= 0; };

  26. Basic CCA mechanism: gPorts • Defining input port requires implementing • Channel type generated by IDL (or by hand) • interface InputGPort extends GPort{}; • Defining output port requires implementing • Channel type • Mechanisms also will allow hierarchical composition • See web pages for details, examples. interface OutputGPort extends GPort{ public void addInputListener(InputGPort ); public void removeInputListener(InputGPort ); public Enumeration getInputPorts(); };

  27. CCA and IDL CCA standard does not require using IDL. CAT/LSA allows seamless interaction between Java and HPC++ method invocations; no IDL Groups can define own “glue” mechanisms Must support C/C++, Java, Fortran77, Python, Perl Fortran 90 support allowed but not required. Reflection capabilities required for interfaces. Run-time discovery of objects by frameworks

  28. CCA Dynamic Registry • First Level: how to register components so that frameworks can locate and instantiate them • Second Level: how a component makes its interfaces known to the framework • Meta-Level: how to describe the internal functionality of a component in the global marketplace • Less of a problem in DOE applications: relatively few components and interfaces to handle. • Will be/is a major problem for academic applications

  29. Current CCA Status Scrupulous avoidance of word “standards”. draft gPort spechttp://z.ca.sandia.gov/~cca-forum/gport-spec draft IDL spechttp://www.llnl.gov/CASC/babel Multiple test frameworks being developed InDEPS (Sandia, Armstrong and Melius) CAT/LSA (IU, Gannon and Bramley) PAWS (LANL, Beckman and Reynders) Existing frameworks likely to be made CCA-compliant POOMA, PetSC, SciRun, ISIS++, ...

  30. Current CCA Participants • Sandia: Rob Armstrong, Robert Clay, Bill Mason,Carl Melius • Indiana University: Dennis Gannon • CACR, Indiana: Randall Bramley • LLNL: Andy Cleary, Scott Kohn, Brent Smolinski • LANL: Pete Beckman, Pat Fasel, Bill Humphrey, Kate Keahey • ORNL: Al Geist, Noel Nachtigal • ANL: Satish Balay, Lori Freitag, Paul Hovland, Lois Curfman McInnes, Barry Smith • Utah: Steve Parker, Chris Johnson • Lawrence Berkeley: Brent Milne • Pacific Northwest Labs: Jarek Nieplocha

  31. An Idiosyncratic View • Components are the future of scientific/eng computing • desktop, business computing always leads the way • is the right level of abstraction for object-orientation • maps naturally to collaborating groups distributed across the globe • integrates weird stuff: symbolic computing, vizualization, DBs • We have to address scalability in terms of numbers of components: 105 components, 108 interface instances • Software agents may be right tool to find, challenge/test, and connect software components

More Related