1 / 27

LBNL VACET Activities Hank Childs Computer Systems Engineer - Visualization Group August 24, 2009

LBNL VACET Activities Hank Childs Computer Systems Engineer - Visualization Group August 24, 2009. Outline Overview: VACET & VisIt Deployment & Outreach: math infrastructure Research: QDV & parallel streamlines Preparing for the Future: hero runs & VisIt infrastructure.

chill
Download Presentation

LBNL VACET Activities Hank Childs Computer Systems Engineer - Visualization Group August 24, 2009

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. LBNL VACET Activities Hank ChildsComputer Systems Engineer - Visualization Group August 24, 2009

  2. Outline • Overview: VACET & VisIt • Deployment & Outreach: math infrastructure • Research: QDV & parallel streamlines • Preparing for the Future: hero runs & VisIt infrastructure Director’s Review of CRD | August 24 - 26, 2009

  3. Outline • Overview: VACET & VisIt • Deployment & Outreach: math infrastructure • Research: QDV & parallel streamlines • Preparing for the Future: hero runs & VisIt infrastructure Director’s Review of CRD | August 24 - 26, 2009

  4. VACET Mission and Vision • Mission: enable scientific insight for petascale data. • Strategy: Leverage sci-vis and analytics software technology as an enabling technology. • Vision: adapt, extend, create, and deploy data understanding technologies for science stakeholders • Why are we a center? • As a center, well positioned to respond to diverse needs/objectives through coordinated R&D, software engineering, outreach efforts. Director’s Review of CRD | August 24 - 26, 2009

  5. VACET Organization • Teams: stakeholder projects, R&D projects, software engineering projects. • Executive committee: cross-institutional, cross-team coordination. PIs Bethel/LBNL, Johnson/Utah Executive Committee Bethel/LBNL, Brugger/LLNL, Johnson/Utah, Joy/UCD, Ahern/ORNL + Pascucci/Utah, Childs/LBNL Chief Software Engineer, Childs/LBNL Stakeholder Projects R&D Teams Software Engineering Teams (per site) Director’s Review of CRD | August 24 - 26, 2009

  6. VisIt is a richly featured, turnkey application for large data. 27B element Rayleigh-Taylor Instability (MIRANDA, BG/L) • VisIt is an open source, end user visualization and analysis tool for simulated and experimental data • Used by: physicists, engineers, code developers, vis experts • >100K downloads on web • R&D 100 award in 2005 • Used “heavily to exclusively” on 8 of world’s top 12 supercomputers Director’s Review of CRD | August 24 - 26, 2009

  7. VisIt has a rich feature set that can impact many science areas. • Meshes: rectilinear, curvilinear, unstructured, point, AMR • Data: scalar, vector, tensor, material, species • Dimension: 1D, 2D, 3D, time varying • Rendering (~15): pseudocolor, volume rendering, hedgehogs, glyphs, mesh lines, etc… • Data manipulation (~40): slicing, contouring, clipping, thresholding, restrict to box, reflect, project, revolve, … • File formats (~85) • Derived quantities: >100 interoperable building blocks • Many general features: position lights, make movie, etc • Queries (~50): ways to pull out quantitative information, debugging, comparative analysis Director’s Review of CRD | August 24 - 26, 2009

  8. More developers Entering repo all the time Institutional support leverages effort from many labs Partnership with CEA is developed Project started UC Davis & UUtah research done in VisIt repo VACET is funded Developers from LLNL, LBL, & ORNL Start dev in repo AWE enters repo LLNL user community transitioned to VisIt GNEP funds LLNL to support GNEP codes at Argonne 2005 R&D100 SciDAC Outreach Center enables Public SW repo Saudi Aramco funds LLNL to support VisIt User community grows, including AWE & ASC Alliance schools 2003 2007 2007 2004-6 ‘07-’08 ‘07-’08 2000 2005 Summer‘07 Spring ‘08 Spring ‘07 2008 VisIt and VACET provide significant mutual leverage. • VisIt represented over 50 person-years of effort at the time of VACET being funded • VisIt contains over one million lines of code • Eric Brugger, NNSA VisIt lead: “VACET has assumed the lead role in ensuring that VisIt will be able to handle the largest data sets.” Fall ‘06 Spring ‘09 Director’s Review of CRD | August 24 - 26, 2009

  9. Outline • Overview: VACET & VisIt • Deployment & Outreach: math infrastructure • Research: QDV & parallel streamlines • Preparing for the Future: hero runs & VisIt infrastructure Director’s Review of CRD | August 24 - 26, 2009

  10. We have made an impact for APDEC • PI: Phil Colella (LBNL), SciDAC Applied Partial Differential Equations Center • Accomplishment(s) • Software engineering to “bring product to market”. Performance improvements, interface enhancements, file readers, visual data exploration techniques. • Science Impact • Visualization on large AMR data sets, and using parallel platforms. • Trail blazed for future VACET work. Director’s Review of CRD | August 24 - 26, 2009

  11. VACET has improved VisIt’s AMR capabilities significantly. • Improvements: • Optimized many, many routines, resulting in end-to-end speedups of >10X • Added critical debugging functionality, such as spreadsheets and the ability to connect with a debugger. • Mapped AMR grids • Macros and python callbacks for customizability. • Responded to over 30 requests for interface changes and bug fixes. Director’s Review of CRD | August 24 - 26, 2009

  12. + see our review document for 9 more letters of support from SciDAC-funded groups using VisIt VACET’s efforts have led to VisIt being adopted by important ASCR stakeholders. Director’s Review of CRD | August 24 - 26, 2009

  13. Outline • Overview: VACET & VisIt. • Deployment & Outreach: math infrastructure. • Research: QDV & parallel streamlines • Preparing for the Future: hero runs & VisIt infrastructure Director’s Review of CRD | August 24 - 26, 2009

  14. QDV and Accelerator Modeling • Accomplishment: • Algorithms + deployment in VisIt to perform interactive visual data analysis (identify, track, analyze beam particles) in multi-TB simulation data. • Science Impact: • Replace serial process that took hours with one that takes seconds. • New capability: rapid data exploration and analysis. • SC08 paper • Customer: C. Geddes (LBNL)/COMPASS • Collaborators: SciDAC SDM Center (FastBit) Tech-X (Accelerator scientists) Director’s Review of CRD | August 24 - 26, 2009

  15. Deployment effort in VisIt by SEG Research effort: Efficient parallel streamline generation Streamlines represent an activity spanning research to SWE to data insight. Stakeholders: “VisIt is bad at streamlines” “We need parallel streamlines” / “Wow! Parallel streamlines is hard” Director’s Review of CRD | August 24 - 26, 2009

  16. Level 0 only (incorrect) Correctly traversing all AMR levels 217 pin reactor cooling simulation. Run on ¼ of Argonne BG/P. Research vehicle (and delivery vehicle) on streamline generation of AMR data sets. This work has had a broad and varied impact. Poincare analysis for fusion community Used in analysis by SHOCKS center (image courtesy SHOCKS website) Streamlines used in visualization of type II supernova collapse

  17. “First light” picture showing mixing of cold and warm air inlets. (Nek5000/Fischer, Argonne, INCITE run) This work has had a broad and varied impact. Director’s Review of CRD | August 24 - 26, 2009

  18. Existing parallel integral curve techniques have suffered from load imbalance. • Two extremes: • Partition data over processors and pass particles amongst processors • Parallel inefficiency! • Partition seed points over processors and process necessary data for advection • Redundant I/O! P0 P4 P1 P3 P2 P2 P0 P4 P1 P3 P1 P2 P0 P4 P3 P2 P1 P3 P0 P4 P1 P4 P3 P2 P0 P0 P1 P3 P2 P4 Notional streamline example Director’s Review of CRD | August 24 - 26, 2009

  19. We have greatly advanced the state of the art for parallel integral curve techniques. • Hybrid solution: • Master-slave approach that adapts between parallel inefficiencies and redundant I/O P0 P0 0: Read • Decision of when to pass or read is simplified here and based on load of processors owning data. • Heuristic weights against I/O, but allows for redundant I/O • - All coordination done by slave masters. P1 1: Pass 0: Read P2 P1 P3 P2 1: Read P4 1: Pass Notional streamline example Director’s Review of CRD | August 24 - 26, 2009

  20. Outline • Overview: VACET & VisIt • Deployment & Outreach: math infrastructure • Research: QDV & parallel streamlines • Preparing for the Future: hero runs & VisIt infrastructure Director’s Review of CRD | August 24 - 26, 2009

  21. Trillion-cell experiment: objectives • Research questions: • Is it possible/feasible to run production-quality visual data analysis s/w on large machines and on large data sets? • What obstacles/bottlenecks do we encounter at this level of scale? • Why? • VACET’s job: provide production-quality tools for large data. • This is demonstrating that these tools are viable. Director’s Review of CRD | August 24 - 26, 2009

  22. Trillion cell experiment methodology: What tools, what machines, what data? • Application: VisIt 1.11.1 • VisIt is the first and only visualization application that is part of the “Joule” metric • Machines: Cray (franklin/LBL, jaguar/ORNL), BG/P (dawn/LLNL), AIX (purple/LLNL), Linux (Juno/LLNL), Sun Linux (ranger/UTexas) • Data: • 5123 sample data, astrophysics simulation (courtesy T. Mezzacappa, B. Messer, S. Bruenn, R. Budjiara) • Upsampled to target problem size • Data sizes ranged to 1T (for 16K cores) to 4T (for 64K cores), with 0.5T/8K for AIX/Purple Director’s Review of CRD | August 24 - 26, 2009

  23. Trillion cell experiment methodology:What techniques? • Two common visualization techniques: • Isocontouring, volume rendering 2T zones, 32K procs on Franklin 2T zones, 32K procs on Jaguar Director’s Review of CRD | August 24 - 26, 2009

  24. Trillion cell experiment “speed bumps” • Application bottleneck: many-to-one status update • Workaround for study, fix to appear in future release of VisIt • Application optimization: an NxN buffer for communication, which works well at small scale, proved problematic at large scale. • Workaround: remove optimization, evaluate best course of action for future VisIt release. • System issues • Loading shared libraries takes a long time, as much as 5 minutes. Potential fix: implement VisIt plugins as static libraries. Director’s Review of CRD | August 24 - 26, 2009

  25. We are here Trillion cell experiment: Significance of Results 1B 1T 1M ~1980 ~2000 ~2020 • Production tool (VisIt) runs effectively at high concurrency, on large problem sizes, and on diverse platforms. • Test of visual data analysis infrastructure • Infrastructure is the underpinning of many different algorithms, not just isocontouring or volume rendering. • Minor fixes to become part of production VisIt release (later this year). • Successfully processed “tomorrow’s data today.” Director’s Review of CRD | August 24 - 26, 2009

  26. VACET has contributed more than just algorithms and research to VisIt. • Production software: bug tracking, regression testing, regular releases • Through our efforts, VisIt now feels much more like an open source project: • Transitioned software repository to NERSC; now accessible to a large community. • ~30 developers with write access from 10 institutions • Public mailing lists, archived and searchable. • ~300 subscribers, get ~300 posts per month • Wiki pages on usage and development. Director’s Review of CRD | August 24 - 26, 2009

  27. Summary • The LBNL portion of VACET has made a huge impact in the ASCR community • In deployment, in research, and in paving the way for the future • VACET is partnering with many researchers in the ASCR community. • VisIt, the production visualization tool for VACET, is being developed by a vibrant community and has demonstrated applicability at the petascale. • LBNL is a major center of gravity for the VisIt project. Director’s Review of CRD | August 24 - 26, 2009

More Related