1 / 11

HPCC Heap Management

HPCC Heap Management. Sheldon Brown, Site Director Erik Hill, Programmer Analyst Kittinan Ponkaew , Student. June 12 2013 UMBC. Background. Growth Tracker Tool Detects growing m emory p roblems Available Memory Analysis Tools Do not detect these problems

kenton
Download Presentation

HPCC Heap Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. HPCC Heap Management Sheldon Brown, Site Director Erik Hill, Programmer Analyst KittinanPonkaew, Student June 12 2013 UMBC

  2. Background • Growth Tracker Tool • Detects growing memory problems • Available Memory Analysis Tools • Do not detect these problems • New terminology for problem • Memory Tumor

  3. Current “Leak” Terminology • In research • Unreachable vs. Reachable Leak – Hound • Physical vs. Logical Leak – Maebe et al. • Lost Object vs. Useless Object Leak – Sleigh • Forgotten vs. Lost Memory Leak – LeakPoint • Industry & dev community • Java heap vs. native leak – IBM • Java vs. C/C++ type leak – IBM • True leak, traditional leak - stackoverflow

  4. Our Terminology Leak Tumor

  5. Results Scalable City

  6. Publication • Paper published • IEEE International Conference on Software Testing, Verification and Validation (ICST 2013)

  7. New Project • High Performance Computing Cluster • C++ tool set

  8. HPCC tool set to be analyzed Courtesy of LexisNexis

  9. Courtesy of LexisNexis

  10. ECLOptimized C++ generated code for HPCC 1. ECL incorporates transparent and implicit data parallelism regardless of the size of the computing cluster and reduces the complexity of parallel programming increasing the productivity of application developers. 2. ECL enables implementation of data-intensive applications with huge volumes of data previously thought to be intractable or infeasible. ECL was specifically designed for manipulation of data and query processing. Orders of magnitude performance increases over other approaches are possible. 3. ECL provides a more than 20 times productivity improvement for programmers over languages such as Java and C++. The ECL compiler generates highly optimized C++ for execution. 4. ECL is a powerful, high-level, parallel programming language ideal for implementation of ETL, information retrieval, information extraction, record linking and entity resolution, and many other data-intensive applications. 5. ECL is a mature and proven language but still evolving as new advancements in parallel processing and data-intensive computing occur. ECL also provides a comprehensive IDE and programming tools that provide a highly-interactive environment for rapid development and implementation of ECL applications. Courtesy of LexisNexis

  11. Work plan • Implement Thor and/or Roxie HPCC system • Determine if Heap Management tool is in pre-ECL or ECL generated code – ideally on both! • Run jobs of various sizes to determine tumor code robustness

More Related