Comparative Study of Parallel Performance Visualization Tools

1. Comparative Study of Parallel Performance Visualization Tools By J. Ramphis Castro December 4, 2002

2. Presentation Overview • Introduction • Evaluation Criteria • Aims • ParaGraph • VAmpir • Conclusions

3. Introduction • The reasons for poor performance of parallel message-passing codes can be varied and complex and users need to be able to understand and correct performance problems. • Performance tools can help by monitoring a program's execution and producing performance data that can be analyzed to locate and understand areas of poor performance.

4. Introduction (cont) • There are many performance visualization tools available to the parallel programming community. The purpose of this review is to analyze the distinctive features that separate them from each other.

5. Evaluation Criteria • There must always be some guidelines with which to separate and gauge the value of a given piece of software. Therefore, we used numerous review standards in order to accurately gauge the individual software’s value.

6. Usability • Intuitive easy-to-use interface • Adequate documentation and support • On-line help and man pages

7. Portability • Amount of platforms it can be used on • Ease of building for a new host • MPI implementations and languages it could handle

8. Versatility • Ability to analyze performance data in different ways and to display performance information using different views • Ability for new analysis methods and views to be added easily

9. Scalability • Ability to handle large numbers of processes • Ability large or long-running programs • Acceptable overhead increases

10. Maturity • Handles error conditions without crashing • Few or no penalties by incurring in mistakes while using the program • Level of support • Size of user base

11. Automated Instrumentation and Monitoring System (AIMS) • AIMS is a software toolkit for measurement and analysis of Fortran 77 and C message-passing programs written using the NX, PVM, or MPI communication libraries.

12. Usability • Software distribution included readme files and man pages • User guide available from main web page • User interface was functional, not aesthetically pleasing

15. Portability • Platforms (HP, SGI, IBM, and SUN) • University of Tennessee is porting AIMS to DoD MSRC • An ASCI effort at LANL is porting AIMS to ASCI • Data files are in ASCII and thus portable • AIMS supporting HPF is being developed jointly with the Portland Group • Developers of AIMS are working on a port for the SGI Origin 2000

16. Versatility • Instrumentation done automatically (xinstrument) • There are a few display types: VK, tally, MK, and Xisk. (provide different display choices)

17. Scalability • PC is the utility name that attempts to remove overhead • No data available on the tool’s impact on overhead

18. Maturity • From what information we could find about AIMS support and development we can conclude that it is still a relatively young program and counts on minimal developer support • There a numerous projects to port AIMS to different platforms

19. Verdict on AIMS • Although there are a few research groups trying to improve upon its availability to different platforms and its robustness, AIMS suffers from reliability issues. • Has good features for all around visualization.

20. ParaGraph • This tool was created for the purpose of providing a detailed, dynamic, graphical animation of the behavior of message-passing parallel programs and graphical summaries of their performance.

21. Usability • Most features can be accessed through “Point and Click” operations • Easy manipulation of the trace file in use • Offers intuitive appeal and aesthetic quality of its displays • Vast amount of documentation available from the developers

22. Portability • Based on X Windows • Is available for most unix based systems • Its availability to different platforms rivals even the most accomplished of commercial tools

23. Versatility • Three categories - utilization, communication, and task information • 25 different displays in its current form • Allows for users to create their own visualization windows

24. Scalability • Due to its association and dependency with PICL, its overhead is basically a function of the frequency and volume of communication traffic • Visualization still provides accurate and useful information

25. Maturity • Fairly mature tool • Minor bugs (“features”) • PG has a display limitation of how many processors can be displayed at one time (1024)

26. Verdict on PG • Regardless of its few setbacks and limitations, PG is a full featured and powerful visualization tool for small to medium sized clusters. Its ease of use, display features, developer support, open-source nature, and its huge community support (which allows for numerous improvements over a shorter period of time) it is the best visualization tool available freely over the internet short of developing one yourself.

27. Visualization and Analysis of MPI Programs (Vampir) • VAMPIR is a graphical tool for analyzing the performance and message passing characteristics of parallel programs that use the MPI and/or SHMEM communication libraries. It is the top commercial tool available today and is developed and supported by Pallas.

28. Usability • On-the-fly segment zoom • Built-in snapshot capabilities • Graph to source code linking • Shortcuts to most of the commands and displays • “Point and Click” interface

31. Portability • Architectures supported is simply impressive • Supported on T3E, IBM SP2, SGI PowerChallenge, and Sun Solaris (just to name a few) • Only limited by its demand on any specific platform or research group

32. Versatility • The amount of displays available is just staggering (if you need it, they probably have it or are in the process of making it)

33. Scalability • Causes the least amount of disturbance on the program behavior out of all the tools (its effects are more noticeable on large clusters) • Logistic curve overhead

34. Maturity • Monthly fee, they can also provide 24/7 customer service and support • Has new releases or patches almost every month • Robustness is one of its best qualities (i.e.-systems without a globally consistent clock, VAMPIRtrace automatically corrects clock offset and skew so that timestamps in the merged trace file are consistent) • Cannot benefit from their user base efforts to improve vampir at a faster rate

35. Verdict on Vampir • If you have the money required to purchase and maintain such a product, vampir is the best visualization tool available on the market today. It has the most features, easiest and most intuitive UI, and the largest support infrastructure (funded by its customers). • Works best for very large cluster, where savings in overhead are more noticeable • Displays contain a lot of information and legends are very informative • Visualization features are easy to use and understand

36. Overall Conclusion • AIMS seemed like a very promising tool, but its lack of support, available platforms and features is very disappointing • Vampir has the most features, support and platform availability out of all the tools. Its display flexibility and the sheer volume of data it can display accurately on screen at any given time, make it the perfect tool for very large cluster of computers.

37. Overall Conclusion (cont) • ParaGraph is the clear winner. Nothing more can be said of a FREE tool that rivals Vampir in features, robustness and versatility whose open-source nature and design structure allow it to be a dynamic and highly adaptive tool for today’s fast paced research market.