While hardware advancements have led to ever-increasing maximum peak performance for modern high-performance computing platforms, most software has not been able to exploit these advances to attain similar performance increases. The gap between theoretical peak performance and real-world performance has been driven further apart by increasing hardware and software complexity. As a result, parallel performance tools are playing an increasingly important role in the software development process.
The existence of several performance tools gives software developers many choices for analyzing performance bottlenecks in their code. While the plethora of performance tools affords developers much flexibility, the sheer number and variety of tools can be daunting. A comprehensive overview of available parallel performance tools aids developers since it can serve as a road map for what tools and analysis methods are currently available. Additionally, an overview also aids researchers developing new tools by enabling them to see what functionality is currently available and what open research issues still exist.
We are currently designing a next-generation performance analysis tool designed specifically for global address languages. As preparation for this task, we have performed an extensive review of performance tools. We have chosen a wide variety of tools that employ different analysis techniques and have evaluated them against an application suite exhibiting known performance problems. This web page presents the findings from that tool study.
Since MPI is a very popular standard for current high-performance computing software, most of the tools we examined are geared towards performance analysis of MPI programs. Also, while techniques such as modeling and simulation may be used to reason about a program's performance, experimental performance analysis has historically been the most effective method for troubleshooting performance problems in real code. Therefore, we focus on experimental performance analysis tools as these types of tools will be the most useful for the majority of software developers.
Summary of all tool evaluations
Also see our scoring criteria and our preliminary tool testing plan.
Dimemas and Paraver (overhead measurements)
DynaProf (overhead measurements)
HPCToolkit (overhead measurements)
Intel Trace Collector and Trace Analyzer (overhead measurements)
KOJAK (overhead measurements)
MPE/Jumpshot (overhead measurements)
MPICL and Paragraph (overhead measurements)
mpiP (overhead measurements)
Paradyn (overhead measurements)
SvPablo (overhead measurements)
TAU (overhead measurements)
AIMS
CrayPAT
dtrace
DynTG
Eclipse PTP
The suite of programs used to perform the bottleneck tests is available here.