Selecting a Benchmark Suite to Profile High-Performance Computing (HPC) Machines

New computing architectures raise questions of technical challenges and investment risks to the high-performance computer HPC community. Difficulties exist in determining runtime performance ahead of time and lowering the high costs associated with code migration. Furthermore, the popularity of heterogeneous systems has added complexity in assessing resource capability. A mechanism to quickly and accurately predict performance would be instrumental in assessing the risk of HPC center procurements. A sound metric for performance prediction will help to determine applicability and feasibility of new architectures. To realize the art of anticipating performance, we surveyed the literature on benchmarking suites. This report describes the central characteristics a benchmarking suite should exhibit and derives a list of pertinent requirements for diverse computational resources at the US Army Research Laboratory. Modern benchmark suites are analyzed with respect to the derived list of requirements. This study leads to a benchmark candidate that can be leveraged for a state-of-the-art and next-generation supercomputing facility.