Standard Performance Evaluation Corporation
SPEC/HPG: Measuring real-world supercomputing performance
As far back as the early 80s, David Kuck and his supercomputing colleagues knew there needed to be something better than kernel-level benchmarks. In response to this need, he formed the PERFECT Club in mid-1980s.
PERFECT stands for PERFormance Evaluation for Cost-Effective Transformations. The Club comprised a couple dozen members from various organizations united in developing a better supercomputing benchmark. The result was a benchmark representing 13 Fortan-based engineering and scientific applications containing more than 50,000 lines of code.
But, Kuck had a problem. He realized he couldn't keep the benchmark moving forward without a lot of help. That's when he and Rudi Eigenmann, a member of Kuck's research team, realized that SPEC would provide the ideal home for continued benchmark development.
Currently, SPEC/HPG covers the supercomputing arena with three benchmark suites:
In March 2018, SPEC/HPG made a bold move designed to provide easy access to tools for research vital to the ongoing development of supercomputing technology: It began offering its benchmarks free of charge to qualified non-profit organizations worldwide.
"SPEC/HPG benchmarks provide a framework that helps to ensure the rigor, reproducibility and application-based versatility that are critical to sound research efforts."
-- Torsten Hoefler, who leads the Scalable Parallel Computing Laboratory (SPCL) at ETH Zurich and co-chairs the MPI Forum's collective operations working group.