Principles for automated and reproducible benchmarking

Koskela, T., Christidi, I., Giordano, M., Dubrovska, E., Quinn, J., Maynard, C., Case, D., Olgu, K. and Deakin, T. (2023) Principles for automated and reproducible benchmarking. In: SC-W 2023: Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis, 2023-11-12 - 2023-11-17, Denver, Colorado, pp. 609-618. doi: 10.1145/3624062.3624133 (ISBN: 9798400707858)

Abstract/Summary

The diversity in processor technology used by High Performance Computing (HPC) facilities is growing, and so applications must be written in such a way that they can attain high levels of performance across a range of different CPUs, GPUs, and other accelerators. Measuring application performance across this wide range of platforms becomes crucial, but there are significant challenges to do this rigorously, in a time efficient way, whilst assuring results are scientifically meaningful, reproducible, and actionable. This paper presents a methodology for measuring and analysing the performance portability of a parallel application and shares a software framework which combines and extends adopted technologies to provide a usable benchmarking tool. We demonstrate the flexibility and effectiveness of the methodology and benchmarking framework by showcasing a variety of benchmarking case studies which utilise a stable of supercomputing resources at a national scale.

Item Type	Conference or Workshop Item (Paper)
URI	https://reading-pure-test.eprints-hosting.org/id/eprint/114121
Identification Number/DOI	10.1145/3624062.3624133
Refereed	Yes
Divisions	Central Services Science > School of Mathematical, Physical and Computational Sciences > Department of Computer Science
Download/View statistics	View download statistics for this item