Latest round Includes over 1,800 performance and 350 power results for leading ML inference systems
Today, MLCommons, an open engineering consortium, released new results for MLPerf Inference v1.1, the organization's machine learning inference performance benchmark suite. MLPerf Inference measures the performance of applying a trained machine learning model to new data for a wide variety of applications and form factors, and optionally includes system power measurement.
MLPerf Inference is a full system benchmark, testing machine learning models, software, and hardware.
The open-source and peer-reviewed benchmark suite provides a level playing field for competition that drives innovation and performance for the entire industry. While the majority of systems improved by 5-30% in just 5 months, some submissions have improved by more than two times previous performance, demonstrating the value of software optimization that will have a real impact on AI workloads.
Similar to past MLPerf Inference results, the submissions consist of two divisions: closed and open. Closed submissions use the same reference model to ensure a level playing field across systems, while participants in the open division are permitted to submit a variety of models. Submissions are additionally classified by availability within each division, including systems commercially available, in preview, and RDI (research, development, and internal).
MLPerf Inference v1.1 results further MLCommons’ goal to provide benchmarks and metrics that level the industry playing field through the comparison of ML systems, software, and solutions.
The latest benchmark round received submissions from 20 organizations and released over 1,800 peer-reviewed performance results for machine learning systems spanning from edge devices to data center servers. This is the second round of MLPerf Inference to offer power measurement, with over 350 power results.
