Probability and Percentage in Tech Benchmarks

A single benchmark score is a statistical summary. Like any summary, it compresses a distribution of results into one number, and that compression loses information. The number feels precise, which is why it gets quoted in headlines and comparison tables, but its precision is often an illusion: it describes one measurement under one set of conditions at one moment, not the reliable expected performance of the hardware.

Understanding what percentage figures and variance mean in benchmark results is not a technical exercise in statistics. It is a practical skill that separates a well-informed hardware decision from one based on marketing-friendly numbers.

Averages Hide the Edges

Average framerate is the figure most gaming benchmarks lead with. It tells you the arithmetic mean of frames delivered over the test period. The problem is that the edges of the distribution, the moments where performance drops, are what determines whether a game actually feels smooth. A session averaging 80 fps is perfectly playable if frames are delivered consistently. The same session averaging 80 fps but dropping to 20 fps for half a second every thirty seconds feels stuttery and unpleasant, even though the average looks fine.

Reading a benchmark distribution works much like understanding odds on a platform like jemputhoki: the average figure hides more than it reveals. A probability distribution shows you not just the expected outcome but the variance around it, which is the information that determines whether the experience will be consistently good or only good on average. The same logic applies to benchmark data.

The 1 percent low framerate, sometimes called 1% low or the first percentile, captures the worst 1 percent of frames in a test sequence. It is a better predictor of whether gameplay feels smooth than the average because it describes the floor of the experience. Hardware that delivers 90 fps average with 55 fps 1% lows will feel noticeably smoother than hardware delivering 95 fps average with 25 fps 1% lows, despite the latter having the higher average. The high variance in the second case means the user regularly experiences those 25 fps moments, even if they do not persist.

Variance in Benchmark Suites

When the same benchmark runs multiple times on the same hardware, the results are not identical. Thermal conditions change as the hardware warms up. Background processes activate briefly and consume CPU or memory. The operating system's scheduler makes different decisions. These sources of variation produce a spread of results even in controlled conditions.

A single benchmark run captures one point in that spread. It may be near the mean, near the high end, or near the low end, and there is no way to know from a single result. This is why I run any benchmark I am relying on at least three times and look at the range before drawing a conclusion. If three runs of a CPU benchmark produce scores of 1640, 1648, and 1651, the results are consistent and any of them is a reliable indicator. If they produce 1640, 1590, and 1660, something is introducing significant variance, and the mean of those three runs is more informative than any individual score.

Standard deviation, when it appears in benchmark reports, quantifies this spread. A low standard deviation means results cluster close to the mean: reliable, consistent performance. A high standard deviation means the hardware's output varies significantly between runs: less predictable in real use. Reviews that show only a single run without acknowledging variance are telling you less than they could.

How to Read a Benchmark Graph

Distribution graphs, where they appear, show how often results fell within each range. A tight bell curve, most results clustered near the middle, indicates consistent performance. A wide, flat distribution indicates variable performance. A bimodal distribution, with two separate clusters of results, usually indicates the hardware is operating in two distinct states, possibly thermal throttling that cuts in above a certain temperature.

Box and whisker plots, used in some detailed technical reviews, show median, interquartile range and outliers explicitly. The median line in the box is the central result. The box covers the middle 50 percent of results. The whiskers extend to capture most remaining results. Points beyond the whiskers are outliers. This format conveys far more than a single number and is worth spending a moment to read correctly.

For storage benchmarks, queue depth matters significantly and changes the interpretation of results. A random read at queue depth 1, where one request is issued at a time, is what everyday desktop use looks like. The same drive's random read at queue depth 32, where 32 requests are issued simultaneously, measures a different capability relevant to server workloads. Consumer SSDs are often benchmarked at high queue depths because the numbers are larger and more impressive. The queue depth 1 result is the one that predicts daily experience.

The Practical Approach

For any hardware purchase decision, collect results from at least three independent sources and look for agreement. Where results agree strongly across sources, the figure is reliable. Where they diverge, understand why before treating any single source as authoritative: different thermal conditions, different test sequences, different driver versions, or different retail units can all produce legitimate variation.

Identify the specific workload you care about and find benchmarks that reflect it. General-purpose synthetic scores are a starting point, not a conclusion. A GPU benchmark designed around ray tracing performance tells you less about rasterisation gaming than a benchmark built around the games you actually play. The score that appears in a headline was chosen because it presented the hardware favourably or because it was the most easily automated test, not because it is the most informative figure for your use case.