How We Push GPUs to Their Limits

AUTHOR:Onie T.
Published:

With all the new advancements in GPU technology, your graphics card is arguably one of the most important parts of your computer, especially if you play modern games. However, with improvements in CPUs and onboard graphics, you might not always need a super powerful GPU.

It's easy to see why we test graphics cards. Different people use their GPUs for different things, so we run lots of tests on different cards. This helps us compare how each GPU handles different tasks. For example, one card might be amazing at playing games like Overwatch, but not great for tasks like machine learning.

Unlike testing PSUs, benchmarking a GPU is something you can do yourself. Testing your GPU can help you find out if anything in your system needs upgrading. You can stress test your GPU to see how it handles power usage, temperatures, and cooling. You can also run non-gaming tests to check its performance for other tasks. But testing it with your favorite game is probably the best way to see if it's working well.

If you're reading this, you may already know how our GPU product pages are set up. If not, here's a quick guide:

  • Hardware: What's In the Box, the GPU's build, and its specs.

  • Features & Software: What features the GPU supports, like encode/decode, and OEM details.

  • Performance: Charts showing how well it performs in games using standard rasterization techniques and some games with Ray Tracing enabled.

  • Productivity: Charts showing how well it performs in non-gaming tasks tests.

Test Setup and Equipment

We use the same equipment for all our tests and regularly update drivers. By standardizing our setup, we eliminate as many variables as possible, ensuring that the only factor affecting our results is the GPU itself. This consistent approach helps provide accurate and repeatable benchmarks that users can rely on when comparing cards. Here's our current test setup as of this post's publication:

  1. CPU: AMD Ryzen 7 7800X3D

  2. Motherboard: Gigabyte X670E AORUS XTREME

  3. Memory: Trident Z5 Neo RGB 6000Mhz 16GB x 2 CL30-38-38-96

  4. Cooling: Noctua NH-D15

  5. Case: STREACOM BC1 V2 - ATX Benchtable

  6. Storage: Samsung 980 PRO 2TB PCIe NVMe 4.0 M.2

  7. PSU: Seasonic Prime PX-1600 ATX 3.0

(Right: Please note the storage (6) and the PSU (7) are not pictured here as they are hidden by other components.)

Test Bench

Unboxing

The first step in testing a GPU is unboxing it. We check what comes in the box and measure important details like the GPU's weight, length, height, and width. We compare these measurements to what the manufacturer says. We also record the specs, like the GPU's architecture, processing units, transistor count, chip manufacturer, clock speed, memory type, shaders, and more.

(Left: If you're unsure what a specific spec means, we usually include a handy 'Tool Tip' - the little question mark bubble next to a spec. Hover over it for a quick explanation.)

Tool Tip GIF

Game Performance

For many people, gaming performance is the most important part of a graphics card. To evaluate a GPU, it's best to compare it with other cards. Testing one card by itself gives us an idea of how well it works, but we really want to know, "How does it stack up against other GPUs?".

In our game tests, we run the same benchmarks on each card under identical conditions, capturing data from every frame. This data includes metrics such as Frames Per Second (FPS) and frame times, which are tracked and recorded for analysis. Frame times are not separately calculated but are part of the captured frame-by-frame data. We then use tools like Grafana, an open-source software, to visualize and present these results in graphs, making it easier to compare performance across different benchmarks.

To make sure our results are accurate, we test each card using the same benchmarks and hardware. We run tests at 1080p, 1440p, and 4K resolutions. Our current test suite includes:

  • Atomic Heart

  • Cyberpunk 2077

  • F1 23

  • Red Dead Redemption 2

  • Returnal

  • Rocket League

  • The Last of Us Part I

(Right: Screencap of Cyberpunk 2077 settings menu.)

Cyberpunk 2077 Games 1080p Screencap

Our goal is to provide a clear view of how each card performs across various titles, demonstrating strengths and weaknesses. By testing at different resolutions, we can also offer insights into scalability, helping users make informed decisions about which GPU suits their gaming preferences best.

Super Sampling

Another key aspect to our game testing approach is super sampling. This technology enhances frame rates by rendering frames at a lower resolution, which is faster, and then upscaling them to the desired resolution using machine learning models. Nvidia's DLSS 3.0 has advanced this by generating additional "free" frames between real frames, effectively smoothing out transitions. While these generated frames might not be perfect and can sometimes show artifacts, the technology can dramatically improve performance, for example, turning a 40 FPS experience at 1440p into a smooth 80 FPS, essentially doubling the frame rate.

To illustrate these effects, we use a bar chart called the settings ladder. This chart compares frame rates from a single graphics card under different scenarios, showing how various settings, like Volumetric Lighting and Fog, impact performance in a game.

Benchmarking

Our benchmarks follow the 3 R's: repeatable, reliable, and relevant. Repeatability ensures the benchmark can be run multiple times with similar results, even with minor variances. We aim for a Percent Relative Standard Deviation of ≤2% between the 95th percentile FPS and average FPS. Reliability refers to the benchmark's consistency in execution. We expect 90% of benchmark runs to be error-free, excluding issues from our own automation. Relevance ensures that the results are meaningful and valuable to stakeholders. By adhering to these principles, we can deliver consistent and accurate performance data.

Productivity & Synthetic Tests

Not everyone uses a GPU just for gaming. After checking gaming performance, we test how GPUs perform for other tasks, like productivity and synthetic tests. We also check how much power the GPU uses and whether it stays cool while running. A card that uses less power and stays cooler will usually give you better performance.

For productivity and synthetic tests, we use programs like Blender, Cinebench, 3DMark, GravityMark, and UniEngine. Occasionally, we also incorporate tests such as Handbrake, PugetBench, and UL Procyon, particularly for products featured in LTT videos. These tools help us gauge how well a GPU handles tasks like rendering, video encoding, and data processing, which are critical for professionals working in creative fields or computational tasks.

In the future, we plan to expand our testing suite to include tools like NVIDIA’s PCAT, HWINFO, and BENCHLAB. These additions will allow us to analyze the power profiles of graphics cards more deeply, providing users with insights into not just performance but also efficiency. Understanding power usage and thermal behavior is essential, as it can significantly impact long-term performance, system stability, and even electricity costs.

(Left: Screenshot of 3DMark benchmarking program.)

3DMark synthetic test screenshot

Lumafield CT Scans

Our product page also includes Lumafield CT scans of the inside of the GPUs. These scans give a nice look at what's underneath the GPU shroud. There's an interactive mode where you can zoom in and out, letting you see the internal layout, measurements, and where the fans and inputs are. This can help PC builders plan their setups without needing the actual GPU in front of them.

The Lumafield scans provide a unique perspective that goes beyond just specs. By revealing the internal architecture of the GPU, users can get a better understanding of the cooling solutions, component placements, and overall design. This level of transparency can be particularly helpful for those who are interested in the technical aspects of GPU construction or for enthusiasts who plan on making modifications, such as adding custom cooling solutions. With these scans, we aim to offer an educational tool that provides more than just performance data, helping users make well-rounded purchasing decisions.

(Right: Lumafield CT Scan of a NVIDIA GeForce RTX 3060 Ti FE graphics card.)

[LumafieldEmbedUrl:<https://app.lumafield.com/project/e0576396-e4f5-4977-9b6f-7c7efaf07eba?embed=true>]

Limitations and Challenges

When testing GPUs, it's important to control as many variables as possible, so the only thing affecting the results is the GPU itself. This is hard because the GPU is just one part of a bigger system. To really test a GPU in isolation, you would need to create custom components that control every factor, which is not something we can do right now.

We strive to maintain a consistent testing environment by conducting tests in a climate-controlled space with a stable ambient temperature. Test benches are arranged to minimize heat interference, ensuring that they are spaced apart rather than clustered together, preventing them from affecting each other's performance.

Software variables, like driver versions, also play a crucial role in graphics card performance. When a new card is released, a dedicated driver often accompanies it to support the new architecture. Over time, as drivers are refined, the card's performance can improve through optimizations and fixes that emerge from widespread use, which aren't always identified during initial QA testing.

There are still some challenges in this area, and no definitive approach has been established yet. Ideally, we would retest all graphics cards in our library with every new driver release. While large-scale regression testing might be possible in the future through automation and additional resources, for now, we will adopt a more flexible driver version policy.

Ready, Set, Benchmark

Testing GPUs is crucial for ensuring that your computer runs smoothly, especially as games and software become increasingly demanding. While the GPU plays a key role, it’s not the only component that matters. Regular benchmarking helps identify potential issues and determine whether an upgrade is needed.

Whether you're a gamer or use your computer for other tasks, understanding how your GPU performs allows you to optimize your system's performance. Benchmarking isn’t just for experts—anyone can do it with the right tools. Check out our MarkBench tests repository and keep an eye out for future articles for guidance on how to benchmark your GPU using our custom tests.