https://lwn.net/Articles/1051430/

[dotstdy@mastodon.social]

https://lwn.net/Articles/1051430/

There's a note here about rng in games obscuring benchmarking, but honestly that's the least of your problems. Games are highly workload adaptive, and recent games even more so. That is, they will adapt the amount of work they do each frame to the amount of space they have available in the frame. This is pervasive, from things like dynamic resolution, down to systems like loading which are inherently asynchronous, and further to systems which are explicitly timesliced.

[dotstdy@mastodon.social]

e.g. Imagine you write a scheduler that de-prioritizes background work heavily. That's going to mean you're streaming less stuff each frame, which limits the amount of per-frame "finish streaming" work which gets scheduled. Double whammy on improving frame times, but you're now at risk of not streaming or destreaming objects fast enough to keep up with game content. Increasing the blurryness of textures, or perhaps causing excessive memory growth as destruction doesn't keep up.

[dotstdy@mastodon.social]

Or e.g. imagine you have a title which dynamically choses how many npcs to tick each frame based on the available frame time. When you reduce the timeslice you can trigger heuristics which reduce the overall load on the system, or visa versa. There's also a big risk using game demo recordings and benchmarks, because they typically aren't running the full host of gameplay systems in the same way the game itself does. Obviously it's attempted to keep it close, but the workloads are different.

[dotstdy@mastodon.social]

Also just broadly surprised they're not making long traces with trace-cmd and gpuvis, with a ring buffer trace you can even automatically trigger a dump once you see an outlier frame.