undefined
@pixelcluster The problem with sampling total memory usage is that the applications don't talk to each other.
If one program is hogging all of the VRAM and another wants more than what's available, then there are really two options: Fail allocations in the new program (allowing the first-comer to monopolize the VRAM), or make the first one lower its memory usage.
So, the point of having budgets is to have some way of coordinating VRAM usage reduction across multiple programs.
@pixelcluster What I mean is if the kernel decides on per-process budgets that total less than the physical memory, then it can guarantee that only over-budget processes will get evicted, and that if it has to evict an under-budget one anyway, that it will always update the budgets when it does.
@mjp @dotstdy You can already request a minimum reservation. Problem is, how much can I use above the must-have? And what happens when the OS says "no you can't have all of that any more?"
And yeah in practice, I know the solution is to not oversubscribe VRAM, problem is I want to be able to either avoid oversubscribing or tell the user they're out of VRAM because otherwise guess who gets blamed for the game running at 2 FPS?
@pixelcluster The kernel can decide to just not evict anything from under-budget processes as long as the total's under physical memory.
It's kinda fine if the OS decides to lower the budget and start evicting memory without warning.
What's not fine is the memory becomes persistently evicted and the program has no (good) way of detecting the problem and recovering, turning what would be a short stall into an ongoing perf hit that makes it basically unusable.
@dotstdy @mjp It already gives some priority (i.e. more budget) to the in-focus program and that's probably a good baseline. Ultimately though, I think it's less important that it comes up with good budget numbers than that it tries to maintain the invariant that the in-focus program stays fully resident if it stays under its budget number.
If it can't do that, then yeah I at least want to know that it's demoting to provide feedback to the user that VRAM is critically low.
@mjp Personally I don't think being notified of demotion is that useful because there isn't a guarantee that freeing X amount of memory will make the OS give it back.
If you're over 250MB and you dump 250MB of allocations and... you've still got 250MB demoted... then what? (Did some other app's usage go up, or are we just waiting?)
If the OS dictates memory demotion, then it should be giving applications accurate budgets.
So basically you're supposed to do something like set up a persistent service with the appropriate user permissions to manage a limited system resource tracking an undocumented kernel event just to get around DXGI giving you a "stay under this number and everything'll be okay" number that doesn't actually work.
Global traces are also not automatically cleaned up on process exit, so if your tracing process crashes, the trace leaks.
Creating traces is also restricted by security permissions, but you won't find that out on a dev machine because Visual Studio installation adds NT AUTHORITY\INTERACTIVE (a.k.a. any locally logged-in user) to the Performance Log Users group.
You have to set up a kernel trace with ETW and monitor the VidMmProcessDemotedCommitmentChange event. That event is not actually documented anywhere except for a random man file in WPT. Searching for it on Google yields SIX hits, and five of those are NSIGHT docs where it just tells you it logs it.
Oh wait you want to do this on a USER'S machine? Buckle up!
ETW in-process traces don't support real-time mode (only logfile), have to use a global trace. There's an OS-wide limit of 64 traces.
Allow me to share a story of the worst thing in D3D12: Handling VRAM exhaustion.
If multiple programs are collectively using too much VRAM, Windows will start demoting VRAM to system memory, causing perf to catastrophically tank.
Fortunately, DXGI provides QueryVideoMemoryInfo which gives you usage+budget info for your program. Unfortunately, THE BUDGET NUMBER IS FAKE AND DOESN'T WORK. You can stay under it and still get demoted.
How are you supposed to actually detect demotion? ...
@TomF @rygorous The Vulkan spec actually says round up but the validation layer rounds down, whee
edit: Bug report deployed! https://github.com/KhronosGroup/Vulkan-Docs/issues/2588
@rygorous Er, since when are mipmap sizes rounded up from the previous dimension?
If I feed a 123x456 image to texconv, the first mip is 61x228. Vulkan Images chapter apparently doesn't specify rounding (time for a bug report!), ARB_texture_non_power_of_two spec explicitly rounds down, DirectXTex rounds down, NVIDIA's got a paper on this saying round down, and I've been dealing with this issue directly for a couple months.