Closed general

Native recorder: zero-copy GPU frame path (DXGI surface buffers -> NVENC/QSV/AMF) instead of the per-frame GPU->CPU staging hop

John Lauer · 2mo ago ·closed by John Lauer

Follow-up to #143 (John, 2026-09-05: "if a user has an nvidia gpu ... do we take advantage of that as well for full-screen recording? cuz we should").

State in ab 2.1.36: the native recorder (wgc_record.rs) creates its Media Foundation sink writer with MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS, so MF instantiates the vendor H.264/HEVC encoder MFT when one is registered (NVENC on NVIDIA, Quick Sync on Intel, AMF on AMD); desktop_record_start reports encoder and hardwareEncoder (hardware detected from the live transform's MFT_ENUM_HARDWARE_URL_Attribute). So the ENCODE is on the GPU where one exists.

What is still on the CPU: every captured frame takes a GPU -> CPU hop (WGC texture -> D3D11 staging copy -> Map -> BGRA slot -> IMFMediaBuffer), and the encoder MFT re-uploads it. At 2560x1600@30 that is ~490 MB/s of PCIe/memory traffic each way; at 4K@60 it is ~2 GB/s and the copy itself starts to cost frames.

Ask: a zero-copy path when a hardware encoder is present.

  1. Create the sink writer with MF_SINK_WRITER_D3D_MANAGER set to an IMFDXGIDeviceManager bound to the same D3D11 device WGC captures into.
  2. Per frame, copy the WGC texture into a pool of encoder-owned NV12 (or BGRA, if the MFT accepts it) default-usage textures on the GPU (a D3D11 video processor does the BGRA->NV12 conversion on the GPU), wrap with MFCreateDXGISurfaceBuffer, and WriteSample that - no Map, no CPU BGRA slot.
  3. Keep the CPU path as the fallback for software encoders (WARP boxes, VMs) and report which path is in use (framePath: "gpu" | "cpu") next to encoder.
  4. Measure before/after on AdomLapper (NVIDIA): CPU % of ab during a 4K@60 take, dropped frames vs requested fps.

Also worth doing in the same pass: expose codec:"hevc" in the hint once the hardware HEVC MFT is confirmed on all three NVIDIA boxes (smaller files at the same quality), and a bitrate arg.

2 Replies

John Lauer · 2mo ago

Shipped in 2.1.51.

The native recorder now keeps frames on the GPU: setup_gpu_path builds an IMFDXGIDeviceManager on the WGC D3D11 device and Media Foundation's own IMFVideoSampleAllocatorEx pool of DXGI-backed samples; FrameArrived does one GPU copy into a 'latest' texture, and the encoder tick copies that into a pooled sample and WriteSample's it — no per-frame CPU download. The MF sample pool is the safety net (a texture is recycled only after the sink writer + encoder release it), so there is no hand-rolled ring to get wrong. CPU staging remains the automatic fallback at both setup stages (path setup + sink-writer setup), and gpu:false forces it for A/B measurement.

Response fields: gpuZeroCopy, framePath (gpu|cpu), bitrate, droppedFrames. Measured 2560x1600@30 for 20s: GPU path ~11% CPU / 555 frames vs CPU path ~33% / 419 frames.

John Lauer · 2mo ago

Closing: shipped in 2.1.51 (DXGI-backed MF sample pool on the WGC D3D11 device, one GPU copy per frame, no per-frame CPU download). Fleet is on 2.1.72 insiders.

John Lauer closed this · 2mo ago

Log in to reply.