gst-nvdsvpistereo

An internal robot platform used a stereo depth pipeline hardcoded to one camera model and tightly coupled to the rest of the project, and no reusable, standalone DeepStream VPI stereo element existed publicly. The goal was a camera-agnostic GStreamer DeepStream element that runs VPI-accelerated stereo depth on NVIDIA Jetson Orin and attaches disparity and confidence data as standard metadata, so any team could add VPI stereo depth without writing the VPI integration code from scratch.

The element is a GstBaseTransform subclass that takes a side-by-side stereo frame and outputs the rectified left-eye frame, with depth data attached to frame metadata through a probe on the downstream muxer, since the element itself runs before that metadata exists. All VPI state sits behind a private implementation class, so the GStreamer element never exposes VPI headers directly, and rectification maps are pre-computed rather than calibrated in-element, keeping the core dependency-free and startup deterministic. Benchmarking surfaced a real bug, the code mapped a confidence setting to the wrong VPI enum value, and correcting it let confidence computation run on a separate accelerator in parallel with the main disparity calculation, directly improving throughput. Several other parameter constraints were checked against the actual VPI header and corrected, since they didn't match the real hardware limits documented in the code.