Marigold V2 sharpens diffusion transformers into better monocular depth estimators

Researchers took Qwen-Image-Edit-2509, a diffusion transformer built for image editing, quantized it to 4-bit, added QLoRA at rank 128, and fine-tuned it for depth estimation on a single 32 GB consumer GPU. Training finishes in a few days. Inference runs in a single flow-matching step without running out of memory even at 2K resolution. All weights, code, and a demo are public; the paper is slated for SIGGRAPH Asia 2026.

Two core changes drive the improvement. V1 compressed depth maps through an image VAE — a choice that sounded wrong but worked. V2 replaces that with internal semantic alignment: the ground-truth depth map passes through DINOv3, and the DiT's representations are forced to align with it (iREPA-depth). The second problem is that synthetic ground truth such as Hypersim is imperfect; grass blades receive broken depth values. The fix is SinkLoss: the image is divided into 25 tiles (5×5), and a soft Sinkhorn matching aligns the 25 predicted values with the 25 ground-truth values instead of a hard pixel-to-pixel loss. The function tolerates noise in the ground truth and produces cleaner maps.

On KITTI and ETH3D the model reports a 16–26% improvement in AbsRel over the previous best model trained on comparable data. Qualitatively it resolves fine edges, fur, foliage, and hair that earlier models missed. On NYUv2, ScanNet, and DIODE benchmarks, Marigold V2 leads among models trained at similar data scale. The gains carry over to other dense regression tasks: surface normal estimation and intrinsic image decomposition.

The same components — semantic alignment and SinkLoss — serve all Marigold modalities: depth, normals, albedo. The researchers note the recipe covers every task "simply better." Unlike robotics depth that stops at glass, computational photography demands transparent depth, and the model is tuned for that as well.