NVIDIA Cosmos models dominate video prediction at ECCV 2026 AI City Challenge

Four of the top five solutions in the generative video prediction track (Track 5) built on NVIDIA Cosmos foundation models. The public leaderboard winner, team Qyn, created CosmosAlign by fine-tuning Cosmos3-Nano, the lightest model in the family, to forecast realistic traffic scenarios from a short visual history and textual descriptions of vehicle and pedestrian motion. The task required systems to predict what happens next in a scene, not merely detect objects in existing frames.

In the multi-camera 3D perception track (Track 1), team EVA won with GaugeAlign. The team diagnosed the sim-to-real gap as fundamentally geometric rather than purely visual. Their approach uses camera calibration to transform the scene into the coordinate convention expected by a frozen Sparse4D checkpoint from NVIDIA, then maps predictions back — without retraining the model.

Track 2 tested traffic safety understanding: models trained on synthetic digital-twin videos were evaluated on real multi-angle traffic footage. The winner, Latent Painter - UTE, introduced Traffic-JEPA, a two-stage system that supplies a language model with an evidence scaffold instead of demanding single-step reasoning and captioning. The core idea: understand the traffic event first, verify consistency, then generate the caption.

For traffic anomaly reasoning (Track 3) — determining whether an event occurred, what happened, its cause, and a scene summary — Stellarview AI won with Reason or Recite. The system runs two expert models: one trained on rewritten training traces that separate scene, question, timestamped evidence, reasoning, and conclusion; the other trained on the answer style from annotations. Both produce a candidate answer for each question, and a frozen language model acts as judge to select between them.

The first out-of-domain leaderboard for Track 3 focused on fisheye intersection video. Systems had to identify a violation, the road user involved, scene motion, path, conditions, and timing, then return a structured report and description. Team UWIPL_ETRI topped this ranking, demonstrating adaptation to a camera type and environment unseen during training.