Cloudflare releases Clef-Omni, a multimodal MoE model for structured decisions

Cloudflare has released Clef-Omni, a 30-billion-parameter mixture-of-experts model with 3 billion active parameters, built on top of Qwen's Qwen3-Omni-30B-A3B-Instruct. Unlike general-purpose language models, Clef-Omni does not generate free-form text. It accepts a state and a schema of typed questions, returning a probability for every allowed option on each question in a single forward pass. Inputs can be text, JSON, images, audio, or video; the output is logits for each option, which after a per-question softmax yield a full probability distribution.

The backbone is Qwen3-Omni's "thinker" with its vision and audio encoders, stored as sharded safetensors. The original speech-generation weights — talker and code2wav — are included in the package but are not loaded at runtime; load_release_model skips them. Sitting above the backbone is a joint schema head: a small transformer that reads the final hidden states, routes evidence from the state to each question, and scores all options across all questions together. The approach eliminates the need to parse textual output and guarantees the model returns exactly the structure defined in the schema.

According to Cloudflare's testing, the model runs on a single H200 with roughly 64 gigabytes of GPU memory in bfloat16, under PyTorch 2.11 and transformers 5.10.2. Image inputs require the Pillow library; audio and video files require av, the Python bindings for FFmpeg. Video is sampled at two frames per second regardless of whether the source is a file, URL, base64 string, RGB frame array, or local path.

The API is fully compatible with Jev and SystemOne. The systemone function accepts a standard SystemOne POST request body and returns decoded answers keyed by question ID: a choice response with the selected option, confidence, and probabilities; a score response with expected score, confidence, legend, and probabilities; or a noul (yes/no) response with a truth probability. The code example demonstrates routing a fault message to the correct department, assessing urgency, and identifying whether a service outage has occurred — all in one call, with the option to attach images, audio, or video as base64-encoded data URLs.

Clef-Omni targets tasks where output must be structured and validated — invoice classification, alert routing, field extraction from multimodal documents — without chaining a language model to external validation. Building on an existing Qwen model rather than training from scratch reduces training costs, but it also means the backbone's limitations — context window, video encoding quality — are inherited. Cloudflare has not published comparative benchmarks against comparable dense models, so the trade-off between active parameter count and decision quality remains open for independent verification.