Jina AI releases lightweight OCR model with built-in speculative decoding

Jina AI, now part of Elastic, has launched jina-ocr-v1, an end-to-end visual document decoder that ingests PDFs, scans, tables, charts or invoices and returns clean Markdown in a single pass. The model carries 3.4 billion parameters in total, yet only roughly 570 million are active per token thanks to a mixture-of-experts architecture, and it is designed to run on relatively inexpensive cards such as the Nvidia L4. The technical report cites 91.14 on OmniDocBench v1.6 and 83.4 on olmOCR-Bench, but accuracy is not the headline here: throughput is.

Lean architecture with a compressed encoder

The model builds on DeepSeek-OCR and retains its two efficiency components. The encoder, called DeepEncoder, weighs about 380 million parameters and chains SAM, a 16× convolutional compressor and CLIP-L, turning a 1,024 × 1,024 page view from 4,096 patches into just 256 visual tokens. A dynamic-resolution mode adds up to nine local tiles of 100 tokens each, capping a page at 1,156 visual tokens maximum. The decoder is DeepSeek-3B-MoE with 12 layers, 64 routed experts and two shared experts; top-6 routing activates roughly 570 million parameters per token, and the position limit sits at 32,768. Output is returned as Markdown, tables in HTML and formulas in LaTeX.

Speculative decoding without accuracy loss

The competitive edge comes from FastMTP, a speculative-decoding head that ships inside the checkpoint itself. Because OCR output is nearly deterministic and locally structured, it suits this approach particularly well. The head adds a single compressed draft block that is executed recursively for K = 3 steps; the draft parameters stay fixed as depth increases. The decoder verifies the drafts greedily, accepts the longest prefix that matches its own picks, and commits one extra token of its own. If all three drafts match, the extra token is a bonus. The committed text is always identical to standard greedy decoding, so the acceleration is lossless. At K = 3 the model commits an average of 2.73 tokens per step.

Post-training with verifiable rewards

Continued training blends instruction alignment, robustness fine-tuning on corrupted pages and GRPO. Every reward component is deterministic code scored against a reference transcript, and the components cover content, formulas, tables, structural validity, unit tests, repetition and format. The components are multiplied together and each is ranked separately, so partially correct pages receive partial credit. Structure, unit-test and format components receive a floor of 0.2, the table component a floor of 0.1, and the repetition component no floor, because loops could inflate the content score. Since formulas and tables appear in only a few samples of natural pages, Jina built JinaOCRSynth, synthetic pages dense in both, each carrying olmOCR-Bench-style unit tests. An agent also merges candidate checkpoints under a fixed evaluation budget, and the draft head is trained last against the frozen final verifier.

High throughput on cheap hardware

In throughput tests the picture flips. On an A100 40 GB at concurrency 32, jina-ocr-v1 processes 2.57 pages per second, the highest among the 14 systems Jina measured, versus 1.22 for olmOCR-2 and 0.38 for chandra-ocr-2. The model emits 1,085 output tokens per page, which Jina says is the shortest output among systems scoring 83. On an L4 at batch size 1, greedy (eager) decoding jumps from 42.7 to 83.1 tokens per second, a 1.95× speedup at a 57.6% acceptance rate. With CUDA graphs the baseline already stands at 158.3 tokens per second, and there K = 1 wins with 185.6 tokens per second, a gain of only 1.17×.

License and run paths

The open weights weigh roughly 6.8 GB in BF16 and run on Transformers or vLLM. The license is CC BY-NC 4.0; commercial use requires contacting Jina. The fastest path is Jina Reader: send a URL to r.jina.ai with the appropriate parameter. Alternatively, the model can be loaded locally through the standard libraries.