Ai2 wraps COLM 2026 with hybrid architecture and scientific tooling

The Allen Institute for AI (Ai2) closed its presence at COLM 2026 in San Francisco with a series of announcements that illustrate its operating model: a nonprofit lab that releases models, data, code and training infrastructure in full. The conference served as a venue for a paper on hybrid architecture, an update on the next generation of the Olmo family, the launch of training infrastructure for mixture-of-experts models, and an agentic platform for scientific work — all under open weights and full reproducibility.

Olmo Hybrid: fewer tokens, same result

The paper "Olmo Hybrid: From Theory to Practice and Back" examines a combination of the classic attention mechanism with linear recurrent layers. Attention retrieves specific details from early context; recurrent layers maintain a compressed state that updates with each token. In controlled experiments, Olmo Hybrid matched the accuracy of Olmo 3 7B on MMLU, a broad general-knowledge benchmark, while using 49% fewer training tokens. The conclusion: adding recurrence lets the model compress the same knowledge into less compute, and the findings are already feeding the next generation of pre-training.

Next generation: hybrid MoE and open infrastructure

The next model in the family is now in pre-training with a hybrid mixture-of-experts architecture that combines attention and recurrence with token routing to parameter subsets — another efficiency lever. At the same time, Ai2 released Olmo-core 3, a redesigned version of its training library that adds a rearchitected system for training large MoE models. Researchers can run the code themselves, inspect training decisions, and use checkpoints and technical reports from the broader Olmo ecosystem. Google recently reported that it reproduced the Olmo 3 7B training run in MaxText on its cloud TPUs, practical proof of the reproducibility that a fully open approach enables.

Asta: a scientific agent with citations

On the applied side, Ai2 is developing Asta, an agentic platform for scientific work built in collaboration with domain experts. The first published component, AstaBrief, is an open-weights model that produces cited reports from a research question and automatically retrieved literature. Scientists can run it in the platform's Fast mode or download the weights for local infrastructure. According to Noah A. Smith, senior NLP research director at the institute, the goal is to understand where general models fail on real scientific tasks and what future open models need to support.

Bolmo in Nature: a byte-level model

During the conference, Nature published the paper "Retrofitting language models to operate over bytes," the work behind Bolmo. Most models today break text into tokens from a fixed vocabulary; Bolmo operates directly on bytes, enabling a uniform representation for any language, code or data format without tokenizer dependence. Publication in a leading scientific journal signals academic recognition of the direction Ai2 is pushing: open artifacts that other researchers can examine, reproduce and extend, without walled gardens or restrictive licensing terms.