Recirculation cuts perplexity by 23% in Gemma 3 models without extra training

Researchers presented a method called Recirculation that adds recursion to existing transformer models at inference time only, without modifying weights, without retraining, and without an additional tensor for token creation. The idea is to inject activations from the upper layers back into lower layers at the next step, allowing the model to operate as a dynamic system that tracks belief states instead of being confined to a fixed network depth.

Instead of relying solely on a one-directional forward pass, Recirculation creates a feedback loop between high and low layers at each inference step. The authors note that this is not Chain-of-Thought (CoT), which is intended for complex reasoning, while Recirculation handles basic state tracking. It is also not classic looping that repeatedly applies the same layers, nor a recurrent transformer that requires costly training. The only overhead is that the prefill stage becomes serial rather than parallel, whereas the decode stage retains the same speed.

Using the adaptive version—which requires only a light hyper-parameter adjustment while freezing the original weights—the approach achieved a 23% reduction in perplexity across a suite of datasets and a 21% increase in exact match on GSM8K. Comparable gains were reported on additional downstream tasks. All experiments used the Gemma 3 family as the base model, and the method is defined as fully training-free.

The work appears on arXiv in the Computer Science > Machine Learning category, indicating a preprint that has not undergone peer review. Code, data, and demo scripts are publicly available, but the reported metrics come from the authors without external validation. Independent replication is recommended before drawing general conclusions.

Conceptually, the approach points toward model-guided architectural evolution: rather than designing arbitrary architectures and training them from scratch, the technique leverages the trained model itself to suggest architectural modifications. If it scales, it could open a path to “architectural evolution,” where a network informs improvements to its own structure without an additional training cycle.