Sapient Intelligence releases HRM-Text-1B checkpoint on Hugging Face

Sapient Intelligence has published a one-billion-parameter language model checkpoint on Hugging Face built on its Hierarchical Reasoning Model architecture. Instead of deepening a standard transformer stack, HRM runs two separate transformer modules — a high-level, slow H module and a low-level, fast L module — through repeated iterations on the same input embeddings with additive state injection between them. The design yields effectively unbounded compute depth at a fixed parameter count.

Dual-scale architecture replaces deep stack

The model executes H_cycles × (L_cycles + 1) compute steps on each input. The slow module updates a global state while the fast module processes locally, and the sum of both is injected back at every step. This inverts the usual "more layers equals more reasoning" paradigm: depth is created at inference time, not during training. The checkpoint is released as pre-alignment only — trained on a PrefixLM objective with condition prefix tokens but without SFT, RLHF, long-context adaptation, or multi-turn dialogue tuning.

Four condition modes and how to use them

Four tags are available at the tokenizer level: direct (straight answer), cot (chain-of-thought), noisy (raw web style), and synth (synthetic/curated style). They map to special tokens and are fed as a single prefix block delimited by ▁…▁. For classic NLP tasks — classification, extraction, structured output, short-form QA — the recommendation is direct with 2 to 8 few-shot examples in context; pure zero-shot is markedly weaker. For mathematical reasoning or open-ended generation, use the composite synth,cot in that order, which emits the <|quad_end|><|object_ref_end|> prefix and triggers CoT-like behavior, though quality is inconsistent and falls short of a similarly sized instruction-tuned model.

Runtime requirements and the PrefixLM mask

The model requires transformers ≥ 5.9.0, which includes native support for the hrm_text class. In the demo code the model loads in bfloat16 on GPU; the critical detail is token_type_ids. A value of 1 marks tokens belonging to the prefix block, where attention is bidirectional within the block; all other tokens receive causal attention. Without this field the mask diverges from training and performance degrades. The example demonstrates deterministic generation (do_sample=False) up to 256 new tokens.

Starting point, not a finished product

Sapient Intelligence stresses that the checkpoint is intended as a launchpad for researchers and developers who want to build their own alignment, SFT, and/or RL on proprietary data. The instructional capability that surfaces under synth,cot is described as a side effect of the pre-training mixture, not a guaranteed feature. For anyone seeking a ready-to-use chat assistant, this is not the target; for anyone investigating recurrent reasoning architectures and wanting open weights for experimentation, it is a low-weight entry point.