AI learns to assemble furniture, changing the game

Researchers have launched the Furniture Assembly Benchmark (FAB), a test that asks vision models to spot errors in partially built IKEA furniture. The premise is straightforward: if a model can catch a missed screw or a backwards panel in the early steps, it stands a chance of guiding a technician repairing a washing machine or a mechanic swapping brake pads. The dataset contains 60 images across three furniture models at varying complexity levels, measured by IKEA's own complexity index (steps multiplied by parts). Each image comes with the original manual, a zoom tool for close inspection, and a sandbox with a Python interpreter.

The progress is sharp. In November 2025, Anthropic's Claude Opus 4.5 led with 28% success. Ten months later, in September 2026, OpenAI's GPT-6 Astra leads with 80%. The gap isn't just quantitative: OpenAI's model returns an answer in a median of three minutes per image, two to ten times faster than the previous leaders. Speed matters for real-time use; a model that takes half an hour won't work for a technician in the field.

Closed-weight models have held the lead throughout. Chinese open-weight models from Moonshot and Alibaba were tested; they trail the frontier by at least seven months. The pattern is consistent and not accidental: Chinese companies release open weights only after their closed versions have already been replaced by the next generation. Anyone relying on an open model for production should plan accordingly.

Methodology: intentional errors, lenient scoring. Researchers bought three furniture pieces, assembled them, and seeded realistic mistakes — a screw left loose, a board installed upside-down, a step skipped — then kept building on top of the error to make detection harder. Every model received an 80-step budget in an agentic sandbox; fewer than 5% of runs hit the limit. Scoring is multi-stage: correctly identifying "no error" counts as correct; when an error exists, the model must name the exact step and describe the problem. A judge model, GPT-5.6 Sol, evaluates the description and is instructed to be lenient: a correct step plus a general description earns full credit.

What's missing: a human baseline and judge bias. The researchers have not yet measured human performance on FAB, and they acknowledge many errors would stump anyone unfamiliar with the assembly. Using GPT-5.6 Sol as a lenient judge may inflate scores — a model that "roughly" identifies the issue gets full credit. Without a human baseline and without strict judging, GPT-6 Astra's 80% is a strong indicator but not definitive proof of field readiness.