GEN-1.5 robot learns tasks without training, showing first few-shot capability at scale

The new robotic foundation model GEN-1.5 demonstrates for the first time learning of physical skills from a single example or a few examples, without gradient updates, without fine-tuning, and in real time. Researchers define the achievement as the “GPT-3 of robotics”: just as GPT-3 showed that in-context learning works at scale for language, GEN-1.5 shows that a physical counterpart is possible. The tasks remain simple and short-horizon, but this is the first model to succeed without restrictions on object types, task types, or sensing modalities.

GEN-1.5 is a large multimodal model that processes video (a 30-second memory window), additional sensors, language and proprioception, and outputs action trajectories at 100 Hz. The combined input enables it to link a visual demonstration to precise motor execution without a separate training phase for each task. Researchers call the use of sensorimotor examples within the context window “physical prompting”, a direct analogy to textual prompting in language models.

The core capability is one-shot learning: a demonstration lasting 3 to 12 seconds suffices for the model to perform the new task within seconds. Beyond that, the model exhibits compositional generalization: when given two different physical prompts in sequence, it chains them into a single longer-horizon behavior. Such a capability has not been observed before in robot models at this scale, and it suggests an understanding of task structure rather than mere motion imitation.

Two additional capabilities stand out: zero-shot transfer from simulation to the real world (sim-to-real), where a demonstration recorded in a simulated environment serves as a valid physical prompt for a real robot even though the pretraining did not include simulation data; and human-to-robot imitation, in which a person demonstrates a motion in front of the robot’s cameras and the model reproduces the action with the robot’s arm. In both cases no explicit mapping between human and robot bodies is required; the model learns the correspondence from the data.

In addition to training-free learning, the model also supports few-shot adaptation via gradient descent when higher precision is needed. Researchers emphasize that the current tasks are simple, short-horizon, and far from general autonomy. Nevertheless, this is the first model to combine one-shot, few-shot, zero-shot, compositional generalization, sim-to-real, and human-to-robot imitation without domain restrictions. If GPT-3 marked the shift from specialized models to a general language model, GEN-1.5 marks the robotic analogue: a model that approaches the formulation “approach a robot, give it a task, and it executes”.