French startup Kog promises 30x inference speedup on existing GPUs

Kog says it can extract far more inference performance from GPUs already deployed in data centers, without buying dedicated chips such as Cerebras. In May the company reached the front page of Hacker News with a tech preview that demonstrated 3,000 tokens per second per request (TPS) on AMD MI300X and Nvidia H200. The catch: the figure was achieved with a small model called Laneformer 2B, only 2 billion parameters, released as open source. CEO Gaël Delalleau claims the same approach will work on larger models, and that the belief GPUs are unsuitable for decoding is a myth, with the bandwidth of new generations just waiting to be exploited.

The first use case, according to Delalleau, is software engineers who wait hours for results. Early feedback from 200 concrete business leads pointed to software engineering as the initial use case. Veteran Claude Code users know the hours-long wait for results, and Anthropic itself charges a premium for Fast Mode. Kog targets customers who are deterred by those delays because they rely on AI pipelines for production tasks. There are also design partners building applications and games with a single prompt, for whom faster inference translates directly into revenue.

The gap between demo and reality is that potential customers are not ready to fine-tune small models. Since launch Kog has focused on accelerating large models to meet the demand it identified. Jumping from 2 billion parameters to models of tens or hundreds of billions is not trivial; memory, bandwidth and KV-Cache management behave completely differently at that scale. Delalleau is convinced the physics of the GPU makes it possible, but for now the 30× figure is measured only under lab conditions on a toy model.

Delalleau’s background is unusual. He studied solid-state physics at École Polytechnique in France, then moved to offensive security, white-hat hacking, reaching the finals of the DEFCON CTF four times. He says that taught him to do assembly-level reverse engineering, understand how things work under the hood, and repurpose them for unintended uses. In his scientific approach he looks for “the physics laws of the GPU” to maximize them. The downside is that each new GPU requires weeks or months of deep manual work.

Kog is not alone in the idea that software optimization can squeeze more out of hardware. ZML, also French, released hardware-agnostic software that outperforms Nvidia’s CUDA for fast inference on competing chips. Delalleau says Kog is closer to the Hazy Research lab at Stanford University, with an even deeper focus on GPU acceleration. The seed round was led by Varsity VC’s Kamel Zeroual, former partner of Delalleau in his first startup Stribe (TechCrunch50 2009 alumnus), the only link between the two ventures.