Amazon buys rare books, strips their covers and scans them for model training

A 404 Media investigation revealed that Amazon is buying large quantities of rare books, removing their covers and scanning them at a facility in Las Vegas marked by a dinosaur clutching a book in its claws. A journalist from the site attached a tracking device to one of the books and followed it to the facility, which is called VGT3.

a new source of training data

Large language models (LLM) have already consumed most of the text available on the web, and in the case of Anthropic, also pirated books obtained illegally. Rare books—especially those out of print or not available online—have become a sought-after source of new data. The main advantage is that any text published before 2022 is guaranteed not to have been written by a language model.

risk of model collapse

When models are trained on text generated by other models, they become vulnerable to “model collapse,” a phenomenon where output quality degrades after repeated ingestion of synthetic content. Books from the pre-AI era therefore provide a protective layer: they are “clean” of fingerprints from earlier models and preserve linguistic variety that is hard to reproduce.

amazon's response and broader context

In response to the 404 Media inquiry, Amazon said it “purchases books through commercial channels to improve the products and services customers use.” The company did not comment specifically on the removal of covers, the value attributed to rare books, or whether the scanning includes works still protected by copyright. The Las Vegas facility operates out of public view, and the dinosaur-with-a-book logo has become an unofficial identifier for the operation.