Google DeepMind researchers have proposed a new solution to the growing AI data shortage, a challenge that threatens to slow progress in developing advanced language models. As AI systems increasingly rely on massive datasets for training, the pool of usable text is shrinking—partly because much of the web’s content contains sensitive, inaccurate, or outdated information that labs avoid using. To address this, the team introduced a technique called Generative Data Refinement, or GDR. The method uses existing generative AI models to rewrite problematic data, removing or replacing harmful or obsolete content—such as Social Security numbers, outdated facts, or personal identifiers—while preserving the rest of the valuable information. Minqi Jiang, one of the lead researchers and now at Meta, explained that current practices often discard entire documents just because a single line contains unusable data. For example, a document with a single phone number or a reference to a former CEO might be thrown out entirely, wasting potentially useful content. GDR aims to fix that by isolating and cleaning the problematic parts, allowing the rest of the text to be retained and used for training. In a proof-of-concept study, the researchers applied GDR to over a million lines of code. They compared the results to human-labeled data and found that GDR significantly outperformed traditional filtering methods. According to Jiang, the approach “completely crushes the existing industry solutions” for data purification. The method also appears superior to synthetic data—artificially generated content used to train models—because synthetic data can degrade model performance and even lead to “model collapse,” where AI systems start repeating or hallucinating information. The GDR-generated data produced better training results than synthetic data created by large language models. The paper, which was developed over a year ago and only recently published, has not undergone peer review, a common practice in the tech industry where internal validation is standard. A Google DeepMind spokesperson did not comment on whether the technique is currently being used in Google’s Gemini models. While the research focused on text and code, Jiang believes GDR could be extended to other data types like video and audio. Despite the challenges of processing complex, multi-document personal data, the method holds promise for unlocking vast amounts of previously unusable information. With predictions suggesting that all human-generated text could be consumed by AI models between 2026 and 2032, innovations like GDR may be essential to sustaining the rapid pace of AI development. As video and audio continue to flood the internet at unprecedented rates, GDR could help turn these data streams into valuable training material for future AI systems.