These three things are the same story: a data shortage, a generative model, and a downstream task. US20210142177A1 (published May 13, 2021; inventors include well-known NVIDIA researchers) describes synthesizing data specifically to train other networks, tagged G06N 3/084 and G06N 3/04, the learning and architecture classes.
Connect the dots and the logic is almost circular in a productive way. You have a model that can generate plausible examples. You use it to manufacture training data. You train a second model on that data. The second model never sees a single real labeled example yet learns the task. When real data is the bottleneck, and by 2021 it increasingly was, this is how you route around it.
“Apparatuses, systems, and techniques are presented to generate data useful for further training of a neural network.”— U.S. Patent Application 2021/0142177 A1 source
Follow both the money and the IP: NVIDIA doesn't just sell the GPUs that train models; its research arm files heavily on the methods that make training more efficient, including how to feed models when human-labeled data is exhausted. Synthetic data is a way to keep the compute fed, which is squarely in NVIDIA's commercial interest.
This is also a quiet early marker of the data wall that dominates 2025 discourse. The worry that we'll run out of high-quality human text to train on has an old answer sitting in patents like this: generate more. The 2021 version targets vision and structured tasks, but the principle scales to language.
House caveat: synthetic data can encode the generator's blind spots, and a publication is a method claim, not proof it works at frontier scale. Still, the filing dates the idea precisely, by spring 2021, manufacturing training data was core enough to an AI-hardware leader to write down and claim.