Synthetic Data Isn't New. The Scale Is.
The mechanism underlying synthetic data for fine-tuning is not new. What has changed is scale and accessibility. Where it works, where it breaks, and what the defensible production pattern actually looks like.
Generate 10,000 synthetic question-answer pairs about your product docs, fine-tune on them, and the model will sound like it is drawing on those docs. Factual accuracy is not guaranteed. What you get is confident wrong answers in the domain where you trained it to sound authoritative. This is knowledge injection, the most common misuse of synthetic data, and it fails for the same structural reason fine-tuning on documentation fails: the model is not storing facts in a lookup table, it is shifting its output distribution toward the patterns in the training data.
The teacher that wrote the set makes mistakes. It confuses similar entities, misremembers specific values, and fills gaps with plausible fabrications. Those errors sit in the dataset and the student learns them next to the correct rows. Unlike a database, fine-tuned weights cannot be opened to point at the facts that landed wrong. Retrieval is the right tool for facts.
An old mechanism with new access
Hinton described knowledge distillation in 2015: a stronger model, the teacher, generates soft labels for a training set, and a smaller model, the student, trains on those labels rather than on hard ground truth. The student learns to approximate the teacher's output distribution. That distribution carries more information than a binary correct/incorrect label, because the full distribution is a richer target than a one-hot. The student is compressed, and it is not degraded the way simple compression would degrade it.
What changed is scale and access. Capable teachers sit behind an API, generating a dataset is a script, and there are now cases where synthetic data outperforms human-labeled data on a bounded task. Most pipelines still get scored on row count, with the teacher that wrote the rows treated as a detail.
Where it works
Format and style normalization is the clean case. You have a base model with decent general capability and unreliable formatting: required JSON sometimes, explanation text wrapped around it sometimes, field names that drift. The correct output for a given input is unambiguous and the model just does not produce it reliably. A teacher configured to emit the format every time can generate (input, correct output) pairs cheaply.
Say you have 500 labeled examples on a low-resource task (500, 5,000 and the 90% and 60% below are round illustrations, not figures from a named run), which is not enough for reliable fine-tuning. A teacher can paraphrase the inputs, reformulate the questions, keep the alternative phrasings, and leave the label distribution in place. The resulting 5,000 examples are not 5,000 human labels, but they are usually better than the 500 you started with, especially when the hard part is input diversity rather than label accuracy.
Reasoning traces are the newer application. Rather than labeling final outputs, a teacher writes step-by-step chains for a set of problems and the student is trained to produce chains of the same shape. The student is trained on the procedure, and that structure is what is supposed to transfer to novel problems. It is the approach behind several recent small-model results compared at a given parameter count.
In all three the synthetic data teaches behavior: how to format, how to vary an input without moving the label, how to expose a reasoning structure, how to imitate a stronger model's procedure on a bounded task. Factual authority is where the same mechanism turns fragile.
Student accuracy against teacher accuracy
A student's capability on the task the synthetic data covers is bounded by the teacher's capability on that task. A teacher with systematic errors, a misunderstood class of reasoning, or a blind spot in a domain passes those on. On common reasoning, general language, and code in popular languages, a smaller student can approach the teacher. On niche technical domains, precise numerical reasoning, or knowledge the teacher barely saw, synthetic data amplifies the teacher's failures rather than compensating for them.
A teacher at 90% on your task can produce a student that approaches 90%. A teacher at 60% produces a student that inherits approximately 60%, and does it with high confidence, which is worse than 60% with uncertainty. Later papers claim students can exceed teachers on some verifiable tasks; this essay does not measure that.
Diversity
A single teacher produces a single style, a single family of mistakes, and a single distribution over variations, and all of it transfers into the set and then the student. A human-labeled set gets diversity from different linguistic styles, different error patterns, and different ways of handling edge cases. That diversity is noise, and it is also a regularizer: it forces the model to generalize across human variation instead of overfitting one annotator's habits.
The usual mitigations are generating with multiple teachers (different checkpoints, temperatures, few-shot examples), injecting variation into the prompts, and mixing synthetic rows with human labels where you have them. Mixing is usually the one that works. Synthetic data augmenting a small human set beats synthetic data alone.
Verification
Humans can label things they do not know how to generate. A rater can tell whether code is correct, whether a translation is accurate, whether a summary is faithful, even when they could not have written the output, and that asymmetry between generation and verification is why human evaluation is hard to replace. The teacher does not have the same gap. It is typically as good at verifying as it is at generating and makes similar errors in both directions, so if it cannot reliably produce a correct solution it also cannot reliably say whether a proposed one is correct.
Code execution, mathematical proofs, and structured data with integrity constraints can be checked independently. Judgment tasks cannot, and a fully automated pipeline stops there. Synthetic data generated without an independent check (a different model, a symbolic checker, a human pass on a sample) will include errors that no downstream filter sees.
The production pattern that holds up generates, filters, verifies, samples, and only then trains; the other pattern generates a million examples and trains on them. The filter can be a symbolic checker or a second model with different failure modes, and a human review of a stratified sample shows the error distribution before the student learns it as signal.