When the Problem Is Simulation, Not Prediction
A category of problem gets approached with predictive models because that's what the field knows how to build. The failure mode is specific: right on average, wrong at the moments that matter most.
There is a category of problem that gets approached with predictive models because predictive models are what the field knows how to build, and the approach partially works, which is enough to prevent the underlying mismatch from becoming obvious. The failure mode is specific: the model performs well on average and fails badly at the moments when accuracy matters most. Not because the model is poorly built, but because the problem is a simulation problem, and a predictive model is the wrong architecture for a simulation problem.
What prediction and simulation are
A predictive model learns a function from inputs to outputs. Given a sufficiently large dataset of (input, output) pairs, it fits a mapping that generalizes to new inputs. The implicit assumption is that this mapping is stable, that the relationship between inputs and outputs does not change in ways the model can't anticipate. Demand forecasting as traditionally practiced is predictive: given historical sales data and some signals describing the current context, predict future sales.
Simulation is different in kind. A simulation models a system, a set of interacting components, each governed by behavioral rules, and observes what the system produces. The output is emergent: it arises from the interactions of the components, not from a fixed mapping between inputs and outputs. Individual component behavior is specified, but aggregate behavior is derived. You don't estimate "demand"; you run a simulation of the agents whose collective behavior produces demand.
The relevant question for any forecasting or prediction task is: is the thing I'm trying to predict a stable function of observable inputs, or is it an emergent property of interacting components whose behavior is conditional on each other?
Where the mismatch shows up
Demand forecasting is the clearest case I've worked in directly. The argument for treating it as a simulation problem is this: demand is not produced by a single consumer making an isolated decision. It is produced by thousands of consumers making decisions that are influenced by what other consumers are doing, what media is saying, what retailers are stocking, what competitors are pricing. These interactions produce emergent behavior: panic buying, trend adoption, sentiment cascades. No mapping from historical inputs to future values can capture them.
Predictive models handle this by including proxy variables for the social dynamics: consumer sentiment indices, social media engagement metrics, news event flags. These variables are measurements of the aggregate state of the system. They are not models of the system. When the social dynamics shift in a way that hasn't appeared in the training distribution, such as a new kind of viral event, a policy change that creates unusual purchasing behavior, or a sentiment cascade triggered by a novel combination of factors, the proxies don't capture the shift, and the model fails.
The same pattern appears in financial market forecasting, social network behavior prediction, epidemiological modeling, and supply chain disruption prediction. In each case, the phenomena being predicted are emergent properties of interacting agents, and predictive models trained on historical correlations work well when the system is behaving as it has behaved before and fail when the system is doing something structurally new.
The structural failure mode
Predictive models have a specific failure mode in this setting: they are right on average, which is enough to make them look like they're working, while being wrong precisely at the moments when accuracy is most consequential.
For demand forecasting: the model nails the routine weeks and misses the disruptions. The overall MAPE looks acceptable, 4%, 5%, sometimes even 3%, because most weeks are routine. But the weeks that drive inventory decisions are the disruption weeks, and those are exactly the weeks where the model fails, because those are the weeks where emergent social dynamics are producing demand patterns that don't appear in the training distribution.
The average accuracy metric obscures this. If 48 weeks are routine and 4 weeks are disruptions, and the model performs well on the 48 and badly on the 4, the average accuracy looks fine. The cost of the 4 bad weeks, excess inventory, missed sales, supply chain scrambles, is not captured in the MAPE.
What simulation adds
Simulation doesn't predict by extrapolating historical patterns. It derives predictions from the behavior of components modeled at a lower level of abstraction: individual agents, rather than aggregate statistics.
The advantage is that emergent behavior falls out of the simulation naturally. If you model thousands of consumers, each with behavioral rules that include "buying behavior is influenced by what peers are buying," sentiment cascades emerge from the simulation without being explicitly encoded. If you inject an external shock, such as a tariff announcement or a viral social media event, you can observe how the simulated system responds, because the response emerges from the behavioral rules of the components, not from a learned mapping that has to have seen this shock before.
The disadvantage is that simulation requires behavioral modeling at the component level, which is harder than training on aggregate data. You need to specify the behavioral rules of individual agents, and those rules are not always obvious. You also need validation: a simulation is only useful to the extent that the emergent behavior it produces matches the emergent behavior of the actual system, and validating this requires comparing simulation outputs to observed aggregate behavior.
That validation has to be more specific than "the simulation looks plausible." A useful simulation should reproduce known aggregate patterns, fail in interpretable ways when its assumptions are wrong, and respond sensibly to counterfactual shocks. If a demand simulation cannot reproduce past panic-buying dynamics when given the conditions that produced them, it has not earned the right to forecast the next disruption. Plausibility is cheap. Calibration against observed system behavior is the expensive part.
The practical question
The question to ask before choosing between a predictive model and a simulation is: are the failure modes I care about statistical or structural?
Statistical failure modes, such as noise, bias, or poor generalization to certain input subgroups, are addressed by better data, better model architecture, better regularization. These are improvements within the predictive modeling approach.
Structural failure modes are different. The model fails when the underlying system is doing something it hasn't done before, specifically when the failure is caused by emergent social dynamics or interacting components. These failures are not addressed by better data or better model architecture. They are addressed by moving to a different approach: one that models the system that generates the observed data, rather than the observed data directly.
Most applied ML projects don't go through this diagnosis. They start with a predictive model because that's what the workflow supports, and they treat the structural failures as data problems or architecture problems. This produces incremental improvement at the cost of never addressing the underlying issue.
The reverse mistake is possible too. Some problems are just prediction problems, and wrapping them in agent-based simulation adds complexity without adding explanatory power. If the system is stable, the causal structure is not changing, and the failure modes are mostly noise, bias, or sparse data, then a simulation is likely to become an elaborate way of producing a worse forecast. The shift only pays for itself when the thing that matters is interaction.
The structural failures in demand forecasting are predictable: they happen during disruptions, which is when social dynamics dominate. The question is whether those disruptions happen often enough, and cost enough when the model misses them, to justify the implementation cost. If the disruptions are rare and cheap, the predictive model is fine. If the disruptions are the scenarios that drive the highest-value decisions, simulation deserves serious consideration. The point is not that simulation is more sophisticated. The point is that sometimes the system, not the sequence, is the object you need to model.