Prediction is not enough: where predictive coding breaks

Ebenezer Gelo

Ebenezer Gelo

6 mins read

Predictive coding is one of the most successful theoretical frameworks in computational neuroscience. It explains a lot: hierarchical visual processing, repetition suppression, certain illusions, the basic shape of cortical microcircuits. The bolder claim, that prediction-error minimization is a universal principle of cognition, is a different proposition, and it does not survive contact with the problem of action. The framework can be extended to cover control, but the extension changes what the theory is doing in a way that its proponents tend to undersell.

What predictive coding gets right

The core idea is elegant. The brain maintains a generative model of its sensory input, runs that model forward to predict incoming signals, and updates the model based on the residual: the prediction error. Hierarchies of such predictors, with errors propagating up and predictions propagating down, give a clean account of how perception could work without anyone having to specify features by hand. Rao and Ballard (1999) made this concrete for visual cortex; the broader framework owes its current shape largely to Friston and collaborators.

For perception, this story is good. It predicts neural phenomena with the right qualitative shape, it gives a normative reason for hierarchical organization, and it connects naturally to variational inference in machine learning. Within its proper domain, the theory earns its keep.

The universal claim

The stronger version, sometimes called the free energy principle or active inference, says that all of cognition reduces to minimizing prediction error, where action is the special case in which the agent changes the world to match its predictions rather than updating predictions to match the world. Perception is inference over hidden states; action is inference over policies. Both fall out of one objective.

This is a real and interesting move. It makes the framework formally able to talk about behavior, planning, and exploration. It also creates the problem.

The dark room

If an agent's job is to minimize prediction error, the optimal strategy is to inhabit an environment where prediction is trivial. A dark, silent, motionless room. Nothing surprises you. Free energy is minimized. This is not a contrived counterexample, it is the most natural reading of the objective. Real organisms do not behave this way. They seek novelty, explore, take risks for uncertain gains. Something has to give.

The standard response is to add priors over preferred observations: the agent does not just predict, it predicts that it will be in states with food, social contact, warmth, and so on. Failure to occupy those states becomes prediction error. The dark room is no longer attractive because the agent's generative model says it should not be there.

This works, in the sense that the math goes through. It also concedes the original game. The framework no longer derives behavior from prediction-error minimization alone; it derives behavior from prediction-error minimization plus a hand-specified set of preferred outcomes. The preferences are doing the explanatory work. Calling them priors instead of utilities is a notational choice.

Where the gap shows up formally

The clean way to see the issue is to compare objectives. A predictive agent minimizes a divergence between its predictions and incoming observations. A control-theoretic agent maximizes expected return under a reward function. These objectives can be aligned: if the reward function and the preferred-observation prior are chosen to match, behavior coincides. They can also be anti-correlated. Consider any environment where reducing immediate prediction error requires actions that reduce long-term reward. Sitting still in a familiar location is highly predictable and often suboptimal. Exploring is unpredictable and often necessary. The agent that minimizes one-step prediction error will systematically under-explore.

Active inference handles this by importing the multi-step structure: minimize expected free energy over future trajectories, where the expectation includes an information-gain term that rewards reducing uncertainty about hidden states. Exploration falls out as epistemic value. The framework can now represent the right behavior. The question is whether it explains the right behavior, or whether it has been retrofitted to produce it. The information-gain term is mathematically equivalent to a specific kind of intrinsic reward, well-studied in reinforcement learning. Calling it free energy minimization rather than reward maximization is a choice about labels, not about mechanism.

What this leaves us with

I do not think predictive coding is wrong. I think it is a strong theory of perception that has been overextended. The pattern is familiar in computational neuroscience: a mechanism that works for one capacity (here, hierarchical perceptual inference) gets promoted to a unifying principle, and the promotion is achieved by adding terms to the objective until the framework can express anything. At that point the framework loses its claim to parsimony, which was its main attraction.

The diagnosis matters because alternatives exist and are doing real work. Reinforcement learning gives a principled treatment of action under uncertainty without needing to be reframed as inference. Model-based RL has its own account of how internal models should be built and used, with the relation between prediction and control made explicit rather than collapsed. Hierarchical RL is currently a better candidate than active inference for explaining how long-horizon plans get composed from short-horizon skills, because it has a clearer story about what the levels are for.

What I would want from a defender of strong predictive coding is a falsifiable prediction that distinguishes it from "RL with a generative world model and an information-gain bonus." If there is none, the substantive claim is that brains do model-based control with intrinsic motivation, which is interesting but not unique to the predictive coding tradition. If there is one, the field needs to know what it is.

The narrower claim, that perception is approximate Bayesian inference implemented hierarchically with prediction errors as the message-passing currency, is in good shape. The broader claim, that this is what cognition is, is doing less work than its presentation suggests. Prediction is necessary. It is not sufficient, and the sufficient version of the theory has absorbed enough additional structure that calling it predictive coding is closer to a branding decision than a substantive thesis.

This essay was written with AI assistance. All final arguments, edits, and responsibility are mine.