Essays, notes, and experiments.
Manipulation Is Intelligence in Contact
Acrobatics is smooth dynamics under your own authority; manipulation is non-smooth dynamics through a contact you can barely observe...
Why gradient descent works at all (and when it shouldn’t)
GD's success in non-convex landscapes is partially explained by the implicit bias of overparameterized training toward specific minima, but this bias is only a feature when it points at solutions that generalize, and the regime where most useful neural network behavior actually happens is still poorly characterized...
The myth of 'understanding' in neural networks
Standard benchmarks cannot distinguish semantic understanding from compression-and-interpolation on a learned manifold, so until we measure invariance under shifts that should be irrelevant to the target abstraction, the parsimonious description of model competence is the latter...
Prediction is not enough: where predictive coding breaks
Predictive coding is a strong theory of perception that becomes a universal theory of cognition only by importing utility as a prior over preferred observations, at which point the explanatory work is being done by the smuggled preferences and the framework's claim to parsimony is gone...