Discussion about this post

User's avatar
Antony Van der Mude's avatar

Great article. Thanks for this.

I have worked in Machine Learning since the late 1970s. Bayes Theorem was involved in Adrian Walker and my work “On the Inference of Stochastic Regular Grammars”, DOI: 10.1016/S0019-9958(78)90106-7

But one should not be slavishly attached to a theory. In Machine Learning, ever since Horning’s 1969 work, it has been known that to get a good learning model you have to balance accuracy versus model simplicity or else you risk overfitting. I convinced Adrian that Bayes statistical theory was only one possibility, in that it measures simplicity in terms of the likelihood of selecting the model from a space of models. Instead, I suggested a simplicity measure in terms of the length of the representation. Adrian saw the wisdom in that.

I am working on a paper on information theory as it relates to the theory of evolution, and I had wondered why Elliot Sober used the Akaike Information Criterion instead of Bayes Theorem in his 2024 book “The Philosophy of Evolutionary Theory” DOI: 10.1017/9781009376037

You explained the reason why very nicely. Thanks.

I certainly remember the brouhaha around Marilyn’s article (recently widowed - R.I.P. Robert Jarvik). She wrote a weekly column in Parade magazine, which appeared as a flyer in the Sunday Newark Star-Ledger among many other newspapers. Reading her columns was one of the highlights of my Sundays in those days. Grand fun!

Expand full comment
Mike Win's avatar

Brilliant!

Expand full comment

No posts