A note from after the Gaussian Process Summer School
Last month I went to the Gaussian Process Summer School 2020. In one of the talks about deep Gaussian processes, which are compositions of Gaussian processes analogous to deep neural networks, Carl Henrik Ek claimed that the models we use should only be hierarchical if our knowledge is truly hierarchical1. He also suggested that perhaps all knowledge is hierarchical in some sense.
This made me think about Fermi estimation (which involves breaking a larger problem down into smaller parts about which we have some, even weak, prior information about) as analogous to hierarchical modelling. However, in Fermi estimation usually point estimates are used rather than a more Bayesian approach in which uncertainty is propagated through the calculation. Perhaps this was the approach Fermi used to arrive at the Fermi paradox based on something like the Drake equation. Sandberg, Drexler, and Ord (2018) find that the paradox apparently disappears if one takes a more Bayesian (hierarchical modelling) approach to Fermi estimation.
This seems a provocative statement, in that it’s unclear that the knowledge hierarchy neural networks learn corresponds to the hierarchical knowledge we possess.↩︎
For attribution, please cite this work as
Howes (2020, Oct. 28). Adam Howes: Fermi estimation as hierarchical modelling. Retrieved from https://athowes.github.io/posts/2020-10-28-fermi-estimation-as-hierarchical-modelling/
BibTeX citation
@misc{howes2020fermi,
author = {Howes, Adam},
title = {Adam Howes: Fermi estimation as hierarchical modelling},
url = {https://athowes.github.io/posts/2020-10-28-fermi-estimation-as-hierarchical-modelling/},
year = {2020}
}