From statistics to deep learning and beyond

Overview

20th century statistics established a tradition of quantitative modeling under restrictive assumptions that yielded mathematically tractable problems but were not always realistic in the contexts they were applied. Among the most important aspects of reality assumed away were recursive structure and prevalence of rare events, both crucial aspects of complex systems. Deep learning is starting to revive, in an ad-hoc way, fixed patterns of recursive structure in a form inspired by hierarchies of features in animal visual systems, and the field is in the very earliest stages of branching out to consider the general case of unbounded recursion in such forms as DeepMind's Neural Turing Machine.We'llcritically examine the assumptions probability theory and statistics are built on, with analogies to parallel questions in classical vs. non-classical logics and an alternative formulation of quantum uncertainty. We'll end with thoughts on novel approaches to modelling general recursive structure.

Anthony Di Franco

works at the intersections of complex adaptive systems, economics, and computing. He is a board member of the East Bay biology hackerspace, Counter Culture Labs (https://counterculturelabs.org/), where he teaches and organizes events on topics at the intersection of computation and biology. He is also the author of the transgressive historical wuxia manhua,Three Sovereigns (http://3sovereigns.com/).