“Weaving together theoretical physics, machine learning and neuroscience: a tale of neurons, atoms and photons in the service of computation”
We are witnessing an exciting interplay between physics, computation and neurobiology that spans in multiple directions. In one direction we can use the power of complex systems analysis, developed in theoretical physics and applied mathematics, to elucidate design principles governing how neural networks, both biological and artificial, can learn and function. In another direction, we can exploit novel physics to instantiate and analyze new kinds of quantum neuromorphic computers built using atomic spins and photons. We will give several vignettes in both directions, including:
(1) deriving the detailed structure of the primate retina from first principles by developing optimal neural networks for processing natural movies
(2) using dynamic mean field theory to understand and optimize the training of deep neural networks used in machine learning
(3) understanding the geometry and dynamics of high dimensional optimization in the classical limit of a dissipative many-body quantum optimizer comprised of interacting photons
CLICK HERE TO JOIN THE ZOOM MEETING
Surya GanguliAssociate Professor of Applied Physics, and by courtesy, of Neurobiology, of Electrical Engineering, and of Computer Science, Stanford University; Faculty Associate Director, Stanford HAI
Click here for recordings of past colloquia
References:
Y. Bahri, J. Kadmon, J. Pennington, S. Schoenholz, J. Sohl-Dickstein, and S. Ganguli, Statistical mechanics of deep learning, Annual Reviews of Condensed Matter Physics, 2020.
M. Advani, S. Lahiri and S. Ganguli, Statistical mechanics of complex neural systems and high dimensional data, Journal of Statistical Mechanics Theory and Experiment (2013), P03014.
S. Deny, J. Lindsey, S. Ganguli, S. Ocko, The emergence of multiple retinal cell types through efficient coding of natural movies, Neural Information Processing Systems (NeurIPS) 2018.
B. Poole, S. Lahiri, M. Raghu, J. Sohl-Dickstein, and S. Ganguli, Exponential expressivity in deep neural networks through transient chaos, Neural Information Processing Systems (NIPS) 2016. J. Pennington, S. Schloenholz, and S. Ganguli, Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice, Neural Information Processing Systems (NIPS) 2017. Y. Yamamoto, T. Leleu, S. Ganguli and H. Mabuchi, Coherent Ising Machines: quantum optics and neural network perspectives, Applied Physics Letters 2020.
B.P. Marsh, Y, Guo, R.M. Kroeze, S. Gopalakrishnan, S. Ganguli, J. Keeling, B.L. Lev, Enhancing associative memory recall and storage capacity using confocal cavity QED, Physical Review X, 2020.
