Walt Whitman Computational Neuroscience Club

Start with the resources marked by an asterisk *

Textbook:

• Deep Learning with Python, Third Edition by Francois Chollet and Matthew Watson (Ch 1 through 4) *

• Deep Learning with Python, Second Edition by Francois Chollet

Basics of Neural Networks

• But what is a neural network? | Deep learning chapter 1 *

• Gradient descent, how neural networks learn | Deep learning chapter 2 *

• Backpropagation, intuitively | Deep L3 *

• Backpropagation calculus | DL4 *

Key Research Papers

• Review papers

  • “Deep Learning” by LeCun, Bengio, Hinton (2015) *

  • “The unreasonable effectiveness of deep learning in artificial intelligence” by Sejnowski (2020) *

• Foundational papers

  • “A Logical Calculus of the Ideas Immanent in Nervous Activity” McCulloch and Pitts (1943)

  • “The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain” by Rosenblatt (1958) *

  • “Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex” by Hubel and Wiesel (1962)

  • “Neocognitron: A Hierarchical Neural Network Capable of Visual Pattern Recognition” by Fukushima (1988)

  • “Learning representations by back-propagating errors” by Rumelhart et al (1986)  *

  • “Backpropagation applied to handwritten zip code recognition” by LeCun et al (1989) *

  • “Gradient-based learning applied to document recognition” by LeCun et al (1998)

  • “Deep Sparse Rectifier Neural Networks” by Glorot et al (2011) [ReLU] *

  • “ImageNet Classification with Deep Convolutional Neural Networks by Krizhevsky, Sutskever, Hinton” (2012) *

  • “Attention Is All You Need” by Vaswani et al (2017) [Transformers]

• Papers showing representational similarity between NNs and the brain 

  • “Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition" by Cadieu et al. (2014) *

  • “Performance-optimized hierarchical models predict neural responses in higher visual cortex” by Yamins et al (2014)

  • “Using goal-driven deep learning models to understand sensory cortex” by Yamins and DiCarlo (2016)

  • “Integrative Benchmarking to Advance Neurally Mechanistic Models of Human Intelligence” by Schrimpf et al (2020)

  • “The neural architecture of language: Integrative modeling converges on predictive processing” by Schrimpf et al (2021) *

• ANNs’ value for neuroscience

  • “A deep learning framework for neuroscience” by Richards et al (2019)

• Brain-inspired learning models

  • “Brain-inspired learning in artificial neural networks: a review” by Schmidgall et al (2023)

  • “Backpropagation and the brain” by Lillicrap et al (2020) 

  • “Towards Biologically Plausible Deep Learning” by Bengio et al (2016) 

  • “The Forward-Forward Algorithm” by Hinton (2022)

• AlphaGo / AlphaFold / IMO Gold

  • “AlphaGo’s Move 37 and Its Implications for AI-Supported Military Decision-Making” by Thomas W. Simpson (2024)

  • AlphaFold

    • "Highly accurate protein structure prediction with AlphaFold" by Jumper at al (2021) [paper]

    • "AlphaFold: a solution to a 50-year-old grand challenge in biology" by the AlphaFold team (2021) [blog post]

    • "‘The game has changed.' AI triumphs at solving protein structures" by Service (2021) [article]

    • “AlphaFold2 and its applications in the fields of biology and medicine” by Zhang et al (2023) [paper]

  • IMO Gold

    • Alexander Wei announcement [tweets]

    • Discussion with team [video]

• “Stochastic parrots” and “just statistics” argument

  • “Deep Learning: A Critical Appraisal” by Gary Marcus (2018)

  • “The False Promise of ChatGPT” by Chomsky (2023) [NYT opinion]

  • Emily Bender’s Lunch with the FT 

  • ”Climbing Towards NLU: On Meaning, Form, and Understanding in the Age of Data” by Bender and Koller (2020)

  • Melanie Mitchell: “Why AI is Harder Than We Think”

• Other

  • “Single cortical neurons as deep artificial neural networks” by Beniaguev et al (2021) *

  • Cover’s Theorem (1965) *

  • “Distinctive properties of biological neural networks and recent advances in bottom-up approaches toward a better biologically plausible neural network” by Jeon and Kim (2023)