Skip to main content

NeuroAI & Visual Representations

Deep neural networks can reproduce many features of visual cortex, but they also differ from biological vision in important ways. We use those similarities and differences in both directions: to build better models of the visual system and to ask what aspects of biological vision current AI models fail to capture.

What We Study

We develop biologically constrained neural networks and test which architectural, learning, and input constraints are needed to reproduce properties of visual cortex. We also study the internal representations learned by artificial networks: how closely they match neural data, whether that alignment predicts behavior or robustness, and whether useful representational structure can be transferred between models.

Visual experience is another important part of the problem. We study how representations change when sensory input is altered or absent, including how visual cortex is organized in people who are blind. These cases provide a strong test of models that are usually developed only for typical vision.

Current Directions

  • Model–brain alignment. Understanding what neural-alignment metrics capture and when similar representations do not imply similar computation.
  • Biologically constrained networks. Testing which architectural and learning constraints are needed to reproduce properties of visual cortex.
  • Representation learning. Studying how internal representational structure affects behavior, robustness, and transfer between models.
  • Digital twins. Probing predictive models of visual cortex to test hypotheses about neural representations.
  • Plasticity and altered visual input. Studying how visual experience shapes cortical representations, including in blindness.

Team


Publications

We show that apparent differences in what neural activity represents can reverse when language inputs are better matched, meaning that model design can strongly shape representational conclusions.

We introduce a multi-level evaluation framework for digital twins of mouse V1 that links neural-prediction accuracy to probe decodability, latent-unit tuning, and hidden-population geometry.

We introduce BIRD (Behavior Induction via Representation-structure Distillation), a flexible framework for transferring aligned behavior by matching the internal representation structure of a student model to that of a teacher.

Spotlight Talk

We present a series of analyses on the shared representations between evoked neural activity in the primary visual cortex of a blind human with an intracortical visual prosthesis, and latent visual representations computed in deep neural networks.

We systematically incorporated neuroscience-derived architectural components into CNNs to identify a set of mechanisms and architectures that comprehensively explain neural activity in V1.

We present a biophysically detailed in silico model of retinal degeneration that simulates the network-level response to both light and electrical stimulation as a function of disease progression.

Brains face the fundamental challenge of extracting relevant information from high-dimensional external stimuli in order to form the neural basis that can guide an organism’s behavior and its interaction with the world. One potential approach to addressing this challenge is to reduce the number of variables required to represent a particular …

To investigate the effect of axonal stimulation on the retinal response, we developed a computational model of a small population of morphologically and biophysically detailed retinal ganglion cells, and simulated their response to epiretinal electrical stimulation. We found that activation thresholds of ganglion cell somas and axons varied …

Using a dimensionality reduction technique known as non-negative matrix factorization, we found that a variety of medial superior temporal (MSTd) neural response properties could be derived from MT-like input features. The responses that emerge from this technique, such as 3D translation and rotation selectivity, spiral tuning, and heading …

We present a two-stage model of visual area MT that we believe to be the first large-scale spiking network to demonstrate pattern direction selectivity. In this model, component-direction-selective (CDS) cells in MT linearly combine inputs from V1 cells that have spatiotemporal receptive fields according to the motion energy model of Simoncelli and …

Back to all research areas