MCCHE Precision Convergence Webinar Series with Michael J. Frank
Clustering and generalization of abstract structures in reinforcement learning and musicality
By Michael J. Frank
Edgar L. Marston Professor of Cognitive, Linguistic & Psychological Sciences at Brown University.
With High-Level Panel of Leaders in Science, Technology, On-the-Ground Action, and Policy
Humans are remarkably adept at generalizing knowledge between experiences in a way that can be difficult for computers. Previous computational models and data suggest that rather than learning about each individual context, humans build latent abstract structures and learn to link these structures to arbitrary contexts, facilitating generalization, but with a cost in efficiency of initial learning. In these models, task structures that are more popular across contexts are likely to be reused in new contexts. Neural signatures of such structure learning are predictive across individuals of the ability to transfer knowledge to new situations. However, these models predict that structures are either re-used as a whole or created from scratch, prohibiting the ability to generalize constituent parts of learned structures. This contrasts with ecological settings, where task structures can be decomposed into constituent parts and reused in a compositional fashion. Moreover in many situations people can transfer structures that they have learned to entirely new situations, by analogy, even when surface aspects of the transition and reward functions change. I will present novel computational models across levels (from neural networks to bayesian formulations) that address how agents and humans can learn and generalize such abstract and compositional structure. Throughout, I will give examples of how such computations can allow a musician to learn to compositionally transfer musical scales and rhythms within and across instruments. Discussion with panelists will follow on the similarity/dissimilarity between human and machine in such abstraction.
About the speaker
Michael J. Frank is Edgar L. Marston Professor of Cognitive, Linguistic & Psychological Sciences at Brown University. He directs the Center for Computational Brain Science within the Carney Institute for Brain Science. He received his PhD in Neuroscience and Psychology in 2004 at the University of Colorado, following undergraduate and master's degrees in electrical engineering. Frank’s work focuses primarily on theoretical models of frontostriatal circuits and their modulation by dopamine, especially their cognitive functions and implications for neurological and psychiatric disorders. The models are tested and refined with experiments across species, neural recording methods, and neuromodulation. Honors include the Troland Research Award from the National Academy of Sciences (2021), Kavli Fellow (2016), the Cognitive Neuroscience Society Young Investigator Award (2011), and the Janet T Spence Award for early career transformative contributions (Association for Psychological Science, 2010). Dr Frank is a senior editor for eLife.
About the series
The Precision Convergence series is launched to catalyze unique synergy between, on the one hand, novel partnerships across sciences, sectors and jurisdictions around targeted domains of real-world solutions, and on the other hand, a next generation convergence of AI with advanced research computing and other data and digital architectures such as PSC’s Bridges-2, and supporting data sharing frameworks such as HuBMAP, informing in a real time as possible the design, deployment and monitoring of solutions for adaptive real-world behaviour and context.
The Precision Convergence Webinar Series is co-hosted by The McGill Centre for the Convergence of Health and Economics (MCCHE) at McGill University and The Pittsburgh Supercomputing Center, a joint computational research centre between Carnegie Mellon University and the University of Pittsburgh.