Vision is an essential domain of human intelligence. We recognize visual objects (e.g., a cat or a cup) quickly, often with no or little apparent effort. Current best models of human vision (deep neural networks) are trained in ways that differ from how humans actually learn from infancy and beyond. With support from the National Science Foundation (BCS 2122961), we ask:  How do infants represent the visual world around them, and how is this process shaped by the social-communicative (e.g., linguistic) context? Answering this question will give us fundamental insights into the perceptual and cognitive strengths and constraints that shape how infants understand and interact with the world. These insights will contribute to inform models of how human vision is achieved through evolution, development, and learning.

While we know that people use facial movements to infer affect, how those facial movements are extracted from visual input and represented in neural activity remains unclear. This is important because models of “facial emotion” understanding tend to presuppose facial movement representations as their input. That is, current computational models of “facial emotion” understanding do not take into account the visual mechanisms  – and associated uncertainty, computational trade-offs, and developmental trajectory – by which patterns of facial movements, also called “action units”, are extracted from visual input. However, learning to represent facial movements – especially across different models – poses a complex challenge to the developing brain, engaging a set of perceptual skills known as high-level vision. How do infants and young children represent facial movements? How may these representations facilitate or constraint infants’ and young children’s ability to learn from other people’s facial expressions?

Infants detect facial movements as early as birth, and perceptually categorize some facial expressions of affect as early as 5- to 6-months. However, little is known about what infants understand about these expressions. If infants understand facial expressions as affective, then models of affective cognition should account for their responses, and individual differences should reflect liability for atypical affective development. Studies in older children and adults have shown that the sensitivity of threat-related facial expressions is altered in disorders such as anxiety, suggesting that affective communication provides a window into broader aspects of affective functioning. What emotional or affective meaning do infants and young children derive from affective expressions? What are the origin and developmental, clinical implications of that knowledge?

Methodological limitations constraint the scientific questions that we can ask; conversely, methodological innovations open new avenues of inquiry. As part of a consortium of developmental scientists, we seek to integrate modern computational tools for the analysis and modeling of developmental data, such as multivariate pattern analysis (MVPA) of near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) data, to  ask new questions about how the developing brain represents complex stimuli.