Grounded language acquisition through the eyes and ears of a single child

2024/02/01 by Wai Keen Vong, Wentao Wang, A. Emin Orhan +1 · 1 voice · 3 citations
Arts and Humanities · Psychology · #Hearing Impairment and Communication #Language Development and Disorders #Language, Discourse, Communication Strategies

paper · doi:10.1126/science.adi1374

openalex publication_date 2024/02/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/23

Abstract

Starting around 6 to 9 months of age, children begin acquiring their first words, linking spoken words to their visual counterparts. How much of this knowledge is learnable from sensory input with relatively generic learning mechanisms, and how much requires stronger inductive biases? Using longitudinal head-mounted camera recordings from one child aged 6 to 25 months, we trained a relatively generic neural network on 61 hours of correlated visual-linguistic data streams, learning feature-based representations and cross-modal associations. Our model acquires many word-referent mappings present in the child's everyday experience, enables zero-shot generalization to new visual referents, and aligns its visual and linguistic conceptual systems. These results show how critical aspects of grounded word meaning are learnable through joint representation and associative learning from one child's input.

Citations

• Ontological categories guide young children's inductions of word meaning: Object terms and substance terms
• Finding Structure in Time
• Conceptual distinctiveness supports detailed visual long-term memory for real-world objects.
• Rapid Word Learning Under Uncertainty via Cross-Situational Statistics
• Variability, negative evidence, and the acquisition of verb argument constructions
• Wordbank: an open repository for developmental vocabulary data
• Mapping the Early Language Environment Using All-Day Recordings and Automated Analysis
• Emerging Properties in Self-Supervised Vision Transformers
• Flamingo: a Visual Language Model for Few-Shot Learning
• Exploiting Generative AI to Scale up Intelligent Tutoring Systems

Cited by

• Are Language Models Models?
• Emulating Human-like Adaptive Vision for Efficient and Flexible Machine Visual Perception
• Humans Learn Language from Situated Communicative Interactions. What about Machines?
• The Structural Sources of Verb Meaning Revisited: Large Language Models Display Syntactic Bootstrapping

Discussions

• Vong. W. et al. (2024). Grounded language acquisition through the eyes and ears of a single child. Science doi.org/10.1126/scie... [bsky, 2 points, 0 comments]

Related

• On the robustness of modeling grounded word learning through a child's egocentric input
• Visual Grounding Helps Learn Word Meanings in Low-Data Regimes
• Assessing the alignment between infants' visual and linguistic experience using multimodal language models
• A Computational Model of Early Word Learning from the Infant's Point of View
• Can phones, syllables, and words emerge as side-products of cross-situational audiovisual learning? - A computational investigation