Order matters: Distributional properties of speech to young children\n bootstraps learning of semantic representations

2018/02/02 by Philip A. Huebner, Huebner, Philip A, Jon Willits +1 · 1 citation
Computer Science · #Speech and dialogue systems #Neural Networks and Applications #Speech Recognition and Synthesis

paper · pdf · doi:10.48550/arxiv.1802.00768

Abstract

Some researchers claim that language acquisition is critically dependent on\nexperiencing linguistic input in order of increasing complexity. We set out to\ntest this hypothesis using a simple recurrent neural network (SRN) trained to\npredict word sequences in CHILDES, a 5-million-word corpus of speech directed\nto children. First, we demonstrated that age-ordered CHILDES exhibits a gradual\nincrease in linguistic complexity. Next, we compared the performance of two\ngroups of SRNs trained on CHILDES which had either been age-ordered or not.\nSpecifically, we assessed learning of grammatical and semantic structure and\nshowed that training on age-ordered input facilitates learning of semantic, but\nnot of sequential structure. We found that this advantage is eliminated when\nthe models were trained on input with utterance boundary information removed.\n

Citations

• Learning and development in neural networks: the importance of starting small

Cited by

• Do Syntactic Categories Help in Developmentally Motivated Curriculum Learning for Language Models?

Related

• Word Acquisition in Neural Language Models
• Language acquisition: do children and language models follow similar learning stages?
• How poor is the stimulus? Evaluating hierarchical generalization in neural networks trained on child-directed speech
• Modelling Child Learning and Parsing of Long-range Syntactic Dependencies
• Is Child-Directed Speech Effective Training Data for Language Models?