2024/05/31 by Jack Bunyan, Bunyan, Jack, Seth Bullock +3
Computer Science · #Adaptation and Self-Organizing Systems (nlin.AO) #Computation and Language (cs.CL) #FOS: Biological sciences #FOS: Computer and information sciences #FOS: Physical sciences #Populations and Evolution (q-bio.PE) #Speech Recognition and Synthesis #Speech and dialogue systems
paper · pdf · doi:10.48550/arxiv.2405.20818
openalex publication_date 2024/05/31 · openalex created_date 2024/06/04 · openalex updated_date 2026/07/28
The iterated learning model is an agent model which simulates the transmission of of language from generation to generation. It is used to study how the language adapts to pressures imposed by transmission. In each iteration, a language tutor exposes a naïve pupil to a limited training set of utterances, each pairing a random meaning with the signal that conveys it. Then the pupil becomes a tutor for a new naïve pupil in the next iteration. The transmission bottleneck ensures that tutors must generalize beyond the training set that they experienced. Repeated cycles of learning and generalization can result in a language that is expressive, compositional and stable. Previously, the agents in the iterated learning model mapped signals to meanings using an artificial neural network but relied on an unrealistic and computationally expensive process of obversion to map meanings to signals. Here, both maps are neural networks, trained separately through supervised learning and together through unsupervised learning in the form of an autoencoder. This avoids the computational burden entailed in obversion and introduces a mixture of supervised and unsupervised learning as observed during language learning in children. The new model demonstrates a linear relationship between the dimensionality of meaning-signal space and effective bottleneck size and suggests that internal reflection on potential utterances is important in language learning and evolution.