2026/04/17 by Andy Bonnetto, Sepideh Mamooler, Chengkun Li +1 · 1 voice
Biochemistry, Genetics and Molecular Biology · Psychology · #Zebrafish Biomedical Research Applications #Animal Vocal Communication and Behavior #Primate Behavior and Ecology
paper · doi:10.1016/j.conb.2026.103201
Understanding animal behavior requires moving beyond specialized assays toward comprehensive assessment. We propose that behavior emerges from three fundamental principles: it is embodied, situated, and hierarchical. While this has been appreciated for a long time, we highlight that recent advances in physics engines, computer vision, and multimodal language models (MLMs) now make quantitative measurement of these aspects feasible. Computer vision tools can reconstruct both animals and environmental contexts, and biomechanics simulators allow us to gain insight into the bodily expression of behavior. Unlike traditional approaches that classify behaviors into fixed categories, MLMs can leverage the flexibility of natural language to recognize semantically meaningful behavioral descriptions. These models can integrate multiple modalities, including pose, audio, and environmental context. This integrated framework enables analysis across different levels, opening exciting possibilities for understanding the structure of behavior as well as the underlying neural mechanisms. • Both embodiment and situatedness enable and constrain behavior, which is hierarchical. • Multi-modal language models are ideally suited for analyzing hierarchical behavior. • Behavior is typically measured at a particular level but one can now integrate these perspectives.