2016/02/22 by Ethan Fast, W. R. McGrath, William McGrath +3 · 1 voice
Computer Science · Social Sciences · #Citizen science #Computer science #Context-Aware Activity Recognition Systems #Data science #Field (mathematics) #Human Mobility and Location-Based Analysis #Human–computer interaction #Innovative Human-Technology Interaction #Interrupt #Knowledge base #Phone #Selfie #Set (abstract data type) #Software deployment #Telecommunications #Vision #Wearable computer #Wearable technology #World Wide Web #cs.AI #cs.HC #cs.IR
paper · pdf · doi:10.1145/2858036.2858528
CHI: ACM Conference on Human Factors in Computing Systems 2016
arxiv published 2016/02/22 · arxiv created 2016/02/25 · arxiv updated 2016/02/25 · openalex publication_date 2016/05/05 · openalex created_date 2016/06/24 · openalex updated_date 2026/08/05
From smart homes that prepare coffee when we wake, to phones that know not to interrupt us during important conversations, our collective visions of HCI imagine a future in which computers understand a broad range of human behaviors. Today our systems fall short of these visions, however, because this range of behaviors is too large for designers or programmers to capture manually. In this paper, we instead demonstrate it is possible to mine a broad knowledge base of human behavior by analyzing more than one billion words of modern fiction. Our resulting knowledge base, Augur, trains vector models that can predict many thousands of user activities from surrounding objects in modern contexts: for example, whether a user may be eating food, meeting with a friend, or taking a selfie. Augur uses these predictions to identify actions that people commonly take on objects in the world and estimate a user's future activities given their current situation. We demonstrate Augur-powered, activity-based systems such as a phone that silences itself when the odds of you answering it are low, and a dynamic music player that adjusts to your present activity. A field deployment of an Augur-powered wearable camera resulted in 96% recall and 71% precision on its unsupervised predictions of common daily activities. A second evaluation where human judges rated the system's predictions over a broad set of input images found that 94% were rated sensible.