2020/06/21 by Emory Hufbauer, Hana Khamfroush, Hufbauer, Emory +1
Computer Science · #Algorithms and Data Compression #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Natural Language Processing Techniques #Social and Information Networks (cs.SI) #Web Data Mining and Analysis #cs.IR #cs.SI
paper · pdf · doi:10.48550/arxiv.2006.11887
arxiv created 2020/06/21 · openalex publication_date 2020/06/21 · arxiv updated 2020/06/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Twitter, like many social media and data brokering companies, makes their data available through a search API (application programming interface). In addition to filtering results by date and location, researchers can search for tweets with specific content with a boolean text query, using \it AND, \it OR, and \it NOT operators to select the combinations of phrases which must, or must not, appear in matching tweets. This boolean text search system is not at all unique to Twitter and is found in many different contexts, including academic, legal, and medical databases, however it is stretched to its limits in Twitter's use case because of the relative volume and brevity of tweets. In addition, the semi-automated use of such systems was well studied under the topic of Information Retrieval during the 1980s and 1990s, however the study of such systems has greatly declined since that time. As such, we propose updated methods for automatically selecting and refining complex boolean search queries that can isolate relevant results with greater specificity and completeness. Furthermore, we present preliminary results of using an optimized query to collect a sample of traffic-incident-related tweets, along with the results of manually classifying and analyzing them.