Adapting to LLMs: How Insiders and Outsiders Reshape Scientific Knowledge Production

2025/05/19 by Huimin Xu, Xu, Huimin, Houjiang Liu +5
Computer Science · Social Sciences · #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Information Systems Theories and Implementation #Open Source Software Innovations #Research Data Management Practices

paper · pdf · doi:10.48550/arxiv.2505.12666

openalex publication_date 2025/05/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

CSCW has long examined how emerging technologies reshape the ways researchers collaborate and produce knowledge, with scientific knowledge production as a central area of focus. As AI becomes increasingly integrated into scientific research, understanding how researchers adapt to it reveals timely opportunities for CSCW research -- particularly in supporting new forms of collaboration, knowledge practices, and infrastructure in AI-driven science. This study quantifies LLM impacts on scientific knowledge production based on an evaluation workflow that combines an insider-outsider perspective with a knowledge production framework. Our findings reveal how LLMs catalyze both innovation and reorganization in scientific communities, offering insights into the broader transformation of knowledge production in the age of generative AI and sheds light on new research opportunities in CSCW.

Citations

• A Systematic Literature Review of Infrastructure Studies in SIGCHI
• How Scientists Use Large Language Models to Program
• How are researchers using AI? Survey reveals pros and cons for science
• Cocoa: Co-Planning and Co-Execution with AI Agents
• OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs
• IdeaSynth: Iterative Research Idea Development Through Evolving and Composing Idea Facets with Literature-Grounded Feedback
• Interpreting Multi-band Galaxy Observations with Large Language Model-Based Agents
• SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning
• Mapping the Increasing Use of LLMs in Scientific Papers
• "I'm categorizing LLM as a productivity tool": Examining ethics of LLM use in HCI research practices
• PAPERCLIP: Associating Astronomical Observations and Natural Language with Multi-Modal Models
• Large language models surpass human experts in predicting neuroscience results
• (A)I Am Not a Lawyer, But...: Engaging Legal Experts towards Responsible LLM Policies for Legal Advice
• Understanding Practices around Computational News Discovery Tools in the Domain of Science Journalism
• Exploring the psychology of LLMs' Moral and Legal Reasoning
• Embedding Democratic Values into Social Media AIs via Societal Objective Functions
• Using an LLM to Help With Code Understanding
• All That's Happening behind the Scenes: Putting the Spotlight on Volunteer Moderator Labor in Reddit
• OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts
• Adaptability and the Pivot Penalty in Science and Technology
• Insiders and Outsiders in Research on Machine Learning and Society
• Language Models are Few-Shot Learners
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• Leadership for organizational adaptability: A theoretical synthesis and integrative framework
• The specificity of the scientific field and the social conditions of the progress of reason

Related

• Evolving Roles of LLMs in Scientific Innovation: Assistant, Collaborator, Scientist, and Evaluator
• Interesting Scientific Idea Generation using Knowledge Graphs and LLMs: Evaluations with 100 Research Group Leaders
• Why LLMs Aren't Scientists Yet: Lessons from Four Autonomous Research Attempts
• Drivers and Barriers of AI Adoption and Use in Scientific Research
• A Survey of AI Scientists