Ask ChatGPT: Caveats and Mitigations for Individual Users of AI Chatbots
2025/08/14 by Wang, Chengen, Kantarcioglu, Murat
#Computers and Society (cs.CY) #FOS: Computer and information sciences
paper · doi:10.48550/arxiv.2508.10272
Abstract
As ChatGPT and other Large Language Model (LLM)-based AI chatbots become increasingly integrated into individuals' daily lives, important research questions arise. What concerns and risks do these systems pose for individual users? What potential harms might they cause, and how can these be mitigated? In this work, we review recent literature and reports, and conduct a comprehensive investigation into these questions. We begin by explaining how LLM-based AI chatbots work, providing essential background to help readers understand chatbots' inherent limitations. We then identify a range of risks associated with individual use of these chatbots, including hallucinations, intrinsic biases, sycophantic behavior, cognitive decline from overreliance, social isolation, and privacy leakage. Finally, we propose several key mitigation strategies to address these concerns. Our goal is to raise awareness of the potential downsides of AI chatbot use, and to empower users to enhance, rather than diminish, human intelligence, to enrich, rather than compromise, daily life.
Citations
- User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policies
- When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models
- Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task
- AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges
- The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers
- Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers.
- Reinforcement Learning from Human Feedback
- A Review of DeepSeek Models' Key Innovative Techniques
- How much energy will AI really consume? The good, the bad and the unknown
- Medical Hallucinations in Foundation Models and Their Impact on Healthcare
- A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy
- AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking
- The Illusion of Empathy: How AI Chatbots Shape Conversation Perception
- Are lonely youngsters turning to chatbots for companionship? The relationship between chatbot usage and social connectedness in Danish high-school students
- The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships
- AI generates covertly racist decisions about people based on their dialect
- Does using artificial intelligence assistance accelerate skill decay and hinder skill development without performers’ awareness?
- Reinforcement Learning from Human Feedback: Whose Culture, Whose Values, Whose Perspectives?
- On the Algorithmic Bias of Aligning Large Language Models with RLHF: Preference Collapse and Matching Regularization
- Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?
- From tools to threats: a reflection on the impact of artificial-intelligence chatbots on cognitive health
- Security and Privacy Challenges of Large Language Models: A Survey
- Security and Privacy Challenges of Large Language Models: A Survey
- Hallucination is Inevitable: An Innate Limitation of Large Language Models
- A Survey of Reinforcement Learning from Human Feedback
- A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
- AI Supported Degradation of the Self Concept: A Theoretical Framework Grounded in Established Cognitive and Computational Mechanisms
- Chain-of-Verification Reduces Hallucination in Large Language Models
- DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
- Bias and Fairness in Large Language Models: A Survey
- Bias and Fairness in Large Language Models: A Survey
- AI Deception: A Survey of Examples, Risks, and Potential Solutions
- Gender bias and stereotypes in Large Language Models
- More human than human: measuring ChatGPT political bias
- "According to ...": Prompting Language Models Improves Quoting from Pre-Training Data
- GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models
- Protecting Society from AI Misuse: When are Restrictions on Capabilities Warranted?
- Will we run out of data? Limits of LLM scaling based on human-generated data
- Large Language Models are Zero-Shot Reasoners
- BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
- Scaling Laws for Neural Language Models
- Attention Is All You Need
- Social Media Use and Perceived Social Isolation Among Young Adults in the U.S.
- Long Short-Term Memory
Related