LLM Driven Processes to Foster Explainable AI

2025/11/10 by Pehlke, Marcel, Jansen, Marc · 1 citation
Computer Science · Social Sciences · #AI-based Problem Solving and Planning #Artificial Intelligence (cs.AI) #Ethics and Social Impacts of AI #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2511.07086

openalex publication_date 2025/11/10 · openalex created_date 2025/11/12 · openalex updated_date 2026/07/28

Abstract

We present a modular, explainable LLM-agent pipeline for decision support that externalizes reasoning into auditable artifacts. The system instantiates three frameworks: Vester's Sensitivity Model (factor set, signed impact matrix, systemic roles, feedback loops); normal-form games (strategies, payoff matrix, equilibria); and sequential games (role-conditioned agents, tree construction, backward induction), with swappable modules at every step. LLM components (default: GPT-5) are paired with deterministic analyzers for equilibria and matrix-based role classification, yielding traceable intermediates rather than opaque outputs. In a real-world logistics case (100 runs), mean factor alignment with a human baseline was 55.5% over 26 factors and 62.9% on the transport-core subset; role agreement over matches was 57%. An LLM judge using an eight-criterion rubric (max 100) scored runs on par with a reconstructed human baseline. Configurable LLM pipelines can thus mimic expert workflows with transparent, inspectable steps.

Citations

• Routoo: Learning to Route to Large Language Models Effectively
• Large Language Model based Multi-Agents: A Survey of Progress and Challenges
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models
• ReAct: Synergizing Reasoning and Acting in Language Models
• Training Compute-Optimal Large Language Models
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• Scaling Laws for Neural Language Models
• Attention Is All You Need

Cited by

• Towards Responsible and Explainable AI Agents with Consensus-Driven Reasoning
• Standardization of Neuromuscular Reflex Analysis -- Role of Fine-Tuned Vision-Language Model Consortium and OpenAI gpt-oss Reasoning LLM Enabled Decision Support System

Related

• Increasing AI Explainability by LLM Driven Standard Processes
• A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems
• Flow-of-Options: Diversified and Improved LLM Reasoning by Thinking Through Options
• LLM-based Agentic Reasoning Frameworks: A Survey from Methods to Scenarios
• Making LLMs Reliable When It Matters Most: A Five-Layer Architecture for High-Stakes Decisions