Not All Options Are Created Equal: Textual Option Weighting for Token-Efficient LLM-Based Knowledge Tracing

2024/10/14 by JongWoo Kim, Kim, JongWoo, Chu, SeongYeub +4 · 2 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computers and Society (cs.CY) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2410.12872

openalex publication_date 2024/10/14 · openalex created_date 2024/10/20 · openalex updated_date 2026/07/28

Abstract

Large Language Models (LLMs) have recently emerged as promising tools for knowledge tracing (KT) due to their strong reasoning and generalization abilities. While recent LLM-based KT methods have proposed new prompt formats, they struggle to represent the full interaction histories of example learners within a single prompt during in-context learning (ICL), resulting in limited scalability and high computational cost under token constraints. In this work, we present LLM-based Option-weighted Knowledge Tracing (LOKT), a simple yet effective framework that encodes the interaction histories of example learners in context as textual categorical option weights (TCOW). TCOW are semantic labels (e.g., ``inadequate'') assigned to the options selected by learners when answering questions, enhancing the interpretability of LLMs. Experiments on multiple-choice datasets show that LOKT outperforms existing non-LLM and LLM-based KT models in both cold-start and warm-start settings. Moreover, LOKT enables scalable and cost-efficient inference, achieving strong performance even under strict token constraints. Our code is available at \hrefhttps://anonymous.4open.science/r/LOKT_model-3233https://anonymous.4open.science/r/LOKT_model-3233.

Cited by

• HISE-KT: Synergizing Heterogeneous Information Networks and LLMs for Explainable Knowledge Tracing with Meta-Path Optimization
• Language Bottleneck Models for Qualitative Knowledge State Modeling

Related

• Language Model Can Do Knowledge Tracing: Simple but Effective Method to Integrate Language Model and Knowledge Tracing Task
• LLM-KT: Aligning Large Language Models with Knowledge Tracing using a Plug-and-Play Instruction
• Next Token Knowledge Tracing: Exploiting Pretrained LLM Representations to Decode Student Behaviour
• A Systematic Review of Knowledge Tracing and Large Language Models in Education: Opportunities, Issues, and Future Research
• DKT2: Revisiting Applicable and Comprehensive Knowledge Tracing in Large-Scale Data