Harnessing LLMs for Reliable Academic Supervision: A Comparative Study

2026/07/15 by Akash Raj
Computer Science · #Advanced Graph Neural Networks #Audit #Baseline (sea) #Built environment #Natural Language Processing Techniques #Pipeline (software) #Topic Modeling

paper · pdf · doi:10.5281/zenodo.21380237

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2026/07/15 · openalex created_date 2026/07/16 · openalex updated_date 2026/08/05

Abstract

Harnessing LLMs for Reliable Academic Supervision: A Comparative Study (By: Akash Raj Singh, IIT Jodhpur, 2026). The paper argues for harness engineering - the deliberate composition of deterministic scaffolding (symbolic filters, retrieval, schema-typed I/O, LLM-as-judge loops, HITL gates, persistent state, audit trails) around a language-model core, as a distinct discipline for building reliable LLM-driven systems. It presents a case study in academic supervision, comparing a multi-module LangGraph pipeline built on GPT-4o-mini (ASuS) against a single yet larger LLM baseline built on GPT-5 (ASA), evaluated through a blind ten-rater hybrid study on various harness-mechanism dimensions and a 2 × 2 model-harness ablation.

Citations

• Toolformer: Language Models Can Teach Themselves to Use Tools
• On the Opportunities and Risks of Foundation Models

Related

• LLMs4All: A Review of Large Language Models Across Academic Disciplines
• Analyzing LLM Usage in an Advanced Computing Class in India
• Towards Pedagogical LLMs with Supervised Fine Tuning for Computing Education
• Advances in LLMs with Focus on Reasoning, Adaptability, Efficiency and Ethics
• From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents