Noise or Nuance: An Investigation Into Useful Information and Filtering For LLM Driven AKBC

2025/09/10 by Clay, Alex, Jiménez-Ruiz, Ernesto, Madhyastha, Pranava
#Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2509.08903

Abstract

RAG and fine-tuning are prevalent strategies for improving the quality of LLM outputs. However, in constrained situations, such as that of the 2025 LM-KBC challenge, such techniques are restricted. In this work we investigate three facets of the triple completion task: generation, quality assurance, and LLM response parsing. Our work finds that in this constrained setting: additional information improves generation quality, LLMs can be effective at filtering poor quality triples, and the tradeoff between flexibility and consistency with LLM response parsing is setting dependent.

Citations

• Enabling LLM Knowledge Analysis via Extensive Materialization
• MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization
• MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents
• ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems
• ALCUNA: Large Language Models Meet New Knowledge
• The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"
• Crawling the Internal Knowledge-Base of Language Models
• An Empirical Study on Few-shot Knowledge Probing for Pretrained Language Models
• How Can We Know What Language Models Know?
• Language Models as Knowledge Bases?

Related

• Resurrecting saturated LLM benchmarks with adversarial encoding
• LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback
• High Accuracy, Less Talk (HALT): Reliable LLMs through Capability-Aligned Finetuning
• NeurIPS 2023 LLM Efficiency Fine-tuning Competition
• Brevity is the soul of sustainability: Characterizing LLM response lengths