QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization

2021/04/13 by Ming Zhong, Zhong, Ming, Da Yin +19 · 39 citations
Computer Science · #Advanced Graph Neural Networks #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2104.05938

openalex created_date 2021/04/13 · openalex publication_date 2021/04/13 · openalex updated_date 2026/07/28

Abstract

Meetings are a key component of human collaboration. As increasing numbers of meetings are recorded and transcribed, meeting summaries have become essential to remind those who may or may not have attended the meetings about the key decisions made and the tasks to be completed. However, it is hard to create a single short summary that covers all the content of a long meeting involving multiple people and topics. In order to satisfy the needs of different types of users, we define a new query-based multi-domain meeting summarization task, where models have to select and summarize relevant spans of meetings in response to a query, and we introduce QMSum, a new benchmark for this task. QMSum consists of 1,808 query-summary pairs over 232 meetings in multiple domains. Besides, we investigate a locate-then-summarize method and evaluate a set of strong summarization baselines on the task. Experimental results and manual analysis reveal that QMSum presents significant challenges in long meeting summarization for future research. Dataset is available at \urlhttps://github.com/Yale-LILY/QMSum.

Cited by

• Coordinated Networking for On-Device Agent-Augmented Real-Time Communication
• Enhancing Long Document Long Form Summarisation with Self-Planning
• Short-Context Dominance: How Much Local Context Natural Language Actually Needs?
• SpecPV: Improving Self-Speculative Decoding for Long-Context Generation via Partial Verification
• WaterSearch: A Quality-Aware Search-based Watermarking Framework for Large Language Models
• Hierarchical Token Prepending: Enhancing Information Flow in Decoder-based LLM Embeddings
• KV Cache Transform Coding for Compact Storage in LLM Inference
• Optimizing Native Sparse Attention with Latent Attention and Local Global Alternating Strategies
• Scalable Processing-Near-Memory for 1M-Token LLM Inference: CXL-Enabled KV-Cache Management Beyond GPU Limits
• APTBench: Benchmarking Agentic Potential of Base LLMs During Pre-Training
• Evaluating Long-Term Memory for Long-Context Question Answering
• Simple Context Compression: Mean-Pooling and Multi-Ratio Training
• Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents
• Taming the Fragility of KV Cache Eviction in LLM Inference
• Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchors
• DACIP-RC: Domain Adaptive Continual Instruction Pre-Training via Reading Comprehension on Business Conversations
• DACP: Domain-Adaptive Continual Pre-Training of Large Language Models for Phone Conversation Summarization
• CAM: A Constructivist View of Agentic Memory for LLM-Based Reading Comprehension
• Adaptive Planning for Multi-Attribute Controllable Summarization with Monte Carlo Tree Search
• SparseServe: Unlocking Parallelism for Dynamic Sparse Attention in Long-Context LLM Serving
• CHOIR: A Chatbot-mediated Organizational Memory Leveraging Communication in University Research Labs
• When Long Helps Short: How Context Length in Supervised Fine-tuning Affects Behavior of Large Language Models
• CogniLoad: A Synthetic Natural Language Reasoning Benchmark With Tunable Length, Intrinsic Difficulty, and Distractor Density
• Value-Guided KV Compression for LLMs via Approximated CUR Decomposition
• From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users
• Topic-Guided Reinforcement Learning with LLMs for Enhancing Multi-Document Summarization
• HoPE: Hyperbolic Rotary Positional Encoding for Stable Long-Range Dependency Modeling in Large Language Models
• On Robustness and Reliability of Benchmark-Based Evaluation of LLMs
• MSRS: Evaluating Multi-Source Retrieval-Augmented Generation
• AdaptCache: KV Cache Native Storage Hierarchy for Low-Delay and High-Quality Language Model Serving
• SurGE: A Benchmark and Evaluation Framework for Scientific Survey Generation
• TracSum: A New Benchmark for Aspect-Based Summarization with Sentence-Level Traceability in Medical Domain
• AttnTrace: Attention-based Context Traceback for Long-Context LLMs
• Teaching Language Models To Gather Information Proactively
• From Benchmarks to Skills: Low-Rank Factors for LLM Evaluation
• Flora: Effortless Context Construction to Arbitrary Length and Scale
• Smooth Reading: Bridging the Gap of Recurrent LLM to Self-Attention LLM on Long-Context Tasks
• Recent Trends in Distant Conversational Speech Recognition: A Review of CHiME-7 and 8 DASR Challenges
• StreamHover: Livestream Transcript Summarization and Annotation

Related

• Investigating Consistency in Query-Based Meeting Summarization: A Comparative Study of Different Embedding Methods
• Meeting Summarization with Pre-training and Clustering Methods
• Meeting Summarization: A Survey of the State of the Art
• Improving Query-Focused Meeting Summarization with Query-Relevant Knowledge
• Learning to Rank Utterances for Query-Focused Meeting Summarization