SLURP: A Spoken Language Understanding Resource Package

2020/11/26 by Bastianelli, Emanuele, Vanzo, Andrea, Swietojanski, Pawel +1 · 25 citations
#Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2011.13205

Abstract

Spoken Language Understanding infers semantic meaning directly from audio data, and thus promises to reduce error propagation and misunderstandings in end-user applications. However, publicly available SLU resources are limited. In this paper, we release SLURP, a new SLU package containing the following: (1) A new challenging dataset in English spanning 18 domains, which is substantially bigger and linguistically more diverse than existing datasets; (2) Competitive baselines based on state-of-the-art NLU and ASR systems; (3) A new transparent metric for entity labelling which enables a detailed error analysis for identifying potential areas of improvement. SLURP is available at https: //github.com/pswietojanski/slurp.

Cited by

• ESF-Bench: Benchmarking Challenging Slot-Filling Scenarios for Real-World Enterprise Applications
• Speech World Model: Causal State-Action Planning with Explicit Reasoning for Speech
• Multi-Intent Spoken Language Understanding: Methods, Trends, and Challenges
• MAC-SLU: Multi-Intent Automotive Cabin Spoken Language Understanding Benchmark
• Two-Dimensional Quantization for Geometry-Aware Audio Coding
• Introducing Visual Scenes and Reasoning: A More Realistic Benchmark for Spoken Language Understanding
• Multimodal Evaluation of Russian-language Architectures
• Beyond Task-Oriented and Chitchat Dialogues: Proactive and Transition-Aware Conversational Agents
• TASU: Text-Only Alignment for Speech Understanding
• M3-SLU: Evaluating Speaker-Attributed Reasoning in Multimodal Large Language Models
• SAC: Neural Speech Codec with Semantic-Acoustic Dual-Stream Quantization
• InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialogue
• Standard-to-Dialect Transfer Trends Differ across Text and Speech: A Case Study on Intent and Topic Classification in German Dialects
• Full-Duplex-Bench-v2: A Multi-Turn Evaluation Framework for Duplex Dialogue Systems with an Automated Examiner
• WavJEPA: Semantic learning unlocks robust audio foundation models for raw waveforms
• Speech Discrete Tokens or Continuous Features? A Comparative Analysis for Spoken Language Understanding in SpeechLLMs
• FocalCodec-Stream: Streaming Low-Bitrate Speech Coding via Causal Distillation
• Exploring Fine-Tuning of Large Audio Language Models for Spoken Language Understanding under Limited Speech data
• MTEB-NL and E5-NL: Embedding Benchmark and Models for Dutch
• SENSE models: an open source solution for multilingual and multimodal semantic-based tasks
• Say More with Less: Variable-Frame-Rate Speech Tokenization via Adaptive Clustering and Implicit Duration Coding
• LLaSO: A Foundational Framework for Reproducible Research in Large Language and Speech Model
• RealTalk-CN: A Realistic Chinese Speech-Text Dialogue Benchmark With Cross-Modal Interaction Analysis
• Deep Learning Approaches for Multimodal Intent Recognition: A Survey
• AudioJudge: Understanding What Works in Large Audio Model Based Speech Evaluation

Related

• SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation\n on Natural Speech
• Fleurs-SLU: A Massively Multilingual Benchmark for Spoken Language Understanding
• STOP: A dataset for Spoken Task Oriented Semantic Parsing
• A Study on the Integration of Pipeline and E2E SLU systems for Spoken Semantic Parsing toward STOP Quality Challenge
• Speech-MASSIVE: A Multilingual Speech Dataset for SLU and Beyond