Automatic Extraction of Clausal Embedding Based on Large-Scale English Text Data

2025/06/16 by Iona Carslaw, Sivan Milton, Carslaw, Iona +7
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques

paper · pdf · doi:10.48550/arxiv.2506.14064

openalex publication_date 2025/06/16 · openalex created_date 2025/10/18 · openalex updated_date 2026/07/28

Abstract

For linguists, embedded clauses have been of special interest because of their intricate distribution of syntactic and semantic features. Yet, current research relies on schematically created language examples to investigate these constructions, missing out on statistical information and naturally-occurring examples that can be gained from large language corpora. Thus, we present a methodological approach for detecting and annotating naturally-occurring examples of English embedded clauses in large-scale text data using constituency parsing and a set of parsing heuristics. Our tool has been evaluated on our dataset Golden Embedded Clause Set (GECS), which includes hand-annotated examples of naturally-occurring English embedded clause sentences. Finally, we present a large-scale dataset of naturally-occurring English embedded clauses which we have extracted from the open-source corpus Dolma using our extraction tool.

Citations

• Syntactic Search by Example
• Frequency, Acceptability, and Selection: A case study of\n clause-embedding
• Multilingual Constituency Parsing with Self-Attention and Pre-Training
• Constituency Parsing with a Self-Attentive Encoder

Related

• Automatic Extraction of Clausal Embedding Based on Large-Scale English Text Data
• Building a Web-Scale Dependency-Parsed Corpus from CommonCrawl
• Gathering Statistics to Aspectually Classify Sentences with a Genetic Algorithm
• Mining Discourse Markers for Unsupervised Sentence Representation Learning
• Multilingual Chart-based Constituency Parse Extraction from Pre-trained Language Models