Fleiss’ kappa statistic without paradoxes

2014/02/12 by Rosa Falotico, Piero Quatto · 228 citations
Decision Sciences · Mathematics · #Cohen's kappa #Computer science #Confidence interval #Econometrics #Index (typography) #Kappa #Mathematics #Multi-Criteria Decision Making #Permutation (music) #Physics #Reliability (semiconductor) #Reliability and Agreement in Measurement #Statistic #Statistical Methods in Epidemiology #Statistical hypothesis testing #Statistics #Test statistic

paper · doi:10.1007/s11135-014-0003-1

published in Quality & Quantity 49(2), 463-470 (Springer Science+Business Media)

openalex publication_date 2014/02/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Citations

• A Coefficient of Agreement for Nominal Scales

Cited by

• ReviewGuard: Enhancing Deficient Peer Review Detection via LLM-Driven Data Augmentation
• Advancing Automated Ethical Profiling in SE: a Zero-Shot Evaluation of LLM Reasoning
• Construcción y validación de un cuestionario para la reflexión pedagógica del profesor universitario centrada en el aprendizaje
• Open Access in language testing and assessment: The case of two flagship journals
• Multi-Modal Requirements Data-based Acceptance Criteria Generation using LLMs
• Hate in Plain Sight: On the Risks of Moderating AI-Generated Hateful Illusions
• Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalities
• Quantifying Uncertainty in Error Consistency: Towards Reliable Behavioral Comparison of Classifiers
• How Robust is Model Editing after Fine-Tuning? An Empirical Study on Text-to-Image Diffusion Models
• Leveraging Network Methods for Hub-like Microservice Detection
• Model Context Protocol (MCP) at First Glance: Studying the Security and Maintainability of MCP Servers
• Rethinking Hate Speech Detection on Social Media: Can LLMs Replace Traditional Models?
• Real Faults in Model Context Protocol (MCP) Software: a Comprehensive Taxonomy
• Evaluating the Quality of Benchmark Datasets for Low-Resource Languages: A Case Study on Turkish
• By the numbers: ratings and utilization of behavioral health mobile applications. [europepmc]
• Content Validation through Expert Judgement of an Instrument on the Nutritional Knowledge, Beliefs, and Habits of Pregnant Women. [europepmc]
• Content Validation of an Instrument for the Assessment of School Teachers' Levels of Knowledge of Diabetes through Expert Judgment. [europepmc]
• Practical applicability of the STAMCO and ChOLE classification in cholesteatoma care. [europepmc]
• Design and Validation of an Instrument for Evaluating Training and Education for Health and Proper Use of Vaccines "VACUNASEDUCA". [europepmc]
• Visual Evaluation of Image Quality of a Low Dose 2D/3D Slot Scanner Imaging System Compared to Two Conventional Digital Radiography X-ray Imaging Systems. [europepmc]
• Feasibility and Reliability of the AWIN Welfare Assessment Protocol for Dairy Goats in Semi-extensive Farming Conditions. [europepmc]
• Semantic Segmentation of Urinary Bladder Cancer Masses from CT Images: A Transfer Learning Approach. [europepmc]
• A computed tomography based survey study investigating the agreement of the therapeutic strategy for fragility fractures of the pelvis. [europepmc]
• Comparison of Concordance between Chuna Manual Therapy Diagnostic Methods (Palpation, X-ray, Artificial Intelligence Program) in Lumbar Spine: An Exploratory, Cross-Sectional Clinical Study. [europepmc]
• The BioExperience Research and Entrepreneurship Challenge: An iGEM-inspired applied research program for BIOSTEM talent and skills development. [europepmc]
• ECAsT: a large dataset for conversational search and an evaluation of metric robustness. [europepmc]
• Systematic Review of Extended Reality Digital Therapy for Enhancing Mental Health Among South Korean Adolescents and Young Adults. [europepmc]
• Assessing food availability and healthier options in an urban Chinese university: a case study using the Chinese Nutrition Environment Measurement Survey for Stores (C-NEMS-S). [europepmc]
• Prevalence of gingivitis and risk factors among pregnant women from Acapulco, Guerrero: a cross-sectional study. [europepmc]
• Prospective Study of Ultrasound Markers of Organ Congestion in Critically Ill Patients With Acute Kidney Injury. [europepmc]
• Triplétoile: Extraction of knowledge from microblogging text. [europepmc]
• Leveraging Large Language Models for Infectious Disease Surveillance-Using a Web Service for Monitoring COVID-19 Patterns From Self-Reporting Tweets: Content Analysis. [europepmc]
• Evidence of Mycobacterium bovis DNA in shared water sources at livestock-wildlife-human interfaces in KwaZulu-Natal, South Africa. [europepmc]
• Accuracy of LLMs in medical education: evidence from a concordance test with medical teacher. [europepmc]
• Diagnosing the innovation atmosphere of industrial parks through urban spatial perception: a multimodal large language model approach. [europepmc]
• The limits of curiosity? New evidence for the roles of metacognitive abilities and curiosity in learning. [europepmc]
• An algorithmic system for arabic fake news detection using neural networks and transformer embeddings with class weighting. [europepmc]
• Practice (Doesn´t) Make Perfect Shooters: The Influence of Experience on Penalty Execution in Elite Soccer. [europepmc]
• Comparing Agreement Indices to Assess Inter-Observer Reliability in the Case of Dichotomous and Trichotomous Animal-Based Welfare Indicators with Three Raters [europepmc]