2024/11/29 by Victoria Gale, Philip A. Powell, Jill Carlton · 1 voice
Decision Sciences · Social Sciences · #Meta-analysis and systematic reviews #Qualitative Research Methods and Ethics #Reliability and Agreement in Measurement
paper · pdf · doi:10.1007/s11136-024-03858-y
openalex publication_date 2024/11/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/23
PURPOSE: Evidence of comprehensibility is frequently required during the development of patient reported outcome measures (PROMs); the respondent's interpretation of PROM items needs to align with intended meanings. Cognitive interviews are recommended for investigating PROM comprehensibility, yet guidance for analysis is lacking. Consequently, the quality and trustworthiness of cognitive interview data and analysis is threatened, as there is no clear procedure detailing how analysts can systematically, and consistently, identify evidence that respondent interpretations align/misalign with intended meanings. METHODS: This paper presents a novel, structured approach to comprehensibility analysis - the 'Comprehensibility Continuum' - that builds upon existing cognitive interview guidance. RESULTS: The Comprehensibility Continuum comprises a structured rating scale to code depth of alignment between intended item meaning and respondent interpretation and consists of five main stages: before cognitive interviews are conducted, researchers must (1) Define intended meanings of PROM items; and (2) Determine comprehensibility thresholds for both participant- and item-level. After conducting interviews, they (3) Prepare data by transcribing interviews 'intelligent' verbatim; (4) Code transcripts using the Comprehensibility Continuum scale in iterative sets, assigning an overall code for each item at participant-level; and (5) Compare participant-level codes across all participants to determine overall item comprehensibility, such that decisions can be made to retain, modify, or remove items. CONCLUSION: Quality in qualitative data analysis is achieved through rigorous methods that are clearly described and justified. Given insufficiency in guidelines, cognitive interviewers must reflect on how best to demonstrate PROM comprehensibility systematically and consistently from interview data, and the Comprehensibility Continuum method offers a potential solution.