2018/05/16 by J. C. Castura, John C. Castura, S. K. King +2
Agricultural and Biological Sciences · Decision Sciences · Mathematics · #Psychometric Methodologies and Testing #Sensory Analysis and Statistical Methods #Statistical Methods and Applications
paper · doi:10.1111/joss.12329
openalex publication_date 2018/05/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/05/21
Abstract This parallel‐groups study ( n = 1,857) investigates compliance of untrained assessors with tetrad instructions. Stimuli are four unique color swatches that differ only in their green chromaticities. Results confirm that the swatches used in this study were generally perceived as nonconfusable visual stimuli ordered A, B, C, D, and that AB and CD were more different than BC. These stimuli thus modeled the scenario in which following the tetrad decision strategy (grouping samples into two groups of two by similarity) gives a correct response, whereas following the select‐two decision strategy (selecting the two most similar samples) gives an incorrect response. Four of the five variants of tetrad task instructions, including the tetrad instructions that are usually recommended, were understood by roughly 90–95% of the assessors. Neither perceptual variance nor guessing provided adequate explanations for the incorrect groupings observed. Results indicate that the optimal cognitive decision strategy is not always followed because some assessors misinterpret instructions systematically. Results imply that d ′ estimates from the conventional tetrad test method with untrained assessors may slightly underestimate δ. As a safeguard, it is suggested that any written task instructions be supplemented with verbal instructions and/or visual demonstrations, with the aim of increasing task understanding. Practical applications This paper uses empirical evidence from untrained assessors to investigate their performance on various tetrad instructions. Conclusions from this study are relevant to both researchers and practitioners: (i) observed results confirm that the best‐practices tetrad instructions performs well against the other candidate tetrad instructions that were tested; (ii) the use of dissimilar samples revealed that some incorrect tetrad responses are due to task misunderstanding, rather than perceptual variance, as would be assumed by a Thurstonian model; (iii) discriminal distance ( d ′) estimates from the conventional tetrad test method with untrained assessors may slightly underestimate the true population parameter (δ); and (iv) practitioners are advised to communicate task instructions to untrained assessors using multiple modes (verbal, visual) with the aim of increasing task understanding.