2018/08/01 by Anna Carobene, Elena Guerra, Massimo Locatelli +7 · 23 citations
Biochemistry, Genetics and Molecular Biology · Chemistry · Medicine · Psychology · #Checklist #Chemistry #Chromatography #Coefficient of variation #Enolase #Medicine #Pathology #Psychology #S100 Proteins and Annexins #Sepsis Diagnosis and Treatment
paper · pdf · doi:10.1373/clinchem.2018.292169
published in Clinical Chemistry 64(10), 1537-1539 (American Association for Clinical Chemistry)
openalex publication_date 2018/08/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29
To the Editor: To obtain reliable estimates of biological variation (BV)1, studies must conform to a standardized approach and be adequately powered and properly documented. The recently published Biological Variation Data Critical Appraisal Checklist (BIVAC) aims to assess whether BV studies contain all elements necessary for the associated BV estimates to be reliable and fit for purpose (1). We here present European Biological Variation Study (EuBIVAS) results for S100-β (S100B) and neuron-specific enolase (NSE) proteins to illustrate the importance of study designs following the BIVAC checklist to ensure the validity of results. NSE and S100B are used in the diagnosis and follow-up of NSE-secreting tumors and melanoma, respectively, and may aid in the diagnosis of brain damage such as concussion and neurological disease. The EuBIVAS results are compared with a recently published study performed in amateur mixed martial arts athletes (2). Briefly, EuBIVAS samples were collected weekly from 91 healthy volunteers (53 females and 38 males; age range, 21–69 years) for 10 weeks. Samples, stored at −80 °C, were measured with a Roche Cobas e801 electrochemiluminescence immunoassay using Roche reagents, calibrators, and control materials, with simultaneous analysis of the hemolysis index because hemoglobin falsely increases NSE concentrations. All samples from the same participant were analyzed in duplicate within a single run. Trend analysis was performed to ensure steady state. The protocol was approved by the Institutional Ethical Board/Regional Ethics Committee in agreement with the World Medical Association Declaration of Helsinki. CVI was estimated using CV-ANOVA for all participants, males, females, and for participants <30 years (n = 22) and ≥30 years of age. Assessment for outliers was performed for replicates and samples on CV-transformed data, and homogeneity of analytical CV (CVA) and within-subject BV (CVI) was examined by the Bartlett and Cochran tests, respectively. Between-subject BV (CVG) estimates were estimated on natural log-transformed data after assessment for outliers between individuals (Dixon criterion). The normality assumption was verified by the Shapiro–Wilk test. Reference change values (RCVs) were calculated using the log normal approach (Table 1). Data analyses were performed using Microsoft Excel 2010 and IBM SPSS statistics, version 23. EuBIVAS and previously published within-subject (CVI) and between-subject (CVG) BV estimates with 95% CI for NSE and S100B. Analytical variation (CVA) estimates were for EuBIVAS based on CV-ANOVA of duplicate analysis of all study samples and for the Johnson study on analysis of manufacturer's quality control samples. RCVs were calculated delivering asymmetric values for increase and decrease at the probability level of 95% for significant unidirectional change, applying CVA estimates based on duplicate measurement of all study samples. RCV as reported by Johnson et al. (2) calculated as Zα√2(CVI2 + CVA2)0.5. EuBIVAS and previously published within-subject (CVI) and between-subject (CVG) BV estimates with 95% CI for NSE and S100B. Analytical variation (CVA) estimates were for EuBIVAS based on CV-ANOVA of duplicate analysis of all study samples and for the Johnson study on analysis of manufacturer's quality control samples. RCVs were calculated delivering asymmetric values for increase and decrease at the probability level of 95% for significant unidirectional change, applying CVA estimates based on duplicate measurement of all study samples. RCV as reported by Johnson et al. (2) calculated as Zα√2(CVI2 + CVA2)0.5. The hemolysis index was <15 (free hemoglobin approximately 150 mg/L) for all samples. Participants were in steady state, and the normality assumption was fulfilled. Estimates of CVI were lower than those recently published by Johnson et al. (2) (Table 1). The sex of the participants in the Johnson et al. study was not provided, and the mean age of participants was 25 years; however, there were no differences in EuBIVAS CVI estimates between sexes or the 2 age-groups (data not shown). Overlapping CIs between the 2 studies were observed. However, the CI widths reported by Johnson et al. (2) are >20 times those of the EuBIVAS. The width of the CI depends on the study design (number of participants, samples, and replicates) and CVA estimates (3). In addition, when assessing the Johnson et al. study with the BIVAC (1), several quality items (QIs) were not fulfilled, namely, those for normally distributed data (QI 9), outliers (QI 8), and estimates of CVA (QI 6), with the latter being based on replicate analysis of control material. Also, it is unclear how CIs were calculated (QI 12). The authors refer to a previously published study (3), but here, CIs depend on replicate analysis of study samples, which was not performed. Furthermore, steady state (QI 7) was not assessed, and some samples were obtained within 48 h or 72 h after the event. This may not be an issue for S100B because it has been reported that steady state is likely to be achieved within 48 h after a sports-related concussion (4). The BIVAC awards overall grades A, B, C, and D indicating decreasing compliance with the BIVAC QIs (1). The Johnson et al. study would receive a grade C and the EuBIVAS a grade A (1). It is likely that publications that miss, or fail to address, essential detail regarding the BIVAC QIs deliver less reliable and, in most cases, overestimates of CVI (1, 3). There are other studies published (5, 6) in which the reported CVI estimates are more similar to those of Johnson et al. (2), but these studies also would receive BIVAC grade C. When performing BV studies, it is important that studies include all elements that may affect veracity and utility of the data, and the studies must also have adequate power, which depends on the study design and the ratio between CVA and CVI (3). The highly powered and fully BIVAC-compliant EuBIVAS delivers considerably lower RCVs and stricter analytical performance specifications than those based on non–BIVAC-compliant studies (Table 1), with clinical implications both for monitoring patients and ensuring that analytical methods are fit for purpose. biological variation Biological Variation Data Critical Appraisal Checklist European Biological Variation Study S100-β protein neuron-specific enolase reference change value quality item. The authors thank Roche for allowing them to work on a dedicated COBAS e801 and for donating all materials used for the measurements. The authors also thank all study participants and the members of the European Federation of Clinical Chemistry and Laboratory Medicine Working Group on Biological Variation and other EuBIVAS partners for their essential contribution to the project: William A. Bartlett, Giulia Cajano, Jorge Díaz-Garzón, Thomas Røraas, Niels Jonker, Gerhard Barla, Una Ørvim Sølvik, Marit Sverresdotter Sylte, Mustafa Serteser, Ibrahim Unsal, Francesca Tosato, and Mario Plebani.