Original ArticleBivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews
Introduction
Diagnostic accuracy studies are a vital step in the evaluation of diagnostic technologies [1], [2], [3] Accuracy studies measure the level of agreement between the results of a test under evaluation and that of the reference standard. There are several different measures of diagnostic accuracy [4], [5], but the majority of diagnostic accuracy studies present estimates of sensitivity and specificity, either alone or in combination with other measures [6].
Because the majority of diagnostic papers report estimates of sensitivity and specificity, meta-analytic approaches have focused on these measures [6], [7], [8], [9], [10], [11], [12], [13]. Pooling pairs of sensitivity and specificity is not straightforward, because these measures are often negatively correlated within studies.
The summary Receiver Operating Characteristic (sROC) approach has become the method of choice for the meta-analysis of studies reporting pairs of sensitivity and specificity [9], [12], [14], [15], [16], [17], [18]. The sROC approach converts each pair of sensitivity and specificity into a single measure of accuracy, the diagnostic odds ratio [19]. The disadvantage of a single measure of diagnostic accuracy is that it does not distinguish between the ability of detecting the sick (sensitivity) and identifying the well (specificity). Discriminating between these abilities is important to determine the optimal use of a test in clinical practice. The bivariate model we propose has the distinct advantage of preserving the two-dimensional nature of the underlying data. It can also produce summary estimates of sensitivity and specificity, acknowledging any possible (negative) correlation between these two measures. We will discuss both approaches and illustrate their use by reanalyzing the data from a published meta-analysis [20].
Section snippets
Pooling pairs of sensitivity and specificity: why simple methods fail
Diagnostic reviews start with a set of individual studies presenting estimates of sensitivity and specificity. One intuitive approach is to do separate pooling of sensitivity and specificity using standard methods for proportions. However, sensitivity and specificity are often negatively correlated within studies, and ignoring this correlation would be inappropriate [7], [11], [12].
One possible cause for this negative correlation between sensitivity and specificity is that studies may have used
Summary ROC approach
We provide a short description of the sROC approach as outlined by Moses and Littenberg. More details can be found elsewhere [9], [12], [14], [15], [16], [17], [18].
The sROC approach starts with plotting the observed pairs of sensitivity and specificity of each study in ROC space (see Fig. 1). The aim of the sROC approach is to find a smooth curve through these points. The key step is to transform the TPR (sensitivity) and FPR (1 − specificity) scale of the ROC graph so that the relation
Bivariate model
The bivariate model uses a different starting point for the meta-analysis of pairs of sensitivity and specificity. Rather than transforming these two distinct outcome measures into a single indicator of diagnostic accuracy as in the sROC approach, the bivariate model preserves the two-dimensional nature of the data throughout the analysis.
The bivariate model is based on the following line of reasoning [10], [24], [25]. We assume that the sensitivities from individual studies (after logit
General discussion of methods of meta-analysis
In this section we discuss the statistical methods for meta-analysis of studies of diagnostic accuracy by comparing them on a few key items. These are: (1) the choice of outcome measure; (2) the effect of covariates; (3) the statistical properties of the model.
References (46)
- et al.
Assessment of the accuracy of diagnostic tests: the cross-sectional study
J Clin Epidemiol
(2003) - et al.
Meta-analytic methods for diagnostic test accuracy
J Clin Epidemiol
(1995) - et al.
The diagnostic odds ratio: a single indicator of test performance
J Clin Epidemiol
(2003) Exploring heterogeneity in meta-analysis: is the L'Abbe plot useful?
J Clin Epidemiol
(1999)Empirical Bayes estimates generated in a hierarchical summary ROC analysis agreed closely with those of a full Bayesian analysis
J Clin Epidemiol
(2004)- et al.
Tumor markers in the diagnosis of primary bladder cancer. A systematic review
J Urol
(2003) - et al.
Computed tomography and magnetic resonance imaging in staging of uterine cervical carcinoma: a systematic review
Gynecol Oncol
(2003) - et al.
The architecture of diagnostic research
- et al.
A framework for clinical evaluation of diagnostic technologies
CMAJ
(1986) - et al.
Selection and interpretation of diagnostic tests and procedures. Principles and applications
Ann Intern Med
(1981)
Analysis of data on the accuracy of diagnostic tests
Reporting of measures of accuracy in systematic reviews of diagnostic literature
BMC Health Serv Res
Meta-analysis of screening data: a survey of the literature
Stat Med
Empirical evidence of design-related bias in studies of diagnostic tests
JAMA
Comparative diagnostic performance of three radiological procedures for the detection of lumbar disk herniation
Methods Inf Med
A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations
Stat Med
Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations
Stat Med
Systematic reviews in health care: systematic reviews of evaluations of diagnostic and screening tests
BMJ
Meta-analysis of screening and diagnostic tests
Psychol Bull
Guidelines for meta-analyses evaluating diagnostic tests
Ann Intern Med
Estimating diagnostic accuracy from multiple conflicting reports: a new meta-analytic method
Med Decis Making
A meta-analytic method for summarizing diagnostic test performances: receiver-operating- characteristic-summary point estimates
Med Decis Making
Properties of the summary receiver operating characteristic (SROC) curve for diagnostic test data
Stat Med
Cited by (2502)
Diagnostic performance of artificial intelligence-aided caries detection on bitewing radiographs: a systematic review and meta-analysis
2024, Japanese Dental Science ReviewRisk scores for prediction of paroxysmal atrial fibrillation after acute ischemic stroke or transient ischemic attack: A systematic review and meta-analysis
2024, International Journal of Cardiology: Cardiovascular Risk and PreventionA Bayesian finite mixture model approach to evaluate dichotomization method for correlated ELISA tests
2024, Preventive Veterinary MedicineDiagnostic value of microRNAs in active tuberculosis based on quantitative and enrichment analyses
2024, Diagnostic Microbiology and Infectious Disease