Validation of the Summary ROC for Diagnostic Test Meta-Analysis: A Monte Carlo Simulation
Section snippets
Materials and Methods
To examine the performance of the logit regression method under a variety of conditions, simulated data sets of known characteristics were created and meta-analyzed. A program was written in the macro language of Excel (Office 2000; Microsoft, Redmond, Wash) to create the simulated data sets. An Excel spreadsheet was programmed to perform the meta-analyses and record results. The spreadsheet was validated by recalculating previously published meta-analyses results and comparing its results with
Results and Discussion
In this initial validation study, the base scenario was a meta-analysis of 10 trials, with a mean sample size per trial of 50 patients (SD, 20 patients). This is consistent with the typical evidence base for emerging diagnostic imaging applications assessed at my institution. The effects a larger study size and larger number of trials have on the meta-analysis will be examined in detail in future experiments. Future experiments will also examine the effect of random variability from trial to
Conclusion
The logit regression method for summary ROC meta-analysis of diagnostic clinical trial results enables accurate estimation of the sensitivity and specificity of the test being studied. Although the method systematically underestimates the sensitivity and specificity for very high levels of test performance, the underestimation is no more than 2% for each parameter. An increase in the size of the clinical trials being meta-analyzed virtually eliminates this bias, which indicates that it is a
References (6)
- et al.
Meta-analysis of diagnostic procedures: a brief overview
Acad Radiol
(1998) - et al.
Should FDG PET be used to decide whether a patient with an abnormal mammogram or breast finding at physical examination should undergo biopsy?
Acad Radiol
(2002)
Cited by (25)
Assessment of 3DTV-related fatigue with resting-state fMRI
2018, Signal Processing: Image CommunicationDiagnostic accuracy of osteopontin plus alpha-fetoprotein in the hepatocellular carcinoma: A meta-analysis
2017, Clinics and Research in Hepatology and GastroenterologyCitation Excerpt :The overall diagnostic sensitivity, specificity, the pooled PLR and NLR were 0.71 (95% CI: 0.69–0.74), 0.80 (95% CI: 0.78–0.82), 3.55 (95% CI: 2.58–4.90) and 0.26 (95% CI: 0.17–0.40) for OPN; 0.61 (95% CI: 0.58–0.63), 0.92 (95% CI: 0.91–0.94), 7.57 (95% CI: 5.00–11.46) and 0.41 (95% CI: 0.33–0.51) for AFP; 0.82 (95% CI: 0.79–0.84), 0.77 (95% CI: 0.74–0.80), 3.68 (95% CI: 2.45–5.53) and 0.19 (95% CI: 0.11–0.32) for OPN + AFP, respectively (Figs. 2–4). In addition, the summary receiver operating characteristics (SROC) curve was employed to obtain an accurate summary estimate of diagnostic test performance [39]. The area under the curve (AUC) was used for summarizing SROC.
Improvement of the lipid profile with exercise in obese children: A systematic review
2012, Preventive MedicineCitation Excerpt :Pooled data were calculated for the studies grouped into two categories: aerobic programs and combined programs. If the study did not provide the data for the mean, standard deviation, standard error, or standard error of the mean, then the mean and standard deviation were computed by Monte Carlo simulation (Mitchell, 2003), a technique which uses mathematical modeling to imitate the random behavior of real non-dynamic systems. The simulation allows the researcher to specify many parameters that affect the simulated data sets and explore the effects of those parameters on the accuracy of the results.
Meta-analysis of B-type natriuretic peptide's ability to identify stress induced myocardial ischemia
2011, American Journal of CardiologyCitation Excerpt :Although random and fixed effect models were used, owing to significant heterogeneity we reported only results from random effect models. A summary receiver operating characteristic curve9 was constructed from pooled data for studies included in the final analysis. Subgroup analyses were performed to explore the significant heterogeneity observed.
The diagnostic accuracy of CT and MRI in the staging of pelvic lymph nodes in patients with prostate cancer: a meta-analysis
2008, Clinical RadiologyCitation Excerpt :To address this problem, SROC analysis was used in this meta-analysis. SROC analysis corrects for variation due to differences in test thresholds in the original studies.9–11,14,40 The present study had the following limitations: first, some of the studies included in the meta-analysis were subsets from a larger study as only a limited number of patients fulfilled the inclusion criteria.