Original Article
Three methods to construct predictive models using logistic regression and likelihood ratios to facilitate adjustment for pretest probability give similar results

https://doi.org/10.1016/j.jclinepi.2007.02.012Get rights and content

Abstract

Objective

To compare three predictive models based on logistic regression to estimate adjusted likelihood ratios allowing for interdependency between diagnostic variables (tests).

Study Design and Setting

This study was a review of the theoretical basis, assumptions, and limitations of published models; and a statistical extension of methods and application to a case study of the diagnosis of obstructive airways disease based on history and clinical examination.

Results

Albert's method includes an offset term to estimate an adjusted likelihood ratio for combinations of tests. Spiegelhalter and Knill-Jones method uses the unadjusted likelihood ratio for each test as a predictor and computes shrinkage factors to allow for interdependence. Knottnerus' method differs from the other methods because it requires sequencing of tests, which limits its application to situations where there are few tests and substantial data. Although parameter estimates differed between the models, predicted “posttest” probabilities were generally similar.

Conclusion

Construction of predictive models using logistic regression is preferred to the independence Bayes' approach when it is important to adjust for dependency of tests errors. Methods to estimate adjusted likelihood ratios from predictive models should be considered in preference to a standard logistic regression model to facilitate ease of interpretation and application. Albert's method provides the most straightforward approach.

Section snippets

Background

Evaluation of the diagnostic value of clinical history, examination, and subsequent tests relies on the ability to combine multiple items of diagnostic information. It is through combining several items that good predictive accuracy is achieved. Application to individual patients requires tailoring results according to pretest probabilities of disease [1].

Studies that evaluate combinations of items usually create a diagnostic rule, scoring system or predictive model, which can be used to rate

Illustrative example

The Clinical Assessment of the Reliability of the Examination-Chronic Obstructive Airway Disease (CARE-COAD) study group designed a series of multinational studies to obtain reliable information on the accuracy of the history and physical examination in diagnosing obstructive airways disease (OAD) [12]. The CARE-COAD1 study [13] recruited 309 consecutive patients and noted four items from the history (age, sex, chronic OAD history, smoking history) and two from the clinical examination (wheeze,

Methods

For illustration, we demonstrate the models using only two binary tests of OAD history and age group (Table 2). We first apply the independence Bayes' approach, and then the conventional and alternative logistic regression approaches.

Results for full illustrative example

Parameter estimates for analyses of the COAD1 data sets including all four tests are shown in Table 3, for the independence Bayes, conventional and Albert's logistic regression models, and the SKJ approach. Likelihood ratios are only estimable from the independence Bayes and SKJ models, and are converted into odds ratios (by computing ratios of likelihood ratios) solely for comparison with the logistic regression models.

Likelihood ratios adjusted for dependence using the SKJ approach are all

Discussion

Predictive models are frequently published in the medical literature, both for diagnostic and prognostic applications. Although some models are constructed using Bayesian reasoning, logistic regression is frequently used to take account of dependence between tests. Logistic regression estimates a log odds ratio for each test, simultaneously taking account of other tests included in the model [19]. Although the log odds ratio provides a measure of test performance, it is difficult for clinicians

Acknowledgments

We are grateful to Sharon Straus for providing the CARE-COAD data sets. The work was supported by the National Health and Medical Research Council (NHMRC) grants Grants No. 211205 and No. 402764 to the Screening and Test Evaluation Program. Jon Deeks is supported by a UK Department of Health NCCRCD Senior Research Scientist in Evidence Synthesis award. This work was undertaken as a Master's thesis by the first author.

References (22)

  • J.J. Deeks et al.

    Diagnostic tests 4: likelihood ratios

    BMJ

    (2004)
  • Cited by (25)

    • The RCT-based and the prognostic likelihood ratio

      2021, Journal of Clinical Epidemiology
    • Diagnostic accuracy retrospectively of electrocardiographic findings and cancer history for tamponade in patients determined to have pericardial effusion by transthoracic echocardiogram

      2013, American Journal of Cardiology
      Citation Excerpt :

      We calculated LRs and, for dichotomous findings, sensitivity and specificity using clinical documentation of tamponade as the reference standard. To allow readers to combine LRs from multiple findings,20,21 we used Spiegelhalter-Knill-Jones multivariate regression modeling to account for correlations between multiple diagnostic tests.22,23 We collapsed categories with similar LRs within each ECG finding and constructed CIs for adjusted LRs using a bias-corrected nonparametric bootstrap.23

    • Risk predictions for individual patients from logistic regression were visualized with bar-line charts

      2012, Journal of Clinical Epidemiology
      Citation Excerpt :

      As an extension, the decision support system could also supply information on which factors that contribute to the size of the interaction LRs. It has been claimed that LR-based risk prediction models can be adapted for different populations solely by accounting for differences in prior probabilities, for example, by adding an offset term to the model [13,17,19]. However, sensitivity, specificity, and thus LRs, of diagnostic information may well vary substantially across populations; usually the sensitivity is higher and the specificity lower, the more selected the population is in a referral chain implying that the net result on the LRs is unpredictable [20,21].

    • Correlation between serial tests made disease probability estimates erroneous

      2009, Journal of Clinical Epidemiology
      Citation Excerpt :

      When important correlation between tests is found, likelihood ratios should be reported for all combinations of the test results [4]. Alternatively, one could avoid biased estimates from correlated tests with valid prediction rules or statistical methods for calculating likelihood ratios that are independent of other test results [6,7]. Knowing the correlation between tests will facilitate the utilization of such prediction rules, because they could be approximated with the substitution of a particular, unavailable component with an available, highly correlated test result [13,14].

    View all citing articles on Scopus
    View full text