Predicting unplanned admissions to hospital in older adults using routinely recorded general practice data: development and validation of a prediction model

Jet H Klunder; Martijn W Heymans; Iris van der Heide; Robert A Verheij; Otto R Maarsingh; Hein PJ van Hout; Karlijn J Joling

doi:10.3399/BJGP.2023.0350

Abstract

Background Unplanned admissions to hospital represent a hazardous event for older people. Timely identification of high-risk individuals using a prediction tool may facilitate preventive interventions.

Aim To develop and validate an easy-to-use prediction model for unplanned admissions to hospital in community-dwelling older adults using readily available data to allow rapid bedside assessment by GPs.

Design and setting This was a retrospective study using the general practice electronic health records of 243 324 community-dwelling adults aged ≥65 years linked with national administrative data to predict unplanned admissions to hospital within 6 months.

Method The dataset was geographically split into a development (n = 142 791/243 324, 58.7%) and validation (n = 100 533/243 324, 41.3%) sample to predict unplanned admissions to hospital within 6 months. The performance of three different models was evaluated with increasingly smaller selections of candidate predictors (optimal, readily available, and easy-to-use models). Logistic regression was used with backward selection for model development. The models were validated internally and externally. Predictive performance was assessed by area under the curve (AUC) and calibration plots.

Results In both samples, 7.6% (development cohort: n = 10 839/142 791, validation cohort: n = 7675/100 533) had ≥1 unplanned hospital admission within 6 months. The discriminative ability of the three models was comparable and remained stable after geographic validation. The easy-to-use model included age, sex, prior admissions to hospital, pulmonary emphysema, heart failure, and polypharmacy. Its discriminative ability after validation was AUC 0.72 (95% confidence interval = 0.71 to 0.72). Calibration plots showed good calibration.

Conclusion The models showed satisfactory predictive ability. Reducing the number of predictors and geographic validation did not have an impact on predictive performance, demonstrating the robustness of the model. An easy-to-use tool has been developed in this study that may assist GPs in decision making and with targeted preventive interventions.

Introduction

Increasing rates of unplanned admissions to hospital in older adults are a major burden on healthcare systems worldwide. For patients, unplanned hospital admissions are associated with functional decline and reduced quality of life.¹ People with dementia are at particularly high risk of unplanned admissions, associated with worsening of pre-existing cognitive problems and an increased risk of readmission and death.²^–⁴

Preventing unplanned admissions is critical to ensure patient safety and wellbeing, and aligns with the World Health Organization’s philosophy of providing tailored care in appropriate settings for older adults.⁵ A proactive approach optimises the allocation of scarce healthcare resources and addresses a pervasive concern in healthcare systems worldwide, where increasing demand outpaces the capacity of healthcare professionals. In the Netherlands, the integral care agreement (ICA) of 2022 prioritises preventive measures for acute care, particularly for older adults. Through education, prevention, and early signalling initiatives, the ICA aims to reduce unplanned admissions to hospital.⁶

Interventions such as providing an anticipatory care plan, telemedicine, and integrating a multidisciplinary geriatric team have been shown to reduce the number of unplanned admissions to hospital.⁷^–¹⁰ However, timely identification of high-risk groups is essential for implementing proactive and targeted interventions.

Unplanned hospital admissions in older adults are a critical concern for patients, family caregivers, healthcare professionals, and service planners. In this study a robust and easy-to-use prediction model has been developed and validated using routinely recorded data from general practices to predict the risk of unplanned hospital admissions in community-dwelling older adults. Identifying older adults at high risk can facilitate targeted preventive interventions, such as case management, telemedicine, or anticipatory care planning. Moreover, the model could also be utilised by policymakers for capacity planning of hospital beds.

How this fits in

GPs are patients’ primary point of contact and act as gatekeepers in many healthcare systems, such as the Netherlands.¹¹ Therefore, they play a pivotal role in identifying those at risk for unplanned admission to hospital and the targeting of preventive interventions. A prediction model that can accurately predict high-risk individuals by reusing patient registration data could help GPs identify these individuals. The use of electronic health record (EHR) data offers opportunities for the development, integration, and automated calculation of an individual’s risk, because it contains comprehensive patient information and is derived from routine health care. The utilisation of these readily available data for the development of a prediction model facilitates ease of use and reduces time burden on GPs. Previous research has shown that administrative data can be useful in accurately predicting unplanned admissions to hospital.¹²^,¹³ However, the methodological quality of these studies was limited and many models required additional data collection, making clinical use difficult. Models based on routine care data have a lower threshold and might therefore be used more frequently. As a result, their potential impact would be greater, even if the predictive power is similar.

The aim of this study was to develop and validate a practical and easy-to-use prediction model for unplanned admissions to hospital using a Dutch representative sample of older people in general practice. The model was developed using current state-of-the-art methods and incorporating readily available EHR data complemented with national administrative data. Also, the study specifically assessed the predictive performance of the model in a subsample of individuals with cognitive decline or dementia.

Method

This study is reported according to the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines.¹⁴

Sources of data

Pseudonymised EHR data from GPs linked with data from national administrative databases were used in this study. The baseline data covered the year 2013; outcomes were assessed in 2014. The routine EHRs used were from a nationally representative sample of 417 Dutch general practices participating in NIVEL-Primary Care Database (NIVEL-PCD).¹⁵ This database covers about 10% of the Dutch population and is representative in terms of practice type, urbanisation level, and age and gender distribution.¹⁵^,¹⁶ Data include information on chronic conditions, medication, and GP consultations. GPs receive support to assist them with coding and also feedback about the quality of their recording.¹⁶ In the Netherlands, all Dutch inhabitants are registered with a GP and have mandatory health insurance. GP care is fully insured, therefore the threshold for consultation is low. Nine out of ten people aged ≥65 years visit their GP at least once a year, with an average of eight consultations per year.¹⁷

Administrative data were provided by Statistics Netherlands, the governmental institution responsible for processing statistical data in the Netherlands. These included demographic information and data on admissions to long-term care facilities and death. Data on admissions to hospital were derived from the Dutch Hospital Data (DHD) database, made available by Statistics Netherlands. In 2013 and 2014, DHD contained data from 87 out of 88 general and academic hospitals in the Netherlands.

Study population

The study population consisted of individuals aged ≥65 years, living at home, and registered uninterrupted in one practice between 1 January 2013 and 31 December 2013 (baseline period). To avoid potential noise from admissions to a long-term care facility and from deaths in predicting the outcome, individuals who did not experience an unplanned admission among those who died or were admitted to a long-term care facility within the prediction period were excluded from the analysis (see Supplementary Figure S1). The number of excluded individuals varied depending on the follow-up period (3, 6, and 12 months) (Figure 1).

Figure 1.

Flow of participants through study. LTCF = long-term care facility.

Outcome

The primary outcome was unplanned hospital admissions with ≥1 overnight stay within 6 months and derived from national administrative data. Admissions were defined as unplanned when immediate treatment or assistance within 24 h was necessary according to the medical specialist.¹⁸ Admissions without an overnight stay and admissions for psychiatric conditions were excluded as these often require different care trajectories. Secondary outcomes were unplanned admissions within 3 and 12 months.

Predictors

Updating existing prediction models was not feasible because of the incomparability of the predictors in this study’s dataset with the predictors in existing models, as well as the low methodological quality of these studies.¹² To (partially) incorporate information from existing models, in the current study variables commonly included in existing models were selected as candidate predictors for the model, for example, prior admissions to hospital and several chronic conditions.¹² In addition, variables were selected based on the insights from a focus group study that was organised among primary healthcare professionals (to be published) and based on the clinical expertise of the authors.

Ultimately, 29 candidate predictors were selected including age, sex, migration background, income, living situation, chronic conditions, prescription medications, and healthcare utilisation (see Supplementary Table S1 for a detailed description). Chronic conditions were derived from World Health Organization International Classification of Primary Care (ICPC-1)¹⁹ coded EHR data recorded up to the end of the baseline period. In NIVEL-PCD, GPs received feedback on the quality of recording and support to assist them with coding.¹⁶ Chronic conditions were selected because of their high prevalence in older adults.²⁰ Dementia was added because of its strong association with admissions to hospital.² Medication variables were derived from prescription data coded with the Anatomical Therapeutic Chemical classification system and included when a medication was prescribed in a chronic fashion (that is, >2 prescriptions²¹) in the year before baseline. Consult declarations (CTG-codes in Dutch) were derived from coded claims data recorded in general practices in the year before baseline.

Missing data

As the data were derived from routine care processes, any undocumented information in the EHR was not indicated. For the data provided by Statistics Netherlands, income data had missing values for 116 individuals (<0.01%). This justified conducting a complete case analysis considering the negligible proportion of missing data and the minimal potential impact on the results.¹²^,¹³^,²²^,²³

Statistical analysis

Linearity was assessed for continuous variables using restricted cubic splines.²⁴ Non-linear variables were tested as splines and as categorical variables in the logistic model. If the spline did not improve performance, the categorical variant was chosen because the authors wanted this to be a practical model. Collinearity was evaluated using variance inflation factors (VIFs). VIFs ranged between 1.01 and 2.43, therefore problematic collinearity was absent.²⁵

Model development

The large sample provided sufficient statistical power to split the sample into a development and validation sample based on geographic region. The larger sample, that is, the six southernmost provinces (n = 142 791/243 324, 58.7%), was used for development and the smaller sample for validation (see Supplementary Figure S2). Geographic validation is considered a stronger approach compared with a random split sample procedure.¹⁴^,²⁶

For model building, this study followed the recommended steps outlined in the TRIPOD guidelines¹⁴ and by Steyerberg.²⁷ Multivariable logistic regression with backward stepwise selection (P<0.01) was performed using all 29 candidate predictors to design an optimal model. Given the sample size, there was sufficient power to fit a more parsimonious model by incrementally removing the variables with weakest association, until the area under the curve (AUC) deteriorated by ≥0.01. Internal validation was performed through bootstrapping (n = 250).

This procedure was repeated twice with smaller subsets of candidate predictors to develop a model with only variables readily available from the EHR (readily available model) and with only easy-to-use variables (easy-to-use model), using 24 and 22 candidate predictors, respectively (see Supplementary Table S1). The easy-to-use model was designed to allow rapid completion by a GP; variables were therefore selected that are quick and easy to fill. All three models were validated in the northern sample.

Model performance

Discrimination was evaluated through AUC and calibration through calibration plots, intercept, and slope. The shrinkage factor was determined to quantify overfitting. Classification measures were also assessed, including sensitivity, specificity, and positive and negative predictive values, for multiple probability thresholds. The optimal probability threshold was determined using the Youden index.²⁸

Sensitivity analysis

Sensitivity analyses were undertaken in the optimal model to assess performance for different follow-up periods (3 and 12 months). Furthermore, performance was assessed in subsamples of people with cognitive decline or dementia (ICPC-1 P20 or P70). Additionally, model performance was evaluated in a sample including individuals who had died or been admitted to a long-term care facility within 6 months. Statistical analyses were performed using IBM SPSS Statistics (version 26) and R Studio (version 4.1.2) using packages rms, pROC, and psfmi.

Results

Participants

Overall, 243 324 individuals were included in the 6-month sample (development n = 142 791/243 324, 58.7% and n = 100 533/243 324, 41.3% validation sample) (Figure 1). Prevalence of candidate predictors and incidence of outcome are shown in Table 1. In both samples, median age was 72 years (interquartile range [IQR] 68–78); in the development and validation samples, 53.8% (n = 76 892/142 791) and 53.9% (n = 54 141/100 533) were female, and approximately 41.9% (n = 59 867/142 791) and 43.3% (n = 43 534/100 533) had ≥2 comorbidities, respectively, of which osteoarthritis was most prevalent. In total, 12.8% (n = 18 292/142 791) experienced ≥1 admission to hospital in the year before baseline in the development sample. In both samples, 7.6% (development cohort: n = 10 839/142 791, validation cohort: n = 7675/100 533) experienced ≥1 unplanned admissions to hospital within 6 months.

View this table:

Table 1.

Characteristics of candidate predictors in 6-month development and validation samples

Model development and validation

The optimal model included eight predictors: sex, age, prior admissions to hospital, chronic obstructive pulmonary disease (COPD), polypharmacy, use of blood thinners, number of GP or practice nurse consultations, and percentage of home visits compared with all contacts with a GP (Table 2). When applied to the validation sample, the AUC was 0.73 (95% confidence interval [CI] = 0.72 to 0.73). Youden’s optimal probability threshold was 0.07, reflecting a sensitivity of 65.7% and a specificity of 68.5% in the validation sample (Table 3 and Figure 2). Performance measures are reported for multiple probability thresholds to accommodate varying clinician preferences for risk estimation.

View this table:

Table 2.

The final prediction models from the multivariable logistic regression based on the development sample together with OR (95% CI) and the AUC in the development and validation samples

View this table:

Table 3.

Measures of predictive performance of the optimal model in the development and validation sample for multiple probability thresholds

Figure 2.

Graphical presentation of performance measures of the optimal model in the validation sample.

The readily available model contained all predictors of the optimal model except for prior admissions to hospital (Table 2). Compared with the optimal model, the AUC in the validation sample was marginally lower (AUC 0.72, 95% CI = 0.71 to 0.72).

The easy-to-use model included age, sex, admissions to hospital in the past year, heart failure, COPD, and polypharmacy (Table 2). When applied to the validation sample, this resulted in an AUC of 0.72 (95% CI = 0.71 to 0.72). To allow for individualised predictions of this model, a Microsoft Excel spreadsheet is provided as a supplement (see Supplementary Information S1).

For all three models, bootstrapping resulted in an optimism of the AUC, intercept, and slope <0.001, therefore no adjustments of the coefficients were required. Calibration of all models was good; the slope and intercept did not deviate to the extent that model updating was undertaken (see Supplementary Figure S3).

Clinical implications of choice of cut-off value

Choosing a cut-off value provides the opportunity to stratify patients into low- and high-risk groups. This facilitates clinical decision making. To illustrate this, a practice consisting of 500 community-dwelling patients aged ≥65 years was considered. The consequences of two different cut-off values (or probability thresholds) were compared: 0.07 and 0.15. A prior probability of 7.6% for each patient (prevalence) was assumed. The 2 × 2 contingency tables for both cut-offs are shown (Tables 4 and 5).

View this table:

Table 4.

Threshold of 0.07: 2 × 2 contingency table

View this table:

Table 5.

Threshold of 0.15: 2 × 2 contingency table

Using a cut-off of 0.07 stratifies approximately one-third of the practice’s older population as high risk, requiring screening or intervention. However, this choice results in a high number of false positives, where individuals are identified as high risk but do not experience the predicted outcome. At the individual level, using a threshold of 0.07 increases the probability of a patient being classified as high risk for unplanned hospital admissions by a factor of 2, from 7.6% to 14.9%. This means that, out of 100 high-risk patients, 15 will have an unplanned hospital admission within 6 months.

Alternatively, using a cut-off value of 0.15, one in ten older patients will be classified as high risk, resulting in a substantially lower number of false positives. However, the number of false negatives doubles, indicating that some potential patients are missed. For a high-risk patient at the 0.15 threshold, the probability of unplanned hospital admissions increases by a factor of 3 to 23.2%. Consequently, out of 100 high-risk patients, 23 would experience an unplanned hospital admission within 6 months.

How much risk a clinician is willing to take to avoid missing an unplanned admission will depend on the clinician’s judgement. Opting for a lower threshold results in a low number of false negatives, but raises the probability of false positives, requiring a more extensive and labour-intensive screening process.

Sensitivity analyses

Testing the optimal model in people with cognitive decline resulted in an AUC of 0.67 (95% CI = 0.65 to 0.69) in both samples. The optimal model showed good predictive ability when fitted in the 3- and 12-month follow-up samples. However, the calibration plots showed systematic over- and underestimation in the 3- and 12-month samples, respectively. Finally, evaluating the optimal model in a sample including those who died or were admitted to a long-term care facility within 6 months resulted in an AUC of 0.72 (95% CI = 0.72 to 0.73). See Supplementary Tables S2–S4 and Supplementary Figure S4 for details of these analyses.

Discussion

Summary

In this study, routinely recorded and linked health and census data were used to develop and validate an easy-to-use prediction model for unplanned admissions to hospital in community-dwelling older adults. Predictors associated with unplanned hospital admission included age, sex, admission to hospital in the past year, polypharmacy, the use of blood thinners, COPD, heart failure, number of consultations including telephone consultations and home visit contacts, and the percentage of home visits. The optimal model showed satisfactory discrimination and good calibration. Moreover, geographic validation, reducing the number of predictors, changing the prediction horizon, and including individuals who died or were admitted to long-term care facilities within the prediction period, all resulted in a negligible decrease in discriminative ability, demonstrating robustness of the model. This model lost discriminatory power in a subsample of individuals with dementia or cognitive decline.

These results should enable GPs to identify patients who may benefit from targeted admission prevention strategies. To improve predictions, the authors of the current study emphasise the importance of routine recording or incorporation of hospital admission data into the EHR.

Strengths and limitations

A strength of this study is the use of multiple approaches for model development, providing valuable insights into relative effectiveness and practical utility. By considering the advantages and limitations of each approach, healthcare providers and policymakers can make informed decisions about which model is suitable for their specific needs and resources. The use of EHR data enriched with national administrative data resulted in the best predictive model, that is, the optimal model. Using structured EHR data allows the readily available model to be implemented nationwide. However, it includes more time-consuming variables compared with the easy-to-use model. By facilitating rapid bedside assessment, the easy-to-use model is more accessible to GPs, while incorporating the most predictive variable: prior admissions to hospital. Furthermore, the large longitudinal sample and its nationwide representativeness suggests these findings could be generalised across the Netherlands.

This study also has limitations. As advocated in the literature,²⁹^,³⁰ updating an existing prediction model is preferred over simply developing a new model, so information from the previous models is not neglected. However, model updating is only valuable provided the original model’s development is appropriately performed, and variables and outcomes are determined in a similar way.²⁹ For this study, however, the low quality of reporting in the previous studies,¹² and the lack of several variables in the current dataset, made updating infeasible. Moreover, differences in care systems between countries complicate the transportability of existing models to other geographical populations,³¹ and no model had yet been developed in the Netherlands. Altogether, this large sample called for the development of a new model rather than updating an existing one. Nevertheless, to incorporate data from previous models as much as possible, the current study assessed the variables most frequently included in previous models as candidate predictors for inclusion in this model. Furthermore, although in this study the data are approximately 10 years old, the relevance remains. Reviews have shown the long-term trends and relative stability over time of included predictors of unplanned hospital admissions, such as prior healthcare use, chronic conditions, and polypharmacy.¹²^,¹³^,³²

Comparison with existing literature

Nineteen existing prediction models to predict admissions to hospital in older adults were identified.¹² The current model showed similar performance and overlap in the most commonly included variables. However, whereas many existing models used a 12-month prediction horizon, in the current study a 6-month horizon was chosen from a clinical perspective because a high predicted probability of hospital admission within 6 months is more likely to trigger timely clinical action than the same probability of hospital admission within 12 months. However, model validation over 12 months demonstrated equivalent discriminatory ability, albeit with systematic underestimation, requiring adjustment of the intercept.

Two previous studies developed a model for people with dementia in primary care, and both demonstrated good predictive performance. These studies found (changes in) psychotropic medication, psychiatric diagnoses, and hypertension to be important predictors, along with previous admissions to hospital and polypharmacy.³³^,³⁴ Other studies found duration and severity of dementia, caregiver burden, and continuity of care associated with hospital admissions.³⁵^–³⁷ To improve predictive accuracy in individuals with cognitive decline, these predictors may need to be considered for inclusion in the model.

Implications for research and practice

The implementation of ageing-in-place policies in the Netherlands in 2015, which included a reduction of the residential care capacity, has increased the number of older adults living in the community.³⁸ As this study used data from before the implementation of this policy, a different case mix of community-dwelling older adults may be expected. Therefore, validation in more recent data is recommended. Additionally, the authors emphasise the importance of systematic recording of admissions to hospital in the EHR to enable practical implementation and to provide the most accurate risk estimates, as prior admissions to hospital is the strongest predictor of future admissions to hospital.

The models in the current study may support timely identification of and proactive interventions for older patients at risk of unplanned hospital admissions. When selecting the appropriate cut-off value for targeting interventions, clinicians should prioritise factors such as patient preference, intervention time burden, and the trade-off between intervention benefits and potential missed patients. Finally, the models could assist policymakers with estimating the required number of hospital beds in the region.

Notes

Funding

The work was supported by the Netherlands Organization for Health Research and Development (ZonMw) (grant number: 733050403), Stichting PvE fonds, and the I-CARE4OLD project (grant agreement number: 96534). More information on the I-CARE4OLD project can be found at http://www.icare4old.eu and https://cordis.europa.eu/project/id/965341. All sponsors had no role in the design, methods, subject recruitment, data collection, analysis, or preparation of the study.

Ethical approval

This study has been approved according to the governance code of the NIVEL-Primary Care Database. This can be found under number NZR-00315.063. As pseudonymised data were used that were collected for routine administrative registration purposes, the informed consent of the participants was not necessary. Patients were informed by their GP about the use of their pseudonymised health data and were given the opportunity to object.

Data

The data set used for this study is not publicly available, because it cannot leave the environment of Statistics Netherlands according to their privacy regulations. Requests to access the data set should be directed to the principal investigator of the BESIDE study (k.joling{at}amsterdamumc.nl).

Provenance

Freely submitted; externally peer reviewed.

Competing interests

The authors have declared no competing interests.

Discuss this article:

bjgp.org/letters

Received July 13, 2023.
Revision requested October 17, 2023.
Accepted February 26, 2024.

http://creativecommons.org/licenses/by/4.0/

This article is Open Access: CC BY 4.0 licence (http://creativecommons.org/licences/by/4.0/).

References

1.↵
1. Long SJ,
2. Brown KF,
3. Ames D,
4. Vincent C
(2013) What is known about adverse events in older medical hospital inpatients? A systematic review of the literature. Int J Qual Health Care 25, 5, 542–554.
OpenUrl CrossRef PubMed
2.↵
1. Shepherd H,
2. Livingston G,
3. Chan J,
4. Sommerlad A
(2019) Hospitalisation rates and predictors in people with dementia: a systematic review and meta-analysis. BMC Med 17, 1, 130.
OpenUrl PubMed
3.
1. Fogg C,
2. Griffiths P,
3. Meredith P,
4. Bridges J
(2018) Hospital outcomes of older people with cognitive impairment: an integrative review. Int J Geriatr Psychiatry 33, 9, 1177–1197.
OpenUrl
4.↵
1. Lehmann J,
2. Michalowsky B,
3. Kaczynski A,
4. et al.
(2018) The impact of hospitalization on readmission, institutionalization, and mortality of people with dementia: a systematic review and meta-analysis. J Alzheimers Dis 64, 3, 735–749.
OpenUrl
5.↵
1. Araujo de Carvalho I,
2. Epping-Jordan J,
3. Pot AM,
4. et al.
(2017) Organizing integrated health-care services to meet older people’s needs. Bull World Health Organ 95, 11, 756–763.
OpenUrl CrossRef PubMed
6.↵
1. Ministry of Health, Welfare and Sport
(2022) [Integral care agreement: working together on healthy care]. [Article in Dutch], https://www.rijksoverheid.nl/documenten/rapporten/2022/09/16/integraal-zorgakkoord-samen-werken-aan-gezonde-zorg (accessed 6 Aug 2024).
7.↵
1. Baker A,
2. Leak P,
3. Ritchie LD,
4. et al.
(2012) Anticipatory care planning and integration: a primary care pilot study aimed at reducing unplanned hospitalisation. Br J Gen Pract, DOI: https://doi.org/10.3399/bjgp12X625175.
8.
1. Pritchard C,
2. Ness A,
3. Symonds N,
4. et al.
(2020) Effectiveness of hospital avoidance interventions among elderly patients: a systematic review. CJEM 22, 4, 504–513.
OpenUrl PubMed
9.
1. Costa AP,
2. Haughton D,
3. Heckman G,
4. et al.
(2017) The Divert-Care Catalyst trial: targeted chronic-disease management for home care clients. Innov Aging 1, Suppl 1, 322–323.
OpenUrl
10.↵
1. Nord M,
2. Lyth J,
3. Alwin J,
4. Marcusson J
(2021) Costs and effects of comprehensive geriatric assessment in primary care for older adults with high risk for hospitalisation. BMC Geriatr 21, 1, 263.
OpenUrl CrossRef PubMed
11.↵
1. Starfield B,
2. Shi L,
3. Macinko J
(2005) Contribution of primary care to health systems and health. Milbank Q 83, 3, 457–502.
OpenUrl CrossRef PubMed
12.↵
1. Klunder JH,
2. Panneman SL,
3. Wallace E,
4. et al.
(2022) Prediction models for the prediction of unplanned hospital admissions in community-dwelling older adults: a systematic review. PLoS One 17, 9, e0275116.
OpenUrl
13.↵
1. Wallace E,
2. Stuart E,
3. Vaughan N,
4. et al.
(2014) Risk prediction models to predict emergency hospital admission in community-dwelling adults: a systematic review. Med Care 52, 8, 751–765.
OpenUrl CrossRef PubMed
14.↵
1. Moons KGM,
2. Altman DG,
3. Reitsma JB,
4. et al.
(2015) Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 162, 1, W1–W73.
OpenUrl CrossRef PubMed
15.↵
1. Donker GA
(2014) [NIVEL primary care database — sentinel practices]. [Article in Dutch], https://www.nivel.nl/nl/publicatie/nivel-zorgregistraties-eerste-lijn-peilstations-2013 (accessed 6 Aug 2024).
16.↵
1. Verheij RA,
2. Curcin V,
3. Delaney BC,
4. McGilchrist MM
(2018) Possible sources of bias in primary care electronic health record data use and reuse. J Med Internet Res 20, 5, e185.
OpenUrl CrossRef PubMed
17.↵
1. Nielen M,
2. Hek K,
3. Weesie Y,
4. et al.
(2020) How often do Dutch people contact GP practices? Care use in general practice in 2019 (Nivel, Utrecht).
18.↵
1. Statistics/Netherlands
(2023) Report on Dutch hospital discharge register (LBZBASISTAB).
19.↵
1. Lamberts H,
2. Wood M
(1987) International classification of primary care (Oxford University Press, Oxford).
20.↵
1. van Oostrom SH,
2. Picavet HSJ,
3. van Gelder BM,
4. et al.
(2012) Multimorbidity and comorbidity in the Dutch population — data from general practices. BMC Public Health 12, 715.
OpenUrl CrossRef PubMed
21.↵
1. Verduijn M,
2. Leendertse A,
3. Moeselaar A,
4. et al.
(2013) [Multidisciplinary guideline for polypharmacy in older adults]. [Article in Dutch], https://researchinformation.amsterdamumc.org/en/publications/multidisciplinaire-richtlijn-polyfarmacie-bij-ouderen (accessed 6 Aug 2024).
22.↵
1. Jakobsen JC,
2. Gluud C,
3. Wetterslev J,
4. Winkel P
(2017) When and how should multiple imputation be used for handling missing data in randomised clinical trials — a practical guide with flowcharts. BMC Med Res Methodol 17, 1, 162.
OpenUrl CrossRef PubMed
23.↵
1. White IR,
2. Royston P,
3. Wood AM
(2011) Multiple imputation using chained equations: issues and guidance for practice. Stat Med 30, 4, 377–399.
OpenUrl CrossRef PubMed
24.↵
1. Marrie RA,
2. Dawson NV,
3. Garland A
(2009) Quantile regression and restricted cubic splines are useful for exploring relationships between continuous variables. J Clin Epidemiol 62, 5, 511–517.e1.
OpenUrl CrossRef PubMed
25.↵
1. Fox J
(2008) Applied regression analysis and generalized linear models (Sage Publications, Thousand Oaks, CA), 2nd edn.
26.↵
1. Steyerberg EW
(2019) Validation of prediction models. Clinical prediction models: a practical approach to development, validation, and updating (Springer International Publishing, Cham), 329–344.
27.↵
1. Steyerberg EW
(2019) Selection of main effects. Clinical prediction models: a practical approach to development, validation, and updating (Springer International Publishing, Cham), 207–225.
28.↵
1. Habibzadeh F,
2. Habibzadeh P,
3. Yadollahie M
(2016) On determining the most appropriate test cut-off value: the case of tests with continuous results. Biochem Med (Zagreb) 26, 3, 297–307.
OpenUrl
29.↵
1. Janssen KJM,
2. Moons KGM,
3. Kalkman CJ,
4. et al.
(2008) Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol 61, 1, 76–86.
OpenUrl CrossRef PubMed
30.↵
1. Binuya MAE,
2. Engelhardt EG,
3. Schats W,
4. et al.
(2022) Methodological guidance for the evaluation and updating of clinical prediction models: a systematic review. BMC Med Res Methodol 22, 1, 316.
OpenUrl CrossRef PubMed
31.↵
1. Klunder JH,
2. Bordonis V,
3. Heymans MW,
4. et al.
(2021) Predicting unplanned hospital visits in older home care recipients: a cross-country external validation study. BMC Geriatr 21, 1, 551.
OpenUrl
32.↵
1. O’Caoimh R,
2. Cornally N,
3. Weathers E,
4. et al.
(2015) Risk prediction in the community: a systematic review of case-finding instruments that predict adverse healthcare outcomes in community-dwelling older adults. Maturitas 82, 1, 3–21.
OpenUrl PubMed
33.↵
1. Maust DT,
2. Kim HM,
3. Chiang C,
4. et al.
(2019) Predicting risk of potentially preventable hospitalization in older adults with dementia. J Am Geriatr Soc 67, 10, 2077–2084.
OpenUrl PubMed
34.↵
1. Tsang G,
2. Zhou S-M,
3. Xie X
(2020) Modeling large sparse data for feature selection: hospital admission predictions of the dementia patients using primary care electronic health records. IEEE J Transl Eng Health Med 9, 3000113.
OpenUrl
35.↵
1. Gungabissoon U,
2. Perera G,
3. Galwey NW,
4. Stewart R
(2022) Potentially avoidable causes of hospitalisation in people with dementia: contemporaneous associations by stage of dementia in a South London clinical cohort. BMJ Open 12, 4, e055447.
OpenUrl Abstract/FREE Full Text
36.
1. Afonso-Argilés FJ,
2. Meyer G,
3. Stephan A,
4. et al.
(2020) Emergency department and hospital admissions among people with dementia living at home or in nursing homes: results of the European RightTimePlaceCare project on their frequency, associated factors and costs. BMC Geriatr 20, 1, 453.
OpenUrl PubMed
37.↵
1. Rudolph JL,
2. Zanin NM,
3. Jones RN,
4. et al.
(2010) Hospitalization in community-dwelling persons with Alzheimer’s disease: frequency and causes. J Am Geriatr Soc 58, 8, 1542–1548.
OpenUrl CrossRef PubMed
38.↵
1. Maarse JAM,
2. Jeurissen PP
(2016) The policy and politics of the 2015 long-term care reform in the Netherlands. Health Policy 120, 3, 241–245.
OpenUrl CrossRef PubMed

In this issue

Download PDF

Download PowerPoint

Email Article

Citation Tools

Keywords

More in this TOC Section

Show more Research

Cited By...

Intended for Healthcare Professionals

Tweets by @BJGPjournal

[1] 1.↵
Long SJ,
Brown KF,
Ames D,
Vincent C
(2013) What is known about adverse events in older medical hospital inpatients? A systematic review of the literature. Int J Qual Health Care 25, 5, 542–554.
OpenUrl CrossRef PubMed

[2] Long SJ,

[3] Brown KF,

[4] Ames D,

[5] Vincent C

[6] 2.↵
Shepherd H,
Livingston G,
Chan J,
Sommerlad A
(2019) Hospitalisation rates and predictors in people with dementia: a systematic review and meta-analysis. BMC Med 17, 1, 130.
OpenUrl PubMed

[7] Shepherd H,

[8] Livingston G,

[9] Chan J,

[10] Sommerlad A

[11] 3.
Fogg C,
Griffiths P,
Meredith P,
Bridges J
(2018) Hospital outcomes of older people with cognitive impairment: an integrative review. Int J Geriatr Psychiatry 33, 9, 1177–1197.
OpenUrl

[12] Fogg C,

[13] Griffiths P,

[14] Meredith P,

[15] Bridges J

[16] 4.↵
Lehmann J,
Michalowsky B,
Kaczynski A,
et al.
(2018) The impact of hospitalization on readmission, institutionalization, and mortality of people with dementia: a systematic review and meta-analysis. J Alzheimers Dis 64, 3, 735–749.
OpenUrl

[17] Lehmann J,

[18] Michalowsky B,

[19] Kaczynski A,

[20] et al.

[21] 5.↵
Araujo de Carvalho I,
Epping-Jordan J,
Pot AM,
et al.
(2017) Organizing integrated health-care services to meet older people’s needs. Bull World Health Organ 95, 11, 756–763.
OpenUrl CrossRef PubMed

[22] Araujo de Carvalho I,

[23] Epping-Jordan J,

[24] Pot AM,

[25] et al.

[26] 6.↵
Ministry of Health, Welfare and Sport
(2022) [Integral care agreement: working together on healthy care]. [Article in Dutch], https://www.rijksoverheid.nl/documenten/rapporten/2022/09/16/integraal-zorgakkoord-samen-werken-aan-gezonde-zorg (accessed 6 Aug 2024).

[27] Ministry of Health, Welfare and Sport

[28] 7.↵
Baker A,
Leak P,
Ritchie LD,
et al.
(2012) Anticipatory care planning and integration: a primary care pilot study aimed at reducing unplanned hospitalisation. Br J Gen Pract, DOI: https://doi.org/10.3399/bjgp12X625175.

[29] Baker A,

[30] Leak P,

[31] Ritchie LD,

[32] et al.

[33] 8.
Pritchard C,
Ness A,
Symonds N,
et al.
(2020) Effectiveness of hospital avoidance interventions among elderly patients: a systematic review. CJEM 22, 4, 504–513.
OpenUrl PubMed

[34] Pritchard C,

[35] Ness A,

[36] Symonds N,

[37] et al.

[38] 9.
Costa AP,
Haughton D,
Heckman G,
et al.
(2017) The Divert-Care Catalyst trial: targeted chronic-disease management for home care clients. Innov Aging 1, Suppl 1, 322–323.
OpenUrl

[39] Costa AP,

[40] Haughton D,

[41] Heckman G,

[42] et al.

[43] 10.↵
Nord M,
Lyth J,
Alwin J,
Marcusson J
(2021) Costs and effects of comprehensive geriatric assessment in primary care for older adults with high risk for hospitalisation. BMC Geriatr 21, 1, 263.
OpenUrl CrossRef PubMed

[44] Nord M,

[45] Lyth J,

[46] Alwin J,

[47] Marcusson J

[48] 11.↵
Starfield B,
Shi L,
Macinko J
(2005) Contribution of primary care to health systems and health. Milbank Q 83, 3, 457–502.
OpenUrl CrossRef PubMed

[49] Starfield B,

[50] Shi L,

[51] Macinko J

[52] 12.↵
Klunder JH,
Panneman SL,
Wallace E,
et al.
(2022) Prediction models for the prediction of unplanned hospital admissions in community-dwelling older adults: a systematic review. PLoS One 17, 9, e0275116.
OpenUrl

[53] Klunder JH,

[54] Panneman SL,

[55] Wallace E,

[56] et al.

[57] 13.↵
Wallace E,
Stuart E,
Vaughan N,
et al.
(2014) Risk prediction models to predict emergency hospital admission in community-dwelling adults: a systematic review. Med Care 52, 8, 751–765.
OpenUrl CrossRef PubMed

[58] Wallace E,

[59] Stuart E,

[60] Vaughan N,

[61] et al.

[62] 14.↵
Moons KGM,
Altman DG,
Reitsma JB,
et al.
(2015) Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 162, 1, W1–W73.
OpenUrl CrossRef PubMed

[63] Moons KGM,

[64] Altman DG,

[65] Reitsma JB,

[66] et al.

[67] 15.↵
Donker GA
(2014) [NIVEL primary care database — sentinel practices]. [Article in Dutch], https://www.nivel.nl/nl/publicatie/nivel-zorgregistraties-eerste-lijn-peilstations-2013 (accessed 6 Aug 2024).

[68] Donker GA

[69] 16.↵
Verheij RA,
Curcin V,
Delaney BC,
McGilchrist MM
(2018) Possible sources of bias in primary care electronic health record data use and reuse. J Med Internet Res 20, 5, e185.
OpenUrl CrossRef PubMed

[70] Verheij RA,

[71] Curcin V,

[72] Delaney BC,

[73] McGilchrist MM

[74] 17.↵
Nielen M,
Hek K,
Weesie Y,
et al.
(2020) How often do Dutch people contact GP practices? Care use in general practice in 2019 (Nivel, Utrecht).

[75] Nielen M,

[76] Hek K,

[77] Weesie Y,

[78] et al.

[79] 18.↵
Statistics/Netherlands
(2023) Report on Dutch hospital discharge register (LBZBASISTAB).

[80] Statistics/Netherlands

[81] 19.↵
Lamberts H,
Wood M
(1987) International classification of primary care (Oxford University Press, Oxford).

[82] Lamberts H,

[83] Wood M

[84] 20.↵
van Oostrom SH,
Picavet HSJ,
van Gelder BM,
et al.
(2012) Multimorbidity and comorbidity in the Dutch population — data from general practices. BMC Public Health 12, 715.
OpenUrl CrossRef PubMed

[85] van Oostrom SH,

[86] Picavet HSJ,

[87] van Gelder BM,

[88] et al.

[89] 21.↵
Verduijn M,
Leendertse A,
Moeselaar A,
et al.
(2013) [Multidisciplinary guideline for polypharmacy in older adults]. [Article in Dutch], https://researchinformation.amsterdamumc.org/en/publications/multidisciplinaire-richtlijn-polyfarmacie-bij-ouderen (accessed 6 Aug 2024).

[90] Verduijn M,

[91] Leendertse A,

[92] Moeselaar A,

[93] et al.

[94] 22.↵
Jakobsen JC,
Gluud C,
Wetterslev J,
Winkel P
(2017) When and how should multiple imputation be used for handling missing data in randomised clinical trials — a practical guide with flowcharts. BMC Med Res Methodol 17, 1, 162.
OpenUrl CrossRef PubMed

[95] Jakobsen JC,

[96] Gluud C,

[97] Wetterslev J,

[98] Winkel P

[99] 23.↵
White IR,
Royston P,
Wood AM
(2011) Multiple imputation using chained equations: issues and guidance for practice. Stat Med 30, 4, 377–399.
OpenUrl CrossRef PubMed

[100] White IR,

[101] Royston P,

[102] Wood AM

[103] 24.↵
Marrie RA,
Dawson NV,
Garland A
(2009) Quantile regression and restricted cubic splines are useful for exploring relationships between continuous variables. J Clin Epidemiol 62, 5, 511–517.e1.
OpenUrl CrossRef PubMed

[104] Marrie RA,

[105] Dawson NV,

[106] Garland A

[107] 25.↵
Fox J
(2008) Applied regression analysis and generalized linear models (Sage Publications, Thousand Oaks, CA), 2nd edn.

[108] Fox J

[109] 26.↵
Steyerberg EW
(2019) Validation of prediction models. Clinical prediction models: a practical approach to development, validation, and updating (Springer International Publishing, Cham), 329–344.

[110] Steyerberg EW

[111] 27.↵
Steyerberg EW
(2019) Selection of main effects. Clinical prediction models: a practical approach to development, validation, and updating (Springer International Publishing, Cham), 207–225.

[112] Steyerberg EW

[113] 28.↵
Habibzadeh F,
Habibzadeh P,
Yadollahie M
(2016) On determining the most appropriate test cut-off value: the case of tests with continuous results. Biochem Med (Zagreb) 26, 3, 297–307.
OpenUrl

[114] Habibzadeh F,

[115] Habibzadeh P,

[116] Yadollahie M

[117] 29.↵
Janssen KJM,
Moons KGM,
Kalkman CJ,
et al.
(2008) Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol 61, 1, 76–86.
OpenUrl CrossRef PubMed

[118] Janssen KJM,

[119] Moons KGM,

[120] Kalkman CJ,

[121] et al.

[122] 30.↵
Binuya MAE,
Engelhardt EG,
Schats W,
et al.
(2022) Methodological guidance for the evaluation and updating of clinical prediction models: a systematic review. BMC Med Res Methodol 22, 1, 316.
OpenUrl CrossRef PubMed

[123] Binuya MAE,

[124] Engelhardt EG,

[125] Schats W,

[126] et al.

[127] 31.↵
Klunder JH,
Bordonis V,
Heymans MW,
et al.
(2021) Predicting unplanned hospital visits in older home care recipients: a cross-country external validation study. BMC Geriatr 21, 1, 551.
OpenUrl

[128] Klunder JH,

[129] Bordonis V,

[130] Heymans MW,

[131] et al.

[132] 32.↵
O’Caoimh R,
Cornally N,
Weathers E,
et al.
(2015) Risk prediction in the community: a systematic review of case-finding instruments that predict adverse healthcare outcomes in community-dwelling older adults. Maturitas 82, 1, 3–21.
OpenUrl PubMed

[133] O’Caoimh R,

[134] Cornally N,

[135] Weathers E,

[136] et al.

[137] 33.↵
Maust DT,
Kim HM,
Chiang C,
et al.
(2019) Predicting risk of potentially preventable hospitalization in older adults with dementia. J Am Geriatr Soc 67, 10, 2077–2084.
OpenUrl PubMed

[138] Maust DT,

[139] Kim HM,

[140] Chiang C,

[141] et al.

[142] 34.↵
Tsang G,
Zhou S-M,
Xie X
(2020) Modeling large sparse data for feature selection: hospital admission predictions of the dementia patients using primary care electronic health records. IEEE J Transl Eng Health Med 9, 3000113.
OpenUrl

[143] Tsang G,

[144] Zhou S-M,

[145] Xie X

[146] 35.↵
Gungabissoon U,
Perera G,
Galwey NW,
Stewart R
(2022) Potentially avoidable causes of hospitalisation in people with dementia: contemporaneous associations by stage of dementia in a South London clinical cohort. BMJ Open 12, 4, e055447.
OpenUrl Abstract/FREE Full Text

[147] Gungabissoon U,

[148] Perera G,

[149] Galwey NW,

[150] Stewart R

[151] 36.
Afonso-Argilés FJ,
Meyer G,
Stephan A,
et al.
(2020) Emergency department and hospital admissions among people with dementia living at home or in nursing homes: results of the European RightTimePlaceCare project on their frequency, associated factors and costs. BMC Geriatr 20, 1, 453.
OpenUrl PubMed

[152] Afonso-Argilés FJ,

[153] Meyer G,

[154] Stephan A,

[155] et al.

[156] 37.↵
Rudolph JL,
Zanin NM,
Jones RN,
et al.
(2010) Hospitalization in community-dwelling persons with Alzheimer’s disease: frequency and causes. J Am Geriatr Soc 58, 8, 1542–1548.
OpenUrl CrossRef PubMed

[157] Rudolph JL,

[158] Zanin NM,

[159] Jones RN,

[160] et al.

[161] 38.↵
Maarse JAM,
Jeurissen PP
(2016) The policy and politics of the 2015 long-term care reform in the Netherlands. Health Policy 120, 3, 241–245.
OpenUrl CrossRef PubMed

[162] Maarse JAM,

[163] Jeurissen PP

Main menu

User menu

Search

Predicting unplanned admissions to hospital in older adults using routinely recorded general practice data: development and validation of a prediction model

Abstract

Introduction

Method

Sources of data

Study population

Outcome

Predictors

Missing data

Statistical analysis

Model development

Model performance

Sensitivity analysis

Results

Participants

Model development and validation

Clinical implications of choice of cut-off value

Sensitivity analyses

Discussion

Summary

Strengths and limitations

Comparison with existing literature

Implications for research and practice

Notes

Funding

Ethical approval

Data

Provenance

Competing interests

Discuss this article:

References

In this issue

Citation Manager Formats

Jump to section

Keywords

More in this TOC Section

Related Articles

Cited By...

NAVIGATE

RCGP

MY ACCOUNT

NEWS AND UPDATES

AUTHORS & REVIEWERS

CUSTOMER SERVICES

CONTRIBUTE

CONTACT US