Abstract
Background There is debate about the efficacy of screening and case detection for common mental health problems in general practice. There is enthusiasm for screening, but also conflicting evidence for its effectiveness and little research concerning implementation.
Aim To explore the efficacy of screening and case-finding for high prevalence psychological disorders in routine general practice.
Design of study A cross-sectional survey (part of the MaGPie study).
Setting Lower North Island of New Zealand.
Method Consecutive patients from a random sample of GPs were screened using the General Health Questionnaire (GHQ-12). A stratified random sample of patients was selected, based on GHQ strata, and invited to participate in an indepth interview to assess their psychological health. Seventy GPs (90% response) and 775 patients (68.5% response) were included in analyses. Patients' GHQ-12 scores were compared with GP assessment of patients' psychological health using a 5-point scale of severity, and with the Composite International Diagnostic Interview (CIDI 1-month assessment).
Results Overall 17.5% (95% CI = 13.8 to 21.1) of patients reached the threshold for CIDI diagnosis. GPs identified at least some psychological symptoms in the past 12 months in 70.3% (95% CI = 58.2 to 82.3) of patients reaching the CIDI threshold for diagnosis. Case-finding with a ‘perfect’ screening instrument had the potential to identify only five new cases per 100. The sensitivity and specificity of the GHQ-12 suggest it is not an appropriate tool for case finding in a general practice setting.
Conclusion The potential value of any screening and case-finding instrument in routine general practice must be considered in the context of current high rates of identification. Where continuity of care is a feature of usual management, case-finding is most efficient when focused on patients the GP has not seen in the past year.
INTRODUCTION
Research into screening and case detection for common mental health problems in general practice has produced conflicting results and raised questions about the practical implementation of any proposed screening tools. There has been enthusiasm for the use of screening both in the form of questionnaires1 and incorporation of specific standard questions into the consultation.2 While the World Health Organisation advocates that every patient in primary care should participate in a mental health screening process with the completion of WHO-5 in the waiting room as a standard first step,3 there has been relatively little research looking at the practical implications of such strategies.
Concern has been expressed over a number of years as to whether screening or case finding influences diagnosis or management and little evidence has been produced for the effectiveness of screening in practice.4 A 10-year review of the validity and clinical utility of depression screening documented the validity of screening instruments compared with formal criteria. Only seven outcome studies relating to depression screening instruments were found and none showed measurable benefit in a screened population.5 A second systematic review examined the effect of routinely administered psychiatric questionnaires on the recognition, management, and outcome of psychiatric disorders in non-psychiatric settings. The authors concluded that the routine measurement of outcome is a costly exercise with little benefit in improving psychosocial outcomes of those with psychiatric disorders managed in non-psychiatric settings.6 The adoption of screening programmes even using self-administered questionnaires and provider follow up has considerable cost implications, with limited evidence suggesting cost-effectiveness only when patients were screened on a single occasion.7
How this fits in
There is currently debate about the efficacy of screening and case detection for common mental health problems in general practice. While there is much enthusiasm for screening, there is little evidence for its effectiveness and little research concerning implementation. In this study GPs had previously (in the last 12 months) identified psychological symptoms in 70.3% of patients reaching a diagnostic threshold. Case-finding with a hypothetical screening instrument with perfect sensitivity and specificity would have the potential to identify only five new and unidentified cases per 100 patients screened. The potential value of any screening and case-finding instrument in routine general practice must be considered in the context of current high rates of identification, especially among patients who had already consulted in the preceding 12 months.
A recent study also suggests that while there is limited usefulness from routine screening the use of screening instruments might be to detect cases in a population who had been deemed to be at high risk from some kind of pre-screening, possibly using a limited number of questions.8
A wide range of instruments has been proposed for screening and subsequent case detection. The general health questionnaire (GHQ) has been used widely in primary care research to detect psychological symptoms, and has been shown to increase identification of patients with emotional distress.9 The GHQ also has wide acceptance as a screening instrument,10 although recognition of the high rate of false positive results and low positive predictive value have led to suggestions that it be combined with other instruments or incorporated into a self-complete computerised format.11
The bulk of the screening and case detection literature using both the GHQ and other putative screening tools imply that considerable change would be required by GPs and within general practice before screening could be successfully implemented. While the rationale for screening is predicated on an assumption that GPs fail to detect many of the mental health problems they encounter,12-14 there is little information on the potential value of screening and case finding approaches when some of the foundations for practice, such as continuity of care, are taken into account. Our previous publications have associated GP recognition of mental health disorders with frequency of consultations in the preceding year, and demonstrated that GPs miss few cases of disorder in patients they know.15
This paper uses data from a cross-sectional survey of GP attenders to consider the value of screening for common mental health problems in general practice by estimating the number of additional patients who would be identified using the GHQ-12 as a screening tool for all patients, and for new or infrequently attending patients.
METHOD
Data were collected as part of the cross-sectional phase of the MaGPie study, a study of the prevalence, outcomes and management of common mental illness in New Zealand general practice. Methods are described in detail elsewhere.15
Ninety GPs were randomly selected from a list of all 299 known eligible GPs in a geographical area in the lower North Island of New Zealand, yielding a mix of urban, small town and rural practices. GPs were eligible to participate if they were currently practicing at least half time without restriction (such as that due to ill health or compulsory supervision).
The GHQ-12 was completed by a total of 3414 consecutive eligible adult patients, approximately 50 from the practice of each participating GP. Patients were eligible if they were ≥18 years, read English well enough to understand and complete the GHQ-12, and were about to consult with the index GP for their own health concerns. Based on GHQ-12 strata, 8% of patients with scores of 0–1, 22% of patients with scores of 2–4 and all patients with scores of ≥5 were invited to participate in an in-depth interview and subsequent 12-month longitudinal study. The measures used in the indepth interview were based on the World Health Organisation's Collaborative Study of Psychological Problems in General Health Care16 and included a 12-month computerised interviewer-delivered version of the Composite International Diagnostic Interview (CIDI) version 2.117 as well as a range of sociodemographic questions.
The GP completed a questionnaire (the Encounter Form) for every patient aged ≥18 years who was seen during the study period. The Encounter Form included 5-point rating scales of the extent to which the presenting symptoms were physical or psychological, and an assessment of the overall severity of the patient's physical and psychological disorders in the past 12 months. A more detailed patient management questionnaire about problems, history, and current management was completed for each patient selected for the in-depth interviews, where the GP considered there was a psychological component to the patient's presentation.
Data for this paper were drawn from responses to the GHQ-12, the CIDI and the GPs' assessment of the patient's psychological health for the subset of 1132 patients who were eligible to participate in the longitudinal stage of the study.
GP identification
GP identification of psychological problems in the past 12 months were defined using data from two sources: the GP's Encounter Form rating of severity of psychological disorder, and the GP's Patient Management Questionnaire about psychological disorders diagnosed in the past 12 months. GP identification was defined as any report of psychological symptoms, distress or disorder whatsoever.
GHQ-12 cases
GHQ ‘cases’ were considered using the ‘0011’ scoring method18 with a range of cut off points.
Composite International Diagnostic Interview (CIDI) cases
Data were scored using WHO algorithms to produce DSM-IV diagnoses from CIDI v2.1. Data reported in this paper are for a CIDI diagnosis in the last month.
Statistical methods
Statistical analyses were carried out using Statistical Analysis Software (SAS) version 8.2. Data were weighted to adjust for differences in probability of being sampled using the method of Kish.19 Weighted prevalence estimates were derived using the SAS procedure SURVEYMEANS, which also adjusted standard errors and 95% confidence intervals (CIs) for the effects of clustering within general practice. Estimates of sensitivity, specificity and predictive value were calculated from a pseudo-frequency based on weighted numbers.
RESULTS
Seventy of the 78 (90%) eligible randomly selected GPs agreed to participate. GHQ screening questionnaires were completed by 3414 of 3687 (93%) eligible general practice attenders. Of the 1151 selected for interview based on the GHQ strata; 311 refused further contact, 19 became ineligible for the indepth interview (because of limited language skills or worsening illness) and 33 were not traceable, yielding 788 (69.6%) interviews. Two interviews were lost after completion, leaving 786 interviews. Eleven patients did not consent to their data being linked with data from their GP, leaving 775 patients for inclusion in the analyses in this paper.
Overall 17.5% (95% CI = 13.8 to 21.1) of patients reached the threshold for CIDI diagnosis (one assessment). GPs identified at least some symptoms of psychological disorders in the past 12 months in 70.3% (95% CI = 58.2 to 82.3) of these patients; 74.7% (95% CI = 62.1 to 87.3) had a GHQ-12 score of three or more and 37.1% (95% CI = 27.6 to 46.6) had a GHQ-12 score of eight or more.
On this sample, if GPs administered a perfect case-finding instrument (with 100% sensitivity and specificity) to 100 patients in order to increase their rate of recognition of patients with psychological symptoms, they would detect five additional patients reaching the threshold for CIDI diagnosis. If they used the GHQ-12 to identify cases in 100 patients, three patients who were CIDI positive and seven who were CIDI negative would be detected using a GHQ-12 threshold of three or more. Using a threshold of eight or more, one additional patient who was CIDI positive would be detected and one patient who was CIDI negative (Figure 1).
Figure 1 Case finding when the GHQ-12 is administered to all patients attending a general practice.
In comparison, 29.1% (95% CI = 16.1 to 42.1) of patients who had not been seen by the GP in the past 12 months reached the threshold for CIDI diagnosis (1-month assessment). GPs identified at least some symptoms of psychological disorders in the past 12 months in 32.8% (95% CI = 10.9 to 54.8%) of those patients; of whom 76.1% (95% CI = 48.3 to 100.0) had a GHQ-12 score of three or more and 17.8% (95%CI = 2.2 to 33.5) had a GHQ-12 score of eight or more.
If the GP administered a perfect case-finding instrument only to patients not seen in the past 12 months, for every 100 patients they would detect an additional 19 patients reaching the threshold for CIDI diagnosis. If the GHQ-12 was used to identify cases, for every 100 patients the GP had not seen in the past 12 months, an additional seven patients who were CIDI positive and four who were CIDI negative would be detected using a GHQ-12 threshold of three or more. Using a threshold of eight or more, one additional patient who was CIDI positive and one patient who was CIDI negative would be detected (Figure 2).
Figure 2 Case finding when the GHQ-12 is administered only to patients the GP has not seen in the past 12 months.
The sensitivity, specificity, positive and negative predictive values of the GHQ-12 using the CIDI (1-month assessment) as a ‘gold-standard’ are shown in Table 1.
Table 1 The sensitivity, specificity and predictive value of the GHQ-12 as a case-finding instrument in routine general practice.
DISCUSSION
Summary of main findings
This paper places the debate about screening and case finding for psychological problems within the context of general practice continuity of care. In routine practice GPs are effective at identifying mental health problems in patients they know. Assuming that a CIDI diagnosis within the last month is taken as an indication of a need for care, GPs identify more patients than they miss, and missing a ‘case’ is associated with a lower frequency of consultation.20 Relatively few additional cases of disorder would be identified if a routine screening procedure was used for all consulting patients.
In this sample, one additional patient with current DSM-IV disorder would be identified for every 20 patients screened if the test had perfect sensitivity and specificity. However, if GPs screened only patients they were seeing for the first time one additional patient would be detected for every five patients screened.
Strengths and limitations of the study
A key strength of the current study is the availability of data about the GP's assessment of psychological disorder to compare with GHQ-12 scores and an independent assessment of the patients mental health in the past one month using the CIDI. This data enables an estimate of the additional number of patients who could potentially be identified by screening in the context of routine general practice.
The GP's recruited into the study were aware that the content concerned mental health problems in general practice settings. This raises the possibility that they would have performed better than their usual practice, and introduced a ‘Hawthorne effect’. We feel, however, the impact of this is minimised by the random sample selection of GPs, high participation rate and the 3-day duration of recruitment.
Comparison with existing studies
Several recent studies have advocated, particularly for depression, shortened versions of a variety of instruments for screening and case finding.2,21 For example, Arroll et al used two questions (asked verbally) to achieve much better sensitivity (97%) and specificity (67%) than that achieved by the GHQ in this study.2 However, it is likely that many of those patients would already have been known to the GP, and the sustainability and long-term benefits of such approaches remain unknown.
The GHQ-12 does not appear to be a very sensitive or specific screening tool, although it has frequently been suggested for this purpose.9,22 In this instance use of the GHQ-12 to screen would have little effect on the management of common mental illness as most of the patients unrecognised had low or medium GHQ-12 scores.
Implications for future research and policy
Based on the results of this study we suggest that the debate regarding screening for common mental health disorders should take continuity of care into account. While there is some evidence that patients may welcome the idea of screening for mental health problems,23 questions remain as to how feasible even very brief screening tools would be outside of research settings. Time is a barrier to screening. This has been reported in previous studies involving focus group perceptions from primary care physicians.24 Although the individual screening demands of ‘special’ interest groups, promoting routine screening for cancer, or mental health problems may appear reasonable, when added together they provide a significant additional load on routine consultations. Routine screening may then militate against the ability to prioritise consultations where it is known there is a high likelihood of co-existent mental health problems, such as patients with diabetes or other chronic physical problems. In the general practice environment of this study it appears that GPs are aware of the great majority of mental health problems in the patients they see on a regular basis.
Effective management of mental health disorders clearly involves more than just recognition. Taking action may not be possible in the context of one consultation because of competing demands,25 patient lack of acceptance of mental health problems or other barriers to care such as cost in a fee-for-service system such as New Zealand's.
There is renewed interest in the importance of continuity of care in general practice26-29 with advantages claimed for outcomes in chronic disease management and preventive care. Where continuity of care is an integral part of routine general practice, the greatest efficiencies in the detection of those with common mental health disorders will come from selective screening of patients not well known to their personal physician. Based on the results of this survey, a cost-effective compromise is to screen new and infrequently seen patients and to restrict more routine screening to situations where continuity is not a feature of care such as after hours or urgent and emergency medical clinics.
Acknowledgments
The MaGPIe (Mental Health and General Practice Investigation) research group consists of a management committee and an advisory committee. The management committee that undertook day-to-day oversight and management of this study consisted of John Bushnell, Deborah McLeod, Anthony Dowell, Clare Salmond, and Stella Ramage. The advisory committee consisted of Sunny Collings, Pete Ellis, Marjan Kljakovic, and Lynn McBain. Members of both committees were involved in the detailed planning of the study and have reviewed this paper. All authors are affiliated to the University of Otago at Wellington School of Medicine and Health Sciences, New Zealand.
Notes
Funding body
The Health Research Council of New Zealand funded the project (grant 99/065). Supplementary funds were also contributed by the Alcohol Advisory Council (ALAC)
Ethics committee
The Wellington and Manawatu-Whanganui Ethics Committees approved the methods and procedures used in the study
- Received May 24, 2004.
- Revision received September 1, 2004.
- Accepted December 1, 2004.
- © British Journal of General Practice, 2005.
