Original Paper
Are we using appropriate self-report questionnaires for detecting anxiety and depression in women with early breast cancer?

https://doi.org/10.1016/S0959-8049(98)00308-6Get rights and content

Abstract

The aim of this prospective study was to identify the psychiatric morbidity associated with the diagnosis and treatment of early breast cancer. At each of five time points, 269 women were interviewed using a shortened version of the Present State Examination (PSE) and 266 completed self-assessment questionnaires, the Hospital and Anxiety Depression Scale (HADS) and the Rotterdam Symptom Checklist (RSCL). This paper compares the ability of the questionnaires to detect psychiatric morbidity with that of the PSE. The majority of women who experienced anxiety and/or depression did so within 3 months of their initial surgery. The clinical interview identified anxiety disorder in 132 of 266 women (49.6%) and depressive illness in 99/266 (37.2%) during the first 3 months. Using the recommended threshold of ≥11 for caseness, the sensitivities for both tests were very low at 24.2% (HADS anxiety) and 14.1% (HADS depression) and 30.6% (RSCL psychological distress scale). Lowering the threshold value to ≥7 on the HADS improved the sensitivity to 72% for the anxiety subscale, but it remained low at 37.4% for the depression subscale. A threshold of ≥7 for the RSCL scale raised sensitivity to 66.7%. Lowering the threshold values raised the sensitivity of both the instruments but decreased their specificity: the lower the threshold, the greater the number of women who were identified as false positives which would increase the work load for clinic staff if used as a screening tool. Given that the HADS was inadequate in discriminating for depressive illness, it was not surprising that its use as a unitary scale with a threshold value as low as 12 resulted in a sensitivity of only 42.7%. In the light of these findings, we question the use of both the HADS and the RSCL as suitable research or screening instruments for detection of psychological morbidity in early breast cancer.

Introduction

Every year in the U.K., around 24 000 women are diagnosed with breast cancer. There has been extensive research into the psychosocial sequelae of the diagnosis and treatment of this disease providing ample evidence that a significant minority of women are at increased risk of developing anxiety and depression1, 2. Systematic studies show that between 20 and 35% of women with breast cancer, irrespective of their stage of disease, have measurable psychiatric morbidity3, 4. In research settings, questionnaires are used frequently to assess quality of life of patients entered into clinical trials, so it is important to determine how reliably they detect differences between groups. In clinical practice, detection rates of this morbidity have been shown to be low[5]. Patients may be reluctant to disclose how they are feeling and doctors and nurses may be unwilling to enquire or lack the appropriate communication skills. Accurate methods of detection of psychiatric distress that are applicable to busy clinics and acceptable to patients would enable doctors and specialist nurses to concentrate their efforts on women who need their help and thus use their time more efficiently. Two questionnaires are widely used as screening instruments to measure psychological morbidity for patients with cancer in both research and clinical settings: the Hospital Anxiety and Depression Scale (HADS)[6]and the Rotterdam Symptom Checklist (RSCL)[7].

The HADS is intended for use with physically ill patients and unlike other psychiatric self-report questionnaires, does not contain somatic symptoms which may be attributable to disease and/or treatment. The 14-item instrument comprises two 7-item subscales designed to discriminate between anxiety and depression. All items refer to symptoms experienced during the previous week. The authors[6]recommended using a threshold of 11 to include all probable cases of anxiety disorder and depressive illness and a threshold of 8–10 to identify all possible cases of the two disorders.

The RSCL was developed primarily as an instrument to measure symptoms reported by patients with cancer participating in clinical research. It is a multidimensional scale for use in assessment of quality of life and contains two subscales: one of these is an 8-item scale assessing psychological distress during the previous three days. The recommended threshold for this scale is 11, according to personal communication with one of the authors, cited by Hopwood and associates[8].

The few studies that have now been published measuring the performance of these questionnaires as screening tools in different populations of patients with cancer, have reported somewhat equivocal findings. Razavi and colleagues reported that the French version of the HADS scale failed to discriminate between anxiety and depression when measured against a psychiatric interview[9]. They suggested summation of both scales to provide a unitary 14-item psychological distress scale and tested its ability to screen for psychiatric morbidity on a sample of in-patients with various cancer sites. Application of receiver operating characteristic (ROC) analysis[10]found that the optimal thresholds on the unitary scale were 19 for major depressive disorder and 13 for depressive illness plus adjustment disorder.

In a study of patients with a variety of cancers, using paired combinations of questionnaires including the HADS and the RSCL, Ibbotson and colleagues[11]concluded that both instruments performed well. In this study, psychiatric diagnosis was made following the administration of the Psychiatric Assessment Schedule[12], using DSM III criteria[13]. The HADS was used as a unitary scale with 14 identified as the optimal threshold value using ROC analysis; the study did not test the performance of the HADS subscales. However, when patients were classified as disease-free, having stable disease or with progressive disease, some differences in the performances of the instruments emerged. The HADS performed well in patients who were disease-free or whose disease was judged stable, but was not sensitive or specific enough for patients with progressive disease. Only the RSCL performed well in those who had progressive disease. In patients who were receiving active treatment, both questionnaires were effective as screening instruments.

Patients with advanced breast cancer, in a study by Hopwood and colleagues[8]completed the HADS and the RSCL and were interviewed using the Clinical Interview Schedule[14]. The HADS was able to discriminate between anxiety and depression. Optimal sensitivity and specificity were found to be in line with the recommended threshold of 11 for both the HADS anxiety and depression subscales and 11 on the RSCL psychological distress scale. However, the misclassification rate for the HADS depression subscale was 25%, compared with 12% for the anxiety subscale and 21% on the RSCL. When these thresholds were applied to a large outpatient sample of women with advanced breast cancer[15], the authors found that although both questionnaires identified very similar percentages of cases, there was considerable discrepancy in the makeup of the two groups: concordance in identifying cases was only 43%.

A study conducted by Ramirez and colleagues[16]focused specifically on women with early breast cancer. The primary aim was to assess the ability of the pre-operative HADS score to detect current mood disorder and to predict disorder at 3 and 12 months postoperatively, measured against a psychiatric interview. The HADS was used as a unitary 14-item scale; no results were reported about the ability of the HADS subscales to discriminate between anxiety and depression. The application of ROC curve analysis demonstrated an optimum threshold of 11 on the unitary scale to achieve a sensitivity of 70% in identifying psychiatric disorder either pre-operatively or at 3 and 12 months postoperatively. The performance of the HADS in identifying concurrent mood disorder at 3 and 12 months was poor. The authors concluded that the performance of a single pre-operative HADS score in identifying mood disorder in the year after diagnosis was superior to using the HADS to detect concurrent mood disorder at intervals throughout that year.

Thus, there appears to be uncertainty, not only about which questionnaire most accurately identifies patients with anxiety and depression, but also about what threshold values should be used. In this paper we compare the performance of the HADS and the RSCL in detecting anxiety and depression with that found following a standardised clinical interview.

Section snippets

Sample

Two hundred and sixty-nine women treated for early breast cancer in the south of England were recruited to a large Cancer Research Campaign funded study designed to assess psychological outcomes of different treatment policies. Detailed results from this study have been reported elsewhere[4]. All the women were aged less than 75 years and had stage I or stage II disease. Of the 115 women who were treated by local excision, 107 (93%) received adjuvant radiotherapy. Adjuvant chemotherapy was

Sample

Results reported elsewhere[18]showed that, in the 3 year duration of the study, the majority of the 269 women who experienced anxiety and/or depression had done so within the first 3 months following diagnosis and treatment. To maximise the number of available cases on which to test the performance of the questionnaires, data were therefore combined from the postoperative and 3 month assessments. Postoperatively and/or at 3 months, 266 (99%) women completed the HADS and 267 (99%) completed the

Discussion

In a clinical trial setting, the misclassification rate of an instrument must be low in order to detect reliably clinically important differences in outcomes between trial arms. In a clinical setting, the sensitivity of an instrument is of primary importance, as it is important to keep rates of false negatives as low as possible. Choosing the optimal threshold value for a test entails a trade-off between sensitivity and specificity. Raising the sensitivity threshold decreases specificity, which

Acknowledgements

The authors would like to thank the Cancer Research Campaign for funding this study.

References (22)

  • D Razavi et al.

    Screening for adjustment disorders and major depressive disorders in cancer in-patients

    Br J Psychiatry

    (1990)
  • Cited by (143)

    • Interaction effect of the serum interleukin-6 level and anxiety on the 12-week pharmacotherapeutic responses of patients with depressive disorders

      2022, Journal of Affective Disorders
      Citation Excerpt :

      Patients were grouped based on the median HADS-A score (<12 vs. ≥12) for the main analysis. In previous studies, the HADS-A cutoffs were 7 (Esser et al., 2018; Hall et al., 1999), 8 (Olsson et al., 2005), and 10 (Gur-Ozmen et al., 2017; Wiglusz et al., 2018) in patients with various physical diseases, including epilepsy and cancer. We included only depressed patients.

    View all citing articles on Scopus
    View full text