Abstract
Background How GPs understand mental health problems determines their treatment choices; however, measures describing GPs' thinking about such problems are not currently available.
Aim To develop a measure of the complexity of GP explanations of common mental health problems and to pilot its reliability and validity.
Design of study A qualitative development of the measure, followed by inter-rater reliability and validation pilot studies.
Setting General practices in North London.
Method Vignettes of simulated consultations with patients with mental health problems were videotaped, and an anchored measure of complexity of psychosocial explanation in response to these vignettes was developed. Six GPs, four psychologists, and two lay people viewed the vignettes. Their responses were rated for complexity, both using the anchored measure and independently by two experts in primary care mental health. In a second reliability and revalidation study, responses of 50 GPs to two vignettes were rated for complexity. The GPs also completed a questionnaire to determine their interest and training in mental health, and they completed the Depression Attitudes Questionnaire.
Results Inter-rater reliability of the measure of complexity of explanation in both pilot studies was satisfactory (intraclass correlation coefficient = 0.78 and 0.72). The measure correlated with expert opinion as to what constitutes a complex explanation, and the responses of psychologists, GPs, and lay people differed in measured complexity. GPs with higher complexity scores had greater interest, more training in mental health, and more positive attitudes to depression.
Conclusion Results suggest that the complexity of GPs' psychosocial explanations about common mental health problems can be reliably and validly assessed by this new standardised measure.
INTRODUCTION
Most patients with common mental health problems are seen solely by GPs.1,2 GP counselling and advice-giving in the consultation is based on how the GP understands and hopes to help patients to make sense of their problems. This understanding may often be intuitive rather than explicitly formulated,3 and may be based on GPs' personal experiences rather than on formal mental health knowledge or training. However, this understanding is likely to influence how the GP responds to the patient and the impact of this on the patient.
This study hypothesises that GPs vary in the complexity of their thinking about, and understanding of, mental health problems presented by their patients. Complexity refers to GP variation in the extent to which, in the consultation time available, they identify possible significant psychological, psychiatric, and psychosocial factors in a patient's problems, and in how they consider these factors may interact to cause difficulties for the patient.
In addition, it is hypothesised that the more complex a GP's thinking about mental health problems is, the more likely it is that the GP will be able to help patients to make meaning of their problems in ways that may assist in easing their distress and helping them engage with potentially helpful treatments.4 Therefore, complexity of thinking may be associated with more effective intervention.
How this fits in
How GPs understand common mental health problems influences how they treat such problems. There is no current standardised method of assessing GPs' thinking about mental health problems. The complexity of GP psychosocial explanations of common mental health problems can be reliably assessed using a new standardised video-vignette measure developed and piloted in this study. GPs using more complex explanations on the measure may be better able to help patients, but this requires further evaluation.
To test these hypotheses, a measure of GPs' thinking about mental health problems is needed. The present study piloted the development and validation of such a measure to assess the complexity of GPs' psychosocial explanations of common mental health problems. The measure uses video vignettes, building on a long tradition of using vignettes in primary care mental health research.5–7 It attempts to measure the degree of complexity of GP psychosocial thinking, by eliciting GPs' thinking in response to the video vignettes.
METHOD
Phase 1: initial development of vignettes and complexity measure
Vignettes from GP consultations with patients who have common mental health problems were scripted and initially tested by reading them out to pilot participants and asking them what they thought was troubling the patients psychologically. The pilot participants were GPs, psychologists, and administrative staff. The six vignettes that the research team considered to have elicited the widest range of responses and to be closest to real cases were selected for video recording using trained actors.
Each vignette covered the first minute of a consultation, with the patient presenting their problems, prompted by the doctor. This vignette length gave sufficient material to be able to suggest complex explanations, but not so much that the subsequent coding became too difficult. Scripted vignettes drawing on material from real consultations were used, rather than extracts from real consultations, to ensure that all doctor responses in the vignettes were neutral, and avoided the potential constraints around gaining informed consent for using live video consultations for such research purposes. Box 1 gives examples of two vignettes.
Box 1 Description of video vignettes used in the GP reliability and validation pilot.
▸ Vignette 1:
A 40-year-old man, complaining of chest pains and tiredness, wants his chest checked out. He says he has stopped using drugs and drink recently as he is worried about the damage he might have done to his heart. His father died a few months ago of a heart attack and his grandmother before that. He describes his father as a bitter, stressed, and depressed man who was left by the patient's mother. The patient does not see his mother. In response to the GP's question, he says he is tense but not depressed. He feels he is dealing with the tension at work, but then starts shouting at someone for no reason, which makes him feel weird.
▸ Vignette 2:
A 55-year-old man has come for a routine blood pressure check. He tells the GP he is fed up with work — it is getting him down. He has been taking time off work because of feeling tense and odd. In response to the GP's questions, he describes changes at work, with former colleagues having moved on because of their age, and been replaced by young men who do not have the same work ethic. They make comments behind his back and see him as a bit of a Hitler. There is also a possibility that he may be made redundant from work. He has thought about taking early retirement and moving to the country, but he is not sure — ‘not old enough to be thrown on the scrap heap yet’.
An anchored complexity of psychosocial explanations measure was developed through iterations of coding the pilot participants' responses to the early versions of the scripted consultation extracts they had read. The measure was constructed so that more complex explanations are those that: mention a greater number of hypothesised elements to the patient's difficulties; link these elements to describe how their interaction resulted in problems for the patient; and give alternative possible explanations. Thus, the highest anchored levels require more than just listing elements; they need to be linked to describe how their interaction give rise to difficulties for the patient. The measure is multi-theoretical, such that ‘lay explanations’ as well as explanations drawn from different psychological, social, and psychiatric explanatory models can equally be rated as complex. Evaluation of accuracy of explanations is not attempted in the measure. A complex explanation may be inaccurate.
Development of the measure involved adapting anchoring descriptions and coding examples in response to pilot responses, to distinguish between explanations of different complexity and to improve agreement between coders. For example, a key distinction in the coding system is whether an explanation links elements or just lists them. Development of the measure included a progressively clearer and tighter specification of the forms of linkage and wording in an explanation (for example, ‘because’, ‘as’, ‘results in’, ‘related to’) that would be counted as making clear that elements were linked.
The final version of the measure (summarised in Box 2; full measure, Appendix 1) has seven anchored points.
Box 2 Summary of scoring on complexity of psychosocial explanations measure.
Scoring on the 7-point anchored scale reflects:
The number of different possible elements to the patient's difficulties that are mentioned in the explanation.
The extent to which the explanation links these elements in describing how their interaction results in problems for the patient.
Elements include the patient's specific feelings/affects/moods, thoughts, psychosocial stressors/circumstances, behavioural tendencies/impulses, and personality structures/schemas. The more elements mentioned in the explanation and the more they are explicitly linked in describing how they result in problems for the patient, the greater the rated complexity of the explanation. Also, the more alternative explanations mentioned, the greater the rated complexity.
Level 1. Only statements of global negative feelings, diagnoses, or circumstances:
Levels 2–4. Levels increase with number of specific elements mentioned and whether these are explicitly linked with the patient's distress/disturbance. But no indication of the way the elements interact to cause difficulties:
He has issues about his stage of life, about how to cope with the ageing process (level 2).
He is anxious about having a heart attack (level 2).
There are problems at work and also isolation from friends (level 3).
The anxiety is brought on by his family problems (level 3).
He has anxiety about job changes, obsessional traits, and is isolating himself (level 4).
His father's death, with his unresolved issues in relation to his dad, have made him depressed (level 4).
Levels 5–7. Explanations that indicate how the interaction between elements results in problems. Higher levels indicate a greater number of distinct elements linked in the overall explanation and/or greater numbers of explanations with linked elements:
Because of his father's recent death of a heart attack he is worried that his own chest pains are heart related (level 5).
He is in a muddle. Does he retire or does he stick with the job he has enjoyed for so long but now has problems? (level 5).
He has been really proud of doing well for so many years in his job, but now the job has changed and he is angry and sad about this (level 6).
His father died of a heart attack and, because of this, he is worried he will die next. Also, he feels guilty about his mother since he moved to London (level 6).
The changes in work, given his rather rigid personality and tendency to insecurity, are making him feel undervalued and relegated to the scrap heap and his response to the threat of change is to anticipate it and say he is going to leave his job before he is made to leave it (level 7).
He had an antagonistic relationship with his father and with his father's death he now feels guilty that he never was able to address any of that. His father's death has also made him miss his mother more and to be frightened of the future (level 7).
Phase 2: pilot of measures across GPs, psychologists, and lay staff
Phase 2 involved the following procedures:
showing the six vignettes to pilot participants;
coding responses of the pilot participants on the complexity scale to assess inter-rater reliability and variability of response;
coding the same responses using a global intuitive rating of how complex the explanation seemed to be, to evaluate whether the complexity scale and global intuitive rating were associated (content validity of the complexity measure); and
selecting two vignettes for further use.
The six videotaped vignettes were shown to six GPs, four clinical psychologists, and two people without any training in mental health (one a secretary and one a podiatrist). Each of the 12 participants was shown three of the six vignettes. Vignettes were presented using a block randomisation procedure to ensure that each of the six vignettes was shown to six participants. After viewing each vignette, the participant was asked by the research assistant: ‘What do you think is troubling the patient psychologically?’. The research assistant prompted each participant with similar neutral questions (for example, ‘Anything else?’, ‘Do you think anything else is troubling the patient psychologically?’), until the participant indicated there was nothing else they could add. Participants' responses were audiotaped and subsequently transcribed.
The 36 transcribed responses of the 12 pilot participants were rated by four raters. Each rated each transcript on a 7-point scale of complexity of explanation of the patient's difficulties. Two of the raters (a GP who also trained as a psychiatrist, and a clinical psychologist with extensive experience in primary care) made a global rating of the transcripts on the basis of their experience and understanding of what constitutes complex explanations of mental health problems. The other two raters used the anchored 7-point complexity of explanation measure which was developed for the study.
Following viewing and responding to the vignettes, the six pilot GP participants were interviewed about their views of the procedure and about the specific video vignettes they were shown. From these interview responses, and from the ratings of the responses to the vignettes, two vignettes were selected (Box 1), these two were considered by the GPs to be of better quality (more akin to ‘real’ cases and where video quality was better), and which had a good range of levels of rated complexity between different participants.
Phase 3: GP inter-rater reliability and validation pilot
All 102 GPs with practices based in the inner-city London borough of Islington were approached to participate in the main inter-rater reliability and validation pilot. None had been involved in the previous pilot work. Fifty-one (50%) agreed to participate. Consenting GPs were seen in their practices and shown the two selected videotaped acted extracts of consultations developed in the first phase of the study. After viewing each extract, the GPs were asked ‘What is troubling the patient psychologically?’. They were prompted by the research assistant to give as full an explanation as they felt able to, and their responses were audiotaped and transcribed as in the second phase of the study. Transcripts were rated by two of three raters using the anchored complexity of explanation measure developed in the first phase of the study, supplemented by a coding manual drawn up to assist with scoring. The coding manual comprised definitions to clarify coding decisions (for example, should the following be counted as one or two different explanatory elements: ‘He has had a bereavement … he is bitter about his father's death’?) and examples from the phase 2 pilot of responses which had been coded at different anchored levels.
GPs were also given two questionnaires to complete:
a brief questionnaire constructed for the study with questions about the GPs' training, experience, and interest in mental health; and
the Depression Attitudes Questionnaire,8 a 20-item paper and pencil questionnaire designed to assess doctors' attitudes to depression.
In addition, information was collected on referrals for counselling and psychological therapies in the previous year for each participating GP. This was collected from both general practice-based systems and personnel (for example, practice counsellors), and from secondary care data systems (the local NHS psychology and psychotherapy departments to which the GPs made referrals).
Analysis
Data analysis was carried out within SPSS. Inter-rater reliability was assessed using the intra-class correlation coefficient (ICC; two-way random effects model for the first pilot, one-way random model for the second pilot).9 An overall measure of complexity of explanation for each participant was constructed by averaging the ratings of both raters across both vignettes. The relationship of complexity of explanation with other measures was evaluated using Pearson and Spearman rank correlations.
RESULTS
Interdisciplinary pilot of measures: phase 2
Inter-rater reliability of the complexity of explanation measure over the 36 coded vignettes of GPs, clinical psychologists, and administrative staff, was high (ICC = 0.78). Consistency in global rating of complexity by the two raters with special experience of primary care mental health, using their judgement rather than the scale, was somewhat lower (ICC = 0.65). The two kinds of rating, complexity measure and global ratings, were moderately associated (Spearman rank ρ = 0.67).
There was a wide range between participants' responses to the vignettes on the complexity measure (average between the two raters), ranging from a low of 3 to a high of 7. Variability was similar for the global rating (2–6). Differences between rated responses of the professional groups were in the expected direction: clinical psychologists' vignette responses were rated as having the highest average complexity; GPs' responses had intermediate complexity; and participants without mental health training had the lowest complexity scores (Figure 1). There was also variability within professional groups, with one GP giving the highest average-rated complexity of explanations responses of any participant (above the four clinical psychologists).
Figure 1 Complexity of explanation ratings for different professional groups.
GP inter-rater reliability and validity pilot: phase 3
Useable responses to the videotaped vignettes were obtained from 50 GPs (responses from one GP were not recorded as a result of equipment failure). Inter-rater reliability of the 50 GPs' responses was a little lower than in the first pilot study, but was satisfactory and similar for each vignette (ICC = 0.71 and 0.73). There was a moderate correlation between the two vignettes in rated level of complexity of explanation (Spearman rank ρ = 0.57).
The overall measure of rated level of complexity (mean rating of both raters across both vignettes) showed a wide range between the 50 GPs from 2.75 to 7.0, that is, from a level of simple psychosocial explanations to explanations of significant complexity.
Table 1 gives associations between the GPs' rated complexity of explanation and GP demographic measures, and measures of their stated interest in mental health, training in mental health, and attitudes to depression. Rated complexity of explanation was associated with greater GP self-reported interest in mental health and with more GP training in mental health. Rated complexity of explanation was not significantly associated with the four Depression Attitudes Questionnaire principal components from previous research,8 but was associated with a composite of all four components, indicating more positive attitudes towards working with depressed patients and to their treatment. The individual Depression Attitudes Questionnaire item most strongly associated with complexity of explanation was ‘I feel comfortable dealing with depressed patients' needs’ (Pearson r = 0.46, P<0.001).
Table 1 Relationship of GP complexity of explanation and their demographic and practice variables, interest and training in mental health, and DAQ scores (n = 50).
Table 1 also indicates that there was no association between complexity of explanation and number of referrals for counselling and psychological therapies.
DISCUSSION
Summary of main findings
The video vignette procedure was successful in eliciting GPs' explanations about common mental health problems, and GPs found it acceptable and interesting. The measure of complexity of psychosocial explanations about mental health problems had satisfactory inter-rater reliability. It correlated with expert opinion as to what constitutes a complex explanation, and differentiated in the expected way between the explanations of psychologists, GPs, and people without mental health training.
The level of complexity of explanation was similar between the two vignettes. A GP giving a complex explanation to one vignette would also tend to give a complex explanation to the second vignette, indicating that the procedure was capturing consistent differences between GPs in their complexity of psychosocial understanding. GPs with higher complexity scores reported a greater interest in mental health, had more positive attitudes to the care of patients with depression, and were more likely to have been on training courses in mental health.
Strengths and limitations of the study
A strength of the study is the three-phase process using different methods to develop and evaluate the reliability and validity of the complexity of explanations measure. A further strength is that half of all the GPs in a primary care trust were recruited, which is a good response rate for this form of investigation; the numbers involved indicate that it is more likely that a range of GP explanatory models and skills were represented.
A potential limitation is that the measure favours certain models of explanation over others. It is limited to psychosocial explanations in that participants are explicitly asked what they think is troubling the patient psychologically, and the scoring excludes consideration of biomedical explanations of common mental health problems (for example explanations in terms of neurotransmitter imbalance or the physiology of anxiety), which can be equally, or more, complex. While designed to allow different types of psychosocial explanation (behavioural, cognitive, existential, lay, narrative, psychodynamic, social, and systemic), to be equally likely to be rated as complex, by incorporating elements that are key to each in the coding, the complexity measure may inadvertently incorporate biases for or against one or the other. This is potentially open to empirical investigation through using the measure to code the explanations of experts from different models to check whether they are all equally rated as complex.
A further limitation is that, being based on brief vignettes, the study neglects the continuing nature of general practice care. Understanding of problems, even newly presented problems, will be mediated by the GP's experience from previous contact with the patient, and will evolve over a series of consultations. In a related area, longitudinal studies of GP identification of mental heath problems have come to different conclusions from cross-sectional studies.10
Comparison with existing literature
The measure developed in this study focuses on complexity of understanding and explanations for patients' mental health problems, rather than on type or model of explanation. The scant previous research on GPs' ways of understanding mental health problems has been concerned with type of explanation; for example, whether doctors see common mental health problems as related to living or biomedical conditions, caused by psychosocial stresses or childhood experiences, or a function of patients' personalities.11–16 By contrast, this complexity of psychosocial explanations measure is designed to be multi-theoretical such that a range of types of psychosocial explanation can be rated as complex, including lay explanations.17,18
Complexity of thinking about mental health problems is likely to be partly a function of empathy, which is known to be important in general practice.19–22 Empathy has both emotional (feeling how the other person feels) and cognitive (thinking into how the other person thinks) components.23,24 It can be hypothesised that there is a relationship between empathic skills and the complexity measure, but this needs to be explored empirically. Similarly, curiosity about other people, psychological mindedness,25 and knowledge of psychiatric and psychological theories will also probably contribute to complexity of explanation, but are conceptually distinct.
The measure aims to evaluate the complexity of GPs' thinking, not the actual explanations GPs give to patients in the consultation. What is a good explanation to give to a patient will depend on factors such as the fit with the patient's problems, what is acceptable to or congruent with the patient's views, and what will enable or empower the patient to manage their problems.26–29 In many cases, ‘simpler’ rather than complex explanations may be more useful to patients. However, it can be hypothesised that the more complex the GP's psychosocial thinking about the patient, the more likely it is that the GP will be able to come up with an explanation that fits, is acceptable to the patient, and helps them to feel less distressed, more understood, and more able to deal with their problems.
The lack of association between individual GPs' rated level of complexity of explanation and their referrals for counselling and psychological therapies is not surprising. GP and practice factors affecting referral rates are complex,30 and in the case of counselling and psychological therapies are particularly affected by constraints of availability of referral options and waiting lists. A more complex psychosocial understanding by GPs of their patients' problems, as with a greater general interest in mental health problems,31–33 could lead GPs either to refer more, or to feel more comfortable about managing problems themselves without referral.
Implications for future research
The key predictive question is whether GPs who are generally better able to understand patients' mental health problems (operationalised as having higher complexity of psychosocial explanation scores), are more likely to obtain better clinical outcomes for their patients. Following the development of the complexity of psychosocial explanation measure, this can now be evaluated. If GPs with higher complexity have better clinical outcomes, this would suggest that training in thinking about and understanding common mental health problems may be as important as, or even more important than, training in communication skills, counselling skills, or cognitive-behavioural or other specific psychological treatment techniques.
Traditionally, Balint groups and similar case discussion groups have focused on helping doctors to understand their patients rather than teaching specific skills.34 This matches training in the psychological therapies, including cognitive-behaviour therapy, which focuses on ways of understanding patients as much as on therapeutic technique.35,36 However, given that GPs with lower complexity scores had less interest in mental health, the dilemma of training, as always, is that those who most need training are least likely to take it up.
Acknowledgments
We would like to thank the GPs who participated and Lisa Horsell for rating some of the vignettes.
Appendix 1 Complexity of psychosocial explanations measure.
In the complexity of psychosocial explanations measure, video extracts from acted consultations between GPs and patients with common mental health problems are shown to a research participant. Following the viewing of each video extract, the participant is asked what he/she considers is troubling the patient psychologically. The participant's responses are rated on an anchored scale of 1–7 reflecting different levels of complexity in the explanation of psychosocial factors related to the patient's difficulties.
Scoring on the 7-point anchored scale reflects:
The number of different possible elements to the patient's difficulties that are mentioned in the explanation.
The extent to which the explanation links these elements in describing how their interaction results in problems for the patient.
The number of alternative explanations.
Evaluation of accuracy of explanations is not attempted in the measure. A complex explanation may be inaccurate.
Level 1. Statements of global negative feelings, diagnoses, or circumstances only. The distinction from the next level is that specific affects or psychosocial stressors are not identified. Examples:
Levels 2–4. Mention of specific affects/moods, thoughts, psychosocial stressors/circumstances, behavioural tendencies/impulses, hypothesised early experiences, personality structures/schemas, or defences. The distinction within levels 2–4 is in the number of such elements mentioned. The distinction from the next level (≥5) is that these elements are mentioned without an indication of the way they interact to cause difficulties/disturbance. However, where one or more of these elements are given as a reason/explanation for symptoms or disturbance or distress (that is, interaction between an element or elements and symptoms/disturbance/distress rather than an interaction between different elements), this is rated as level 3 (3.2) or 4 (4.2).
Level 2. Only one specific element mentioned (where closely-related affects or thoughts are mentioned, they are counted as one). Examples:
She has marital problems.
She is lonely and distraught.
He is anxious about having a heart attack.
He thinks he is going to die.
Level 3. Either two specific unrelated elements mentioned. Examples level 3.1:
He is jealous and angry.
There is the bereavement and also the job stress.
She is insecure. She wishes she had a relationship.
Or, one specific element (or more than one related element) given as an explanation for symptoms, disturbance or generalised distress. Examples level 3.2:
The anxiety symptoms are brought on by his family problems.
She is depressed and anxious because of jealousy and envy.
Level 4. Either: three or more specific unrelated elements mentioned, of which at least two must of different types (that is, not each an affect, or each a psychosocial circumstance). Examples level 4.1:
He has anxiety about job changes, obsessional traits, and is isolating himself.
There is a greater lack of confidence. The patient does not feel she can do what she has to about her son and there are issues about her mother.
Or: two or more specific unrelated elements given as an explanation for symptoms, disturbance, or generalised distress. Examples level 4.2:
His father's death, with his unresolved issues in relation to his dad, have made him depressed.
She's got generalised anxiety linked with the loss of her daughter and worry about her husband.
Levels 5–7. Explanations that link specific psychosocial stressors/circumstances, affects/moods, thoughts, hypothesised early experiences, personality structures/schemas, behavioural tendencies/impulses, defences, indicating how the interaction between elements results in problems. The distinction within levels 5–7 is in the number of distinct elements linked in the overall explanation.
Level 5. Explanations linking two distinct elements. Examples:
She is in a muddle. She wants to be with him but is also very angry at him.
He thinks he's going to faint, and this frightens him and triggers a panic attack.
Level 6. Either examples linking three distinct elements, of which at least two must be of different types (not each an affect or each a thought). Examples level 6.1:
She didn't go out, as since her mother's death she feels guilty about enjoying herself.
He feels so down that he doesn't do anything, but when he is inactive he thinks he is inadequate and this just makes him feel worse.
Or: two separate independent explanations each linking two distinct elements (where each of the two explanations meets criteria for level 5). Example level 6.2:
His father died of a heart attack and, because of this, he is now worried he will die next. Also, he feels guilty about his mother since he moved to London.
Level 7. Either: explanations linking four distinct elements, of which at least three must be of different types (for example, not each an affect or a thought). Examples level 7.1:
She is angry and disappointed about the marriage not working out. I think she really wants another relationship, but feels she might get hurt again, so avoids any possibility of meeting anybody.
The changes in work, given his rather rigid personality and tendency to insecurity, are making him feel undervalued and relegated to the scrap heap and his response to the threat of change is to anticipate it and say he is going to leave his job before he is made to leave it.
Or: two separate independent explanations each linking three distinct elements, of which at least two must be of different types (plus each of the two explanations must meet criteria for level 6). Example level 7.2:
He had an antagonistic relationship with his father and with his father's death he now feels guilty that he never was able to address any of that. His father's death has made him miss his mother more and to be frightened of the future.
Notes
Funding body
Camden and Islington Community Health Services NHS Trust
Ethical approval
The pilot studies were approved by the Camden and Islington Community Local Research Ethics Committee (01/05)
Competing interests
The authors have stated that there are none
- Received September 19, 2007.
- Revision received November 19, 2007.
- Accepted February 19, 2008.
- © British Journal of General Practice, 2008.
