Assessment of decision making
Our analysis showed that almost all examiners started the questions by presenting candidates with a dilemma, asking them to discuss what they would do if faced with the situation. A few examiners explicitly asked the candidate to identify the dilemma, and in doing so, encouraged them to specifically discuss the possible options. For example:
‘A consultant asks you to prescribe some cimetidine for the unlicensed use of wart treatment. What is the dilemma here?’ (Examiner 36.A.)
Having asked the candidate about one dilemma, we found that rather than moving their line of questioning through the process of decision making there was a tendency among examiners to make the original case increasingly complex. Thus, having responded to the dilemma by outlining the possible options, the candidate is not then asked to explain why one option may be preferable to another.
In making the original case more complex, we found that examiners often present the candidate with a further dilemma; one that renders the previously stated options inappropriate. This is illustrated in Box 1 where the candidate largely responds to the increasingly complex dilemma being presented by highlighting the options that they considers to be appropriate. The candidate provided new options each time that the examiner added a further ‘layer’ to the case in question. At times, the examiner asked the candidate to make a choice, asking what they would do in the given situation. While the candidate stated their preferred option, they did not offer any justification for this selection or state any implications of the choice, and neither were they asked to provide this more detailed information. Consequently, by the end of the series of questions, the candidate was not able to demonstrate the full range of skills required in making decisions, but rather was encouraged to reveal their knowledge about the possible options that a GP faces when presented with a variety of dilemmas, and at times to make choices on the action that they would take. In other words, the candidate displayed the early stages of the decision-making process.
Box 1. An example of an examiner (47.A) asking a series of questions that make a dilemma increasingly complex for the candidate.
Examiner: ‘I would like to ask you a question about doctors treating themselves. Is it reasonable for a GP to take treatment for indigestion?’
The candidate suggests that there are many different views about this issue and that there are no clear-cut answers. They suggest that there may be a good reason to self-treat for something like indigestion — for example, some doctors may be too busy at work to see their own GP for treatment or they may feel they would be wasting the GP's time by consulting for something like this.
Examiner: ‘Let's be more precise. A partner who is a 50-year-old man. You go into his room just as he swallows a Zoton tablet from a free sample he's got. What are your views on that?’
The candidate suggests that they would ask the GP about his general health.
Examiner: ‘He says “well I just went out for a curry last night and it's a bit of indigestion”.’
The candidate suggests that this kind of self-treatment would be reasonable. However, they also state that GPs should be cautious about self-treating as they may miss something important and this could have a knock-on effect for patient care.
Examiner: ‘How can it impact on our patients?’
The candidate suggests that there is a lack of objective assessment as to whether the GP is fit to continue to work.
Examiner: ‘Well suppose a few days later you find a result in the post, it's actually one of your partners who has got the practice nurse to send blood for Helicobacter serology and you see that result. What would you do about that?’
The candidate suggests that the GP has a right to confidentiality, but also probably expects the result to be seen as he has requested that a member of staff take the blood. This, they say, might indicate that the GP is worried about his health.
Examiner: ‘Would you take any action on that?’
The candidate suggests that they would act on the result, but that the way this was done would depend on their relationship and whether they knew the GP well enough to ask about his health and need for treatment.
Examiner: ‘So what if he says to you “I am not registered with any other doctor?”.’
The candidate states that they would advise the GP to register with someone.
Examiner: ‘What if he says, “well in the practice in the town and I fell out with them a few years ago”?’
The candidate says that they would enquire about why the GP has not registered with his own GP and would encourage him to do so.
Examiner: ‘OK would you take any further action other than doing that?’
The candidate suggests that they would not be in a position to take the matter any further.
Examiner: ‘What if you discover he is then taking antidepressants as well, that he was supplying himself.’
The candidate simply states that this is ‘a bit difficult’.
Examiner: ‘What would you do then?’
The candidate suggests that the GP's competency needs to be considered.
Examiner: ‘So what action do you think you'd take?’
The candidate states that they would speak to the GP about taking antidepressants.
Examiner: ‘And he refuses to cooperate?’
The candidate says that they would look for further help and support to deal with this situation.
Examiner: ‘What would you do?’
The candidate lists a number of possible options — places that they could find help. For example, another partner in the practice, LMC, PCT.
In order to demonstrate the full range of decision-making skills, the candidate needs to be asked to discuss the implications of each of the options they outline and justify the choices that they make. This would indicate that they do not simply recognise that there are many different choices, but that they have the ability and knowledge required to weigh up one option against another and to use this information to make the most appropriate choice.
Although we found that the majority of examiners focused on the early stages of decision making, some examiners were clearly very skilled at eliciting candidates' ability to make decisions, encouraging them to discuss how they weigh up the options and then make choices. In the following oral examination the examiner (43.A) asks the candidate questions that encourage the candidate to demonstrate decision-making skills:
Examiner: ‘I would like to ask you a question about racialism. A patient makes a racist remark about one of your partners. What are your options?’
The candidate responds confirming that action is required.
Examiner: ‘Tell me [what] the options are rather than what you would do.’
The candidate lists a number of options from ignoring the comment to taking action and by discussing the comment with the patient. The candidate also proposes several ways of approaching the patient.
Examiner: ‘What are the strengths and weakness of each of those options?’
The candidate goes through some of the strengths and weaknesses of suggested options but decides on the need to take action and address the patient.
Examiner: ‘Why would it be the best thing to do?’
The candidate defends their stance by suggesting that it is important to try and stamp out this behaviour before it gets any worse and perhaps becomes violent.
Examiner: ‘What are the wider implications of this? I agree with what you have said so far.’
The candidate discusses the importance of the doctor–patient relationship and the need to respect this relationship. Furthermore they suggest it is important that doctors are respected by society.
Examiner: ‘Any other implications? What are the implications to the practice itself?’
The candidate suggests that it is important that action is taken otherwise the practice could be labelled as being racist.
As can be seen, the examiner focused on one dilemma — a racist remark made by a patient — and asked the candidate to outline the options. They then asked the candidate to explain the strengths and weaknesses of the options, encouraging them to consider the implications of each of the choices. The examiner then continued by asking the candidate to justify their choice and to consider the implications in a wider context. By taking the candidate through these stages of decision making, the examiner was able to adequately assess their skills in this area.
Professional values underpinning decision making
The tendency to remain at the level of knowledge assessment was also a feature of the questions that address candidates' professional values underpinning decisions in general practice. The data show that while examiners frequently ask questions relating to professional values, they rarely encourage the candidate to demonstrate how these values support their decisions. Moreover, these questions are generally asked right at the end of the examination. For example, in a question about a patient repeatedly using the ‘out of hours service’, the examiner's final question to the candidate was:
‘Would you strike her off?’ (Examiner 46.A)
The candidate responds by suggesting that they would want to meet the patient to express their concerns about the misuse of the services.
Rather than encouraging the candidate to discuss why they might want to express their concerns about misusing the service and what she would want to achieve from this, the examination came to an abrupt end. A few examiners encouraged the candidate to discuss how their professional values supported their decisions. This can be seen in a series of questions asked by the examiner (18.A) encouraging a candidate to demonstrate how their professional values support decision making.
Examiner: ‘In some countries it is not the GP, for example, who signs the sickness certification. What do you see as the pros and cons of GPs doing sickness certification for patients?’
The candidate suggests that the benefits of a GP signing sick certificates are that they will know the patients fairly well and will have an understanding of whether their current health problem is consistent with their medical history. The candidate emphasises the importance of having knowledge about the patient and how this allows the GP to assess whether a sick certificate is warranted. On the negative side, the candidate suggests that this very same situation of knowing the patient may make it difficult for the GP to deny the patient a sick certificate.
Examiner: ‘Because?’
The candidate suggests that the relationship that the GP has with the patient is important and that the GP may want to give the patient the benefit of the doubt and offer a sick certificate as a means of preserving the doctor–patient relationship.
Examiner: ‘Is there a conflict of interest there sometimes?’
The candidate agrees that there probably is.
Examiner: ‘Why is that?’
The candidate suggests that GPs may feel under pressure to sign a sick certificate even for fairly trivial reasons, which don't really justify 1 or 2 weeks off of work.
In this oral examination the candidate was asked to explain why it might be difficult for the GP to deny the patient a sick certificate and whether there might be a conflict of interest. When the candidate did not provide an ‘in-depth’ answer, the examiner attempted to get the candidate to justify their response. Without being given this opportunity to justify their view, it is possible that the candidate's final response would have inappropriately influenced the examiner's overall subjective impression.
Responder validation
Ten examiners were asked to comment on our analysis of their individual examination transcript, and all appeared to be in broad agreement with the findings. Many examiners also commented on the usefulness of seeing their dialogue transcribed, often reflecting on how they could improve their examination technique now that they had seen it in this way:
‘It was very interesting to see it played out in writing.’ (Examiner 22.)
‘What your transcript has taught me is not to increase the complexity of the scenario to make it more difficult, but to concentrate more on trying to elucidate the process of decision making.’ (Examiner 21.)
‘There is little decision making here except that no decision can be sensibly made without the ability to consider all the possible issues.’ (Examiner 34.)
The added (and unexpected) benefit of carrying out the responder validation, therefore, was that it appeared to provide a useful learning tool, encouraging examiners to reflect on the effectiveness of their examining technique and allowing them to identify areas for potential improvement.