The development of COMRADE—a patient-based outcome measure to evaluate the effectiveness of risk communication and treatment decision making in consultations

Abstract

Some instruments have been developed to evaluate ‘decision effectiveness’ such as the ‘satisfaction with decision (SWD)’ and ‘decision conflict (DCS)’ scales and are validated for the US context. Patients identify further outcome domains that are not fully covered in these scales. We developed a patient-based outcome measure to evaluate risk communication and decision making effectiveness, addressing these further domains and validated for use in the UK. Formulation of items was based on literature review and key informant interviews. The instrument was piloted and developed (including psychometric assessment) in the context of a trial of decision making and risk communication interventions in general practice. Patients were recruited to the trial with known atrial fibrillation, prostatism, menorrhagia or menopausal symptoms. High response rates (96% in first phase) indicated acceptability to patients. A 20 item patient-based outcome measure, with two sub-scales for ‘risk communication’ and ‘confidence in decision’, was produced. It allows for paternalistic, shared or informed choice decision making models. Usage in further studies will facilitate systematic reviews. Consequently, lessons for wider application of these interventions in practice, and policy implications regarding greater patient involvement should then be clearer.

Introduction

Health care professionals spend much of their time discussing the risks and benefits of treatments or care with their patients—often termed risk communication. The methods and the effects of such communication have been widely researched [1]. Interventions have been found to be most effective on key outcomes (often behavioural) if they address treatment choices in health care (as opposed to screening or prevention issues) and if they use individualised calculations of risk. However, risk communication occurs in a context of health care processes, and recently attention has focused on the ‘shared decision making’ model [2]. The competences of shared decision making (i.e. the key stages that professionals can work through in consultations to achieve appropriate levels of patient involvement in decisions) have been identified [3], [4]. The competences are listed below in Box 1.

The sharing of risk information is just one of the competences of ‘shared decision making (competence 4), although it is also addressed in competences 3, 5 and 6. Risk communication is also not necessarily linked with decision making, as shown by patients’ greater desires for information than involvement in decision making per se [3], [4]. However, the competences illustrate how conceptually there is an overlap between risk communication and decision making. This is relevant to issues about how to evaluate interventions to improve them. Given the increasing attention to both risk communication and shared decision making, methods of evaluating them are also increasingly important.

To explore how risk communication should be evaluated, we should look at what it is (a definition) and what it seeks to achieve (its goals). The definition is important when considering the elements that can or should be evaluated. One definition of risk communication is the ‘open two-way exchange of information and opinion about risk, leading to better understanding and better (clinical) management decisions’ [5], [6]. This definition moves away from notions that information is communicated only from clinician to patient, and that ‘acceptability’ (or not) of the risk is communicated back. The two-way exchange about information and opinion is important if treatment decisions are to reflect the risk attitudes of those who will live with the outcomes.

There has been debate about the precise goals of risk communication [7]. Is it to achieve behavioural change and usually thus a reduction in risk factors or actual behavioural risk exposure? Or is it to provide information alone, or to discuss and explore reactions to this information (developing the definition and the competences of shared decision making) and to facilitate informed decision making? Informed decision making is now widely regarded as a more appropriate goal, even if it may not result in the same level of (public) health gain as more behaviourally orientated risk communication and risk reduction interventions [8]. (The latter concept of ‘informed decision making’ does not pre-suppose the model of decision making which can be paternalistic, shared or informed patient choice, as suited to the individual’s preferences.)

There is little empirical evidence about what constitutes a ‘good’ decision [9] or informed decision making. O’Connor [10] defines ‘effective’ decisions as informed, consistent with personal values and acted upon. Others have assessed whether a consumer has made ‘an informed choice’, based on knowledge and consistency between attitudes towards a screening test and whether or not the test was taken [11].

The previous literature in the risk communication field focused largely on the cognitive and behavioural outcomes of interventions [7]. These include, for example, accurate risk perception or knowledge of risk factors regarding cervical cancer after interventions to improve the uptake of cervical cytology surveillance [12], and smoking cessation rates after interventions addressing the risks of smoking [13]. Increasingly, however, attention has focused more on the affective outcomes of interventions in this field [7], [14]. These include satisfaction (either global or relating to certain elements of the consultation or intervention), assessment of the information given, anxiety, certainty about making the best treatment choice and so on. These may be more important to some consumers of health care [15]. They may also be more specific outcomes to evaluate in this field, and more likely to be responsive to change after intervention [7].

Certain instruments are available in the literature to assess some of the affective outcomes, or aspects of them. These derive more from the mainstream of decision making research, such as the evaluation of decision aids for health care decisions [16]. Their appropriateness for risk communication evaluation per se requires confirmation. It is clear at least that several of these instruments may evaluate some of the effects of risk communication (and decision making itself) but they may only be a proxy for evaluation of the process itself.

Table 1 summarises seven published instruments which seek to address these outcomes most closely and which are generically applicable [10], [17], [18], [19], [20], [21], [22]. The table describes their origin, basis and data on psychometric properties. Other scales also address specific aspects of the process of decision making, including patients’ desire for autonomy [23] and their perceived involvement in the process [22]. There is also an extensive literature on or more general aspects of ‘quality decision making’ [24], general satisfaction [25] or more specific issues such as making choices about screening tests [11]. Other condition-specific measures of satisfaction with decision making or decisions made, such as for prostatism treatment decisions, are also available [26]. However, these are insufficient to use for evaluating the process of risk communication and the wider process of decision making (with reference to the competences) across a range of conditions.

Scrutiny of the main published instruments also suggests that further development may be required in this field, even to assess the effects of decision making or decision aid interventions (Table 1). This may be especially so when considering the effect of research in different countries or other clinical and contextual variations. The language used in the instruments may be helpful in some settings but potentially confusing for other groups of research participants, and may vary from one country to another. Issues of translation require careful attention [27]. The expectations and previous experience of participants may also affect the validity of some items—for example, if scales assume that participants are making health care choices themselves, the scales may not be applicable in settings where treatment decision making is usually more paternalistic. Even where scales and items appear quite easily applicable to another setting, first principles of instrument development suggest the need to examine the validity of using the instrument in that setting [27], [28], [29].

Given this apparent need for further validation and development work, we therefore used items from some of the existing instruments in the field and explored their applicability in the context of UK primary care. Developments were made, integrating earlier empirical work with consumers as well [15]. This work identified outcomes which were important to patients or consumers when considering risk communication and shared decision making (Section 2). It seems imperative to integrate these perspectives into the evaluation of risk communication and decision making—interventions and a paradigm which are themselves fundamentally ‘patient-centred’.

A new composite outcome measure was produced (referred to as ‘COMRADE’—combined outcome measure for risk communication and treatment decision making effectiveness). The measure’s validity was then evaluated according to current recommendations [28], [29]. The measure was intended to be generically applicable across clinical conditions or scenarios and to focus on treatment decision making and risk communication. This paper reports the development of ‘COMRADE’.