Abstract
Background Because pragmatic trials are performed to determine if an intervention can improve current practice, they often have a control group receiving ‘usual care’. The behaviour of caregivers and patients in this control group should be influenced by the actions of researchers as little as possible. Guidelines for describing the composition and management of a usual care control group are lacking.
Aim To explore the variety of approaches to the usual care concept in pragmatic trials, and evaluate the influence of the study design on the behaviour of caregivers and patients in a usual care control group.
Design of study Review of 73 pragmatic trials in primary care with a usual care control group published between January 2005 and December 2009 in the British Medical Journal, the British Journal of General Practice, and Family Practice. Outcome measures were: description of the factors influencing caregiver and patients in a usual care control group related to an individual randomised design versus cluster randomisation.
Results In total, 38 individually randomised trials and 35 cluster randomised trials were included. In most trials, caregivers had the freedom to treat control patients according to their own insight; in two studies, treatment options were restricted. Although possible influences on the behaviour of control caregivers and control patients were more often identified in individually randomised trials, these influences were also present in cluster randomised trials. The description of instructions and information provided to the control group was often insufficient, which made evaluation of the trials difficult.
Conclusion Researchers in primary care medicine should carefully consider the design of a usual care control group, especially with regard to minimising the risk of study-induced behavioural change. It is recommended that an adequate description of the information is provided to control caregivers and control patients. A proposal is made for an extension to the CONSORT statement that requires authors to specify details of the usual care control group.
INTRODUCTION
Many trials in primary care require a pragmatic design. In contrast to explanatory trials, which are performed under ideal and controlled conditions, pragmatic trials measure the effect of an intervention in real clinical practice. Because pragmatic trials are performed to determine whether the intervention can improve current practice, they often have a ‘usual care’ control group. The care received by this control group is supposed to reflect the care as usually received by patients in daily practice.1
The design of a trial with a usual care control group requires specific attention (Box 1). The main difficulty is to ensure that this control group receives genuine usual care as supplied in everyday practice.2 However, various actions by the researchers may influence the behaviour of caregivers and patients. For example, behavioural change of control caregivers may be induced when they are informed about the issues under study, or because of a learning effect when they have to provide usual care to one patient and an intervention to another. Behavioural change of control patients may be induced when they are briefed about the trial and asked to give informed consent, or when they are asked to complete questionnaires and undergo examinations, drawing their attention to their condition and possible interventions. These actions may change their help-seeking behaviour or influence their complaints. This, in turn, may affect the outcome and the interpretation of the trial. Consequently, researchers have to think carefully about the influence of the study information and the intervention on the control group and on how to minimise this effect.
Box 1. Example of dilemmas faced by the present authors when designing a pragmatic trial with a usual care control group.
Guidelines give clear advice about prophylactic medication for migraine patients with frequent headaches. However, GPs do not often prescribe this medication. This study's authors decided to perform a randomised clinical trial to establish the advantages of a new intervention aimed at improving migraine treatment by GPs. Most likely this will be a pragmatic trial with one group of GPs applying the intervention and another group continuing their normal way of working (‘usual care’). For an optimal comparison it is preferable if control patients and control physicians are unaware of the study. However, patients have to be invited to participate, need to provide informed consent, and have to complete questionnaires to measure the outcome. Furthermore, control physicians have to be informed and agree to participate. Therefore, physicians and patients will be aware of the study and this could change or adapt their normal behaviour. For example, physicians could start studying the available guidelines on migraine and subsequently re-evaluate the therapy options for their migraine patients; patients might decide to visit their physician after reading the information and filling in the questionnaire; and, just after the study has started, an update of the national headache guideline might be sent to all GPs. For both the researcher and the reader it is difficult to estimate the impact of all these possible influences.
Researchers have different opinions about the meaning of the term ‘usual care’ and, consequently, apply different methods when designing and reporting a trial with a usual care control group. Guidelines on the reporting of trials are available to researchers in the CONSORT statement, including its extension to cluster randomised trials, and its recently published extension to pragmatic trials.3–5 However, these guidelines do not cover the specific requirements for describing the composition and management of a usual care control group. Two recent reviews on the validity of pragmatic cluster randomised trials focused on other issues, that is, mainly on the risk of selection bias. Until now, the risk of behavioural change of caregivers and patients in trials with a usual care group has been discussed only marginally.6,7 A recent review on low back pain reported a poor description of the usual care control arm in 26 of 33 reviewed studies.8,9
To gain insight into the variety of approaches to the usual care concept in primary care research, this study explored a cohort of pragmatic trials with a usual care control group, reporting for individually randomised and cluster-randomised studies separately. Furthermore, it examined the possible influences of the study information and awareness of the intervention on the behaviour of control caregivers and control patients. This overview shows the problems researchers face when designing pragmatic trials with a usual care control group.
METHOD
One of the authors manually searched three medical journals: the British Medical Journal, the British Journal of General Practice, and Family Practice, from January 2005 to December 2009. These particular journals were selected because they regularly publish articles on pragmatic trials in primary care; thus, this overview was not intended to be exhaustive. The two criteria for inclusion were:
A randomised pragmatic or effectiveness trial in a primary care or nursing home population; the aim of the trial should be to evaluate the overall effectiveness of an intervention in a ‘real-life’ situation, when people may not receive all of the treatment, and may use other treatments as well, the trial has broad eligibility criteria, and patient and practice-oriented outcomes.10,11
The control group was reported to receive usual care; in case the term ‘usual care’ was not explicitly used, the paper reported on a comparison between the intervention(s) under study and normal practice (that is, it did not compare two different unrelated interventions).
How this fits in
For pragmatic trials researchers often use a usual care control group. However, this concept is defined in a variety of ways. In addition, the information and treatment provided to the control group is often scarcely described. This makes it difficult for readers of trial articles to assess the applicability of trials with a usual care control group to their own population. This review proposes a more detailed description of the control group in CONSORT statement as a first step to the solution of this problem.
The articles were assessed by two of the authors independently. In case of disagreement, consensus was reached by discussion with a third author. A distinction was made between cluster randomised trials and individually randomised trials, because these study designs have different methodological problems. First, the study assessed how researchers applied the concept of usual care to the control group. For this purpose, an inventory was made of the descriptions that were given of the care in the intervention group and the care in the control group. Second, the risk of behavioural change was assessed according to the following criteria:
Influences on control caregivers: could the behaviour of the control caregivers possibly be influenced by the researchers? The study assessed whether (a) caregivers were informed about the allocation of the patients, (b) caregivers were informed about the content of the intervention, (c) caregivers had to complete questionnaires, (d) caregivers received extra information (for example, guidelines) or training on the subject of the trial, and (e) a learning effect was possible because of contact with patients in the intervention group.
Influences on control patients: could the behaviour of the patients possibly be influenced by the researchers? The study evaluated whether control patients (a) gave informed consent, (b) were informed about the content of the intervention, (c) were provided with extra information about their condition, (d) knew their allocation status, (e) had to complete questionnaires or undergo examinations, and (f) could have contact with patients in the intervention group.
Differences between individually randomised and cluster randomised trials were evaluated with a χ2 test.
RESULTS
A total of 73 articles were identified that met the selection criteria for this overview. Figure 1 presents a flow chart of the search and its results. Overall, 38 individually randomised trials (Tables 1 and 2)12–49 and 35 cluster randomised trials (Tables 3 and 4)50–84 were found. Table 1 shows that the term ‘usual care’ was used in 33 of the 73 articles. Other expressions were also used, such as ‘routine care’,15,29,35,63 ‘usual practice’,41 ‘normal care’,34 ‘standard care’,49,78 ‘care as usual’,55 ‘usual policy’,60 ‘usual service’,76 and ‘usual management’.84 Other authors did not use a specific expression.
Descriptions of usual care by researchers
In two individually randomised trials, the content of the care in the control group was prescribed by the researchers. Control caregivers were instructed to provide standard asthma care by 6-monthly check-ups via a dedicated asthma appointment with a diploma-level asthma nurse,23 or perform a 3-monthly measurement of glycosylated haemoglobin (HbA1c).19 In the remaining 71 trials, control caregivers were allowed to provide care according to their own insight, with some researchers even explicitly asking the caregivers not to change their usual practice.
In four articles it was reported that patients were included in the study after preliminary examinations. Caregivers and patients were informed about the results of these pre-trial investigations, regardless of allocation to the intervention group or control group.25,28,41,82
Influences on control caregivers
Half of all trials described that control caregivers were informed about the allocation status of their patients or practice. About half of the authors made no mention of caregivers' awareness of the allocation status. The proportions were similar for individually and cluster randomised trials?(Table 5).
In 16 individually randomised trials, caregivers provided the intervention as well as usual care, indicating that they were aware of both arms of the study. In 22 of the individually randomised trials, the information given to caregivers was not described in the article. In two trials it was reported that caregivers were not informed about the content of the intervention. This was possible because caregivers had no involvement in the development and distribution of the intervention (an educational booklet),16 or were blinded to the treatment protocol, which was performed by a separate team.40
In the majority of articles on cluster randomised trials, the information provided to control caregivers was not described. Only three papers explicitly stated that control caregivers were not aware of the exact content of the intervention; they received either an unrelated educational module,70,71 or neutral information about the trial without an explanation of the intervention.73
In some individually randomised and some cluster randomised trials, control caregivers were asked to complete questionnaires before or during the intervention period. In individually randomised trials, control caregivers recorded patient complaints,12,25 or evaluated the consultation.14 In cluster randomised trials, caregivers were asked about the care they provided before the start of the trial,50,65 were asked to evaluate the consultation,21,68 or were asked to record patient complaints.80
Understandably, control caregivers in the individually randomised trials were more often given extra information on the subject under study compared to caregivers in the cluster randomised trials. In 11 individually randomised trials, caregivers of control patients received training because they also had to provide the intervention to patients allocated to the intervention group. In one cluster randomised trial, all caregivers, including the usual care group, received the national guidelines on the subject under study,54 and in two trials they were invited to an initiation symposium.63,84 In another trial, the risk of behavioural change among caregivers was reduced by excluding those who were involved in the development of study guidelines.53
The possibility of a learning effect of caregivers was more often seen in individually randomised trials than in cluster randomised trials. This problem was more evident in individually randomised trials because the caregiver who provided the intervention to the intervention group also often continued to provide (usual) care to control patients. Less than half of individually randomised trials and one cluster randomised trial showed a risk of a learning effect in caregivers. In cluster randomised trials, caregivers might be influenced by information about the intervention communicated via patients who had this information.72
Influences on control patients
In nine of the cluster randomised trials, it was not necessary to inform control patients about the trial because the study data were gathered from anonymous electronic patient records and therefore consent was not considered necessary. In these latter trials, no risks of behavioural change in control patients existed. In almost all the individually randomised trials it was described that informed consent was obtained from control patients (Table 5).
In most trials, the article did not describe whether (or not) any information was provided to control patients. Therefore, it was not possible to judge these trials with respect to what influence study information may have had on control patients. If information had been given to control patients, most researchers did not inform patients about the content of the intervention.
In eight individually randomised trials, patients in the control group received information that could have influenced their behaviour. They were offered an information sheet,33,36,49 received extra treatment advice,19,47 or underwent a medical investigation before entering the trial.25,28,41 This problem was not found in cluster randomised trials.
Patients were informed about their allocation in more than half of individually randomised trials. This was the case in only two of the cluster randomised trials. Unfortunately, in 25% of the papers the authors did not mention whether patients knew about their allocation.
In most individually randomised trials, control patients were asked to complete questionnaires or to undergo examinations. About half of the cluster randomised trials gathered their information from electronic patient records or after the intervention period.59
The risk of influencing behaviour through contact between control patients and intervention patients was not evident. It was, however, difficult to properly assess this risk from the limited descriptions provided by the researchers. In two trials the authors acknowledged that between-group contact could have taken place.24,37 They described the potential problem and stated that they had no reason to believe that chance contact between the intervention group and the control patients had led to behavioural changes.
DISCUSSION
In this overview, different interpretations of the concept of usual care were observed across the studies. Factors in the design of trials with a usual care control group influencing behaviour of control caregivers and participants (such as information about allocation or intervention) were present not only in individually randomised trials but also in cluster randomised trials.
Description of usual care by researchers
This overview shows that the usual care concept is interpreted differently across the research community. According to some authors, one specific, predefined treatment should be chosen and be given to all subjects in the control group.11,85 This point of view argues that variation in treatments for control patients makes trial results difficult to interpret and generalise. In contrast, others try to improve external validity by advising that patients in a usual care control group should be confronted with the heterogeneity of treatments available in real, daily practice, rather than receiving a treatment chosen by the researchers.86,87 This view corresponds with the conclusion of two meetings of the National Institute of Mental Health (US) where usual care was defined as ‘the wide range of care that is provided in a community whether it is adequate or not, without a normative judgment’.88,89 In these meetings it was agreed that trials should have a usual care control group when they aim to prove superiority of a new intervention or approach over usual care in the community.
Thus, in the ideal situation, the usual care control group is not influenced at all. However, the ethical requirements of research and the proper conduct of trials can disturb the naturalistic character of a usual care approach.
Behavioural change in a usual care control group
This overview shows that risk of (unwanted) behavioural changes among caregivers and patients in a usual care control group is often present. Cluster randomisation is often seen as the solution for this problem. However, this study found that control caregivers' awareness of study conditions could influence their behaviour even in cluster randomised trials. Thus, in designing a study, researchers should ask themselves if caregivers need to be informed about the allocation of their patients or practice and about the content of the intervention. They should at least avoid providing information about the intervention to the control caregivers.
Also, when designing a trial, researchers should ask themselves to what extent control patients need to be informed about the trial, about the content of the intervention, and about their allocation. Possible solutions to diminish the risk of study-induced behavioural change include giving patients in the control group neutral information,40,51 or not disclosing the presence of the other study arm. The latter solution is called the Zelen design.86,90,91 However, many medical ethical committees refuse to accept this design, because they do not support withholding of information.92
Regarding the use of questionnaires and examinations, researchers should ask themselves whether these are needed and, if so, what questions are essential. In some trials, participants were asked to complete extensive baseline questionnaires or think about questions that would otherwise not have crossed their minds.72 This may have changed their help-seeking behaviour. Finally, especially in individually randomised trials, researchers should be alert to the possibility of contact between control patients and intervention patients.
Strengths and limitations of the study
This overview has revealed that authors rarely describe the information given to control caregivers and control patients in their paper, making it difficult for readers to evaluate the adequacy of the design. The fact that not all the required information was actually described in the research papers is a limitation of this overview. Often, it was not possible to fully appreciate the attempts of the researchers to ensure usual care was maintained in the control group. Because of this scarce information, it was often impossible to establish with certainty to what extent the observed problems actually influenced the final study results. It has been reported that results are difficult to interpret without knowing the nature of usual care.8,9 This study has provided an overview of the risks of behavioural change, but without the intention to be exhaustive. However, it has given an impression of the key issues affecting usual care control groups in this type of trial.
This overview addresses an overlooked issue in the medical literature, namely the design of a usual care control group in pragmatic trials. Because the term ‘usual care’ is not used consistently by researchers, trials could not easily be identified by a search in electronic databases. For this reason, the researchers manually searched three journals that regularly publish pragmatic trials in primary care with a usual care control group. In this way it was possible to provide an adequate impression of the current issues.
Recommendations for future research
In conclusion, researchers should carefully consider the design of a usual care control group, and consider ways to diminish the risk of study-induced behavioural change of caregivers and patients. As cluster randomised trials seem to be less sensitive to this problem, it may be advisable to consider this type of study design in preference to individually randomised trials. However, even when using cluster randomisation, researchers should bear in mind the risks of behavioural change in the usual care control group. In addition, this overview supports the argument that researchers should provide a better description of the design of the control group and the care provided to them. Many researchers do not report which, if any, information has been given to control caregivers and control patients, making it difficult to evaluate the adequacy of the study design in terms of internal and external validity. In the CONSORT statement extension for pragmatic trials, it is recommended to describe the control group in as much detail as the intervention group. The authors of the present study recommend making this recommendation more specific (Table 6). This implies that, in addition to a description of the intervention group, the reader requires a detailed description of the instructions given to control caregivers, as well as the information given to control patients that are supposed to receive usual care. This will allow the reader to evaluate whether the care provided to the control group is sufficiently representative for the usual care in daily practice.
Notes
Competing interests
The authors have stated that there are none.
Discuss this article
Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp–discuss
- Received July 26, 2009.
- Revision received August 27, 2009.
- Accepted January 22, 2010.
- © British Journal of General Practice, 2010.
REFERENCES
- 1.↵
- 2.↵
- 3.↵
- 4.
- 5.↵
- 6.↵
- 7.↵
- 8.↵
- 9.↵
- 10.↵
- 11.↵
- 12.↵
- 13.
- 14.↵
- 15.↵
- 16.↵
- 17.
- 18.
- 19.↵
- 20.
- 21.↵
- 22.
- 23.↵
- 24.↵
- 25.↵
- 26.
- 27.
- 28.↵
- 29.↵
- 30.
- 31.
- 32.
- 33.↵
- 34.↵
- 35.↵
- 36.↵
- 37.↵
- 38.
- 39.
- 40.↵
- 41.↵
- 42.
- 43.
- 44.
- 45.
- 46.
- 47.↵
- 48.
- 49.↵
- 50.↵
- 51.↵
- 52.
- 53.↵
- 54.↵
- 55.↵
- 56.
- 57.
- 58.
- 59.↵
- 60.↵
- 61.
- 62.
- 63.↵
- 64.
- 65.↵
- 66.
- 67.
- 68.↵
- 69.
- 70.↵
- 71.↵
- 72.↵
- 73.↵
- 74.
- 75.
- 76.↵
- 77.
- 78.↵
- 79.
- 80.↵
- 81.
- 82.↵
- 83.
- 84.↵
- 85.↵
- 86.↵
- 87.↵
- 88.↵
- 89.↵
- 90.↵
- 91.↵
- 93.↵
- 93.