Abstract
Background Recruiting adequate numbers of participants represents a major problem to the completion of randomised clinical trials in primary care. Information on different recruitment strategies applied in one trial is scarce.
Aim To evaluate the application of two recruitment strategies in one trial.
Design of study The study was performed within the framework of a randomised clinical trial on the effectiveness of a behavioural treatment for patients with chronic shoulder complaints.
Setting Thirty-two general practices in the Netherlands.
Method Patients recruited during a consultation with their GP for chronic shoulder complaints were compared with patients recruited by advertisement in a local newspaper as regards baseline characteristics, withdrawals (drop-outs and losses to follow-up) and post-treatment clinical outcomes.
Results Patients recruited by the GPs (n = 83) were similar to those recruited by advertisement (n = 83) in terms of demographic characteristics and clinical outcome measures at baseline, but differed slightly in disease characteristics and treatment preferences. Recruitment strategy was not related to reasons for or numbers of withdrawals. Improvements on outcome measures were greater in patients recruited by the GPs, irrespective of allocated treatment. Results on the clinical effectiveness of treatments at the end of the treatment period or during follow-up were neither modified by recruitment strategy, nor by differences between the two strategy groups in patient characteristics found at baseline.
Conclusion Using two recruitment strategies did not influence the outcomes on clinical effectiveness in this trial. However, recruitment strategy should be considered as a putative modifying factor in the design of a study.
INTRODUCTION
Recruiting adequate numbers of participants currently represents a major problem to the completion of randomised clinical trials, in particular in primary care.1 Several studies on recruitment in primary care have been done so far, mainly focusing on preventing and overcoming recruitment problems before starting a study. Common outcome measures in these studies are numbers of referrals and recruitment rates.
Several studies have reported on characteristics of patients, the general public or GPs, and on methodological options to overcome recruitment problems in primary care settings. Willingness to participate and attendance among patients seem to be associated with demographic factors (age, sex, marital status, social class, occupation and education); attitudes (readiness to change); and disease characteristics (clinical measures, health status, risk factor status).2–8 In general, demographic characteristics and attitudes seem to be better predictors of recruitment rates and attendance than disease characteristics. Treatment preferences of participants might influence the outcome in non-blinded studies.9–11 Patient recruitment by GPs seems largely determined by the GPs' motivation, involvement, forgetfulness, time pressure and financial reimbursement.12–15 Methods and strategy of recruitment, for example, screening by a GP, waiting-room screening, or recruitment by advertisements, seem to be related to the yield in numbers of qualified patients and the costs per recruited patient.5–16 Organisational characteristics and the simplicity of printed information for the patients and GPs seem to influence recruitment rates as well.1,17,18
How this fits in
Recruiting adequate numbers of participants currently represents a major problem to the completion of randomised clinical trials in primary care. Earlier studies mainly focused on recruitment rates and how to overcome recruitment problems before starting a trial. The present study evaluates whether patients recruited during their consultation with the GP were comparable with patients recruited by advertisement in a local newspaper as regards baseline characteristics, withdrawals (drop-outs and losses to follow-up) and post-treatment clinical outcomes in one trial.
One question, which has not been answered so far, is what should be done if, despite all efforts, the recruitment stage delays or hampers the proper conduct of a trial? The present study evaluated the application of two recruitment strategies in one trial examining the effectiveness of a behavioural treatment for patients with chronic shoulder complaints. Since the numbers of patients recruited during GP consultations were smaller than expected, subjects were also invited to enrol by means of advertisements in a local newspaper. Since patient inclusion and exclusion criteria were similar for both strategies, one might assume that the characteristics of patients enrolled by both strategies would be comparable and that the recruitment method would not influence the course and outcomes of the trial. To evaluate this assumption, we compared the effects of these two strategies with respect to baseline characteristics, withdrawals (drop-outs and losses to follow-up) and post-treatment outcomes.
METHOD
Design
This study was performed within the framework of a trial on the effectiveness of a behavioural graded exercise therapy programme for patients with chronic shoulder complaints. The programme aims to improve performance of daily activities and healthy behaviours and to reduce healthcare utilisation during the treatment period and the 1-year follow-up period. More details of the design and the results of this trial have been presented elsewhere.19–20
Two strategies were used to enrol patients in this trial: recruitment by GPs during consultations for shoulder complaints (from January 2002 until July 2003) and recruitment by advertisement in a local newspaper (from March 2003 until July 2003). We used block randomisation, with blocks of 10 patients, to allocate participants either to graded exercise therapy or to usual care. To compare the effects of the two recruitment strategies, we assessed patient characteristics at baseline, number of withdrawals (drop-out rate and losses to follow-up), reasons for withdrawal, and the influence on primary outcome measures (main complaint instrument and Shoulder Disability Questionnaire) at the end of the treatment period (12 weeks) and during follow-up (26 and 52 weeks).
Participants
Patients with shoulder complaints for at least 3 months, aged 18 years and over and living in the Province of Limburg (the Netherlands), were invited to participate in the study.
Recruitment strategies
Recruitment during consultation in general practice
Forty-nine GPs working in the area of the 20 physiotherapists involved in the study, were invited to participate. Thirty-two GPs agreed to participate and recruited patients who visited their surgery and met the selection criteria. A research assistant visited potential participants at home within 20 weeks after the GP had seen them.
Recruitment by advertisement in a local newspaper
An advertisement in a local newspaper (circulation rate of 159 000 addressees) was used to enrol patients. After the research assistant had checked their eligibility by telephone, patients received written information about the content of the study. The research assistant then contacted the patient's GP. If the GP was not able or did not want to check the selection criteria, an independent GP associated with the trial team checked these. If patients met the selection criteria, a research assistant visited them at home within 2 weeks.
Outcome measures
Baseline characteristics
Demographic variables, disease characteristics, putative prognostic factors, treatment preferences and outcome variables of clinical effectiveness were determined at baseline.
Drop-out rate and loss to follow-up
Drop-out rates during treatment and losses to follow-up were assessed. We also determined reasons to withdraw for both strategies in relation to allocated treatment.
Outcomes after treatment and during follow-up
Differences between the two recruitment strategy groups in terms of clinical outcome measures were established for both treatments at 12, 26 and 52 weeks. We assessed the influence of the recruitment strategy and possible confounders or modifying factors on the primary outcome measures of clinical effectiveness of treatments.
Statistical analysis
Analyses were carried out with SPSS statistical software (version 12.0) according to the intention-to-treat principle. A P-value of <0.01 was considered to be statistically significant (two-tailed) for all comparisons.
First, the sample size was calculated based on figures of perceived recovery.21 Since we aimed to increase the recovery rate from 25% to 50%, at a one-sided α of 5%, a statistical power (1 - β) of 0.90, and a 10% dropout rate, we needed 66 persons per treatment group.
Subsequently, the standardised difference (the ratio of the differences between groups to the standard deviation of the observations) was determined according to the sample size of the trial as a measure of the expected differences in outcome post treatment.22 The standardised difference was determined to be 0.52. The differences between recruitment groups on clinical outcome measures at baseline were considered with respect to the determined standardised difference to evaluate the influence of recruitment strategy at baseline.
Then, effect sizes (the ratio of the differences in mean change scores to the standard deviation of the average change scores of the total population) were calculated to evaluate the differences between recruitment groups on clinical outcome measures after treatment and during follow-up. Effect-sizes of 0.2, 0.5, and 0.8 were considered as small, medium and large beneficial effects respectively.23 Confidence intervals (95%) of the mean differences were determined and analysed by Student's t-tests for data with a Gaussian distribution, and by Mann–Whitney tests for non-Gaussian distributions. Furthermore, we analysed the differences between the recruitment groups for both treatments.
Finally, the influence of recruitment strategy and possible confounding or modifying factors that differed between recruitment groups at baseline on the primary outcome measures, was analysed using multivariate linear regression and a stepwise forward procedure (P<0.10). Covariates that were significantly associated with the primary outcome measure after the variable ‘treatment’ was included in the model, remained in the final model and were considered as confounders. If the interaction term (the product of the covariate and ‘treatment’) was significantly associated as well, the covariate was considered as a modifying factor. Adjusted estimates were reported in case of confounding or effect modification.
RESULTS
Ninety patients were recruited by the GPs during consultations (Figure 1). Seven of these patients (8%) were excluded before randomisation. One hundred and two patients recruited by advertisement were eligible to participate according to their GPs. Nine of these patients (9%) were excluded before randomisation. A total of 176 patients were randomised and allocated to either graded exercise therapy (n = 87) or usual care (n = 89).
Table 1 shows that the two recruitment groups were comparable on most demographic variables and all clinical outcome variables at baseline. Patients recruited by advertisement had a higher average educational level. Duration of complaints was longer for these patients, whereas periods of sick leave were shorter.
Figure 1 shows that reasons to withdraw from the study, drop-out rates, and losses to follow-up at 52 weeks were similar for both recruitment strategy groups (16% for GP recruitment versus 17% for recruitment by advertisement).
Patients recruited by advertisement were slightly less ambivalent in their preferences for commonly applied shoulder complaints treatments and prior treatments had been less successful in these patients (Table 2).
Table 3 shows that mean improvements among patients recruited by the GPs were statistically significantly greater for the main complaints instrument at 52 weeks (P = 0.001; effect size = 0.50) and for the Shoulder Disability Questionnaire at 26 weeks (P = 0.005; effect size = 0.43) and 52 weeks (P = 0.004; effect size = 0.44).
Table 4 shows that patients recruited by the GPs showed greater improvement on both primary outcome measures, irrespective of treatment allocation.
Multivariate linear regression analyses (Table 5) shows that recruitment strategy was significantly associated with both primary outcome measures (the main complaint instrument at 52 weeks and the Shoulder Disability Questionnaire at 26 and 52 weeks). This indicates that recruitment strategy was a confounding factor. There was no interaction between graded exercise therapy and recruitment strategy (the interaction term of recruitment and ‘treatment’ failed to reach significance and was excluded from all models). The lack of interaction indicates that the clinical effectiveness of the programme, as assessed by the primary outcome measures at the end of treatment and during follow-up, was not modified by the recruitment strategy. Other possible modifiers (educational level, duration of pain, duration of limitations, sick leave or treatment preferences) were not significantly associated with these clinical outcome measures and were excluded from all models.
DISCUSSION
Summary of main findings
Patients recruited for a randomised clinical trial by GPs and by advertisement were comparable in terms of most baseline characteristics and numbers of withdrawals (drop-outs and losses to follow-up). Patients recruited by the GPs showed greater improvement on both primary outcome measures, irrespective of treatment allocation. Multivariate regression analyses showed that clinical effectiveness at the end of the treatment or during follow-up was not modified by the recruitment strategy, or by baseline differences between the two recruitment strategy groups.
Strengths and the limitations of the study
Our results show that findings on effectiveness in the trial are applicable to both patients who consulted their GPs for their complaints and patients who did not intend to visit their GPs. However, the use of two recruitment strategies did introduce a confounding factor. Although multivariate regression analyses showed that the recruitment strategy did not modify the outcome of the trial in terms of effectiveness, we found that mean improvements among patients recruited by the GPs were greater, irrespective of treatment allocation. In our trial, clinical effectiveness was not influenced, because patients recruited by the GPs and by advertisement, were allocated by block randomisation and equally balanced for both treatment groups. If this had not been the case, outcomes could have been biased. For example, small differential effects between recruitment groups would have been more pronounced if 75% of patients had come from advertisement and 25% from the GP.
Patients recruited by the GPs and those enrolled by advertisement turned out to be only slightly dissimilar with respect to level of education, duration of complaints, sick leave and treatment preferences. Sick leave among patients recruited by the GPs during consultations was longer, which suggests that sick leave might be a reason for patients to visit their GPs because of shoulder complaints. Since blinding was not feasible at the patient or provider level, treatment preferences may have influenced outcome measures of effectiveness.10,11 However, multivariate linear regression showed that these possible modifiers did not alter the results on clinical effectiveness.
Patients recruited by the GPs showed greater mean improvements on the primary outcome measures than those recruited by advertisement, irrespective of treatment allocation. Effect sizes between recruitment groups reached up to the level of the expected difference in outcome post treatment (0.50). We were unable to explain these findings by possible confounders or modifying factors in our dataset.
Comparison with existing literature
Several studies on the effectiveness of cognitive behavioural treatment have reported on influences of pre-treatment attitudes (readiness to change and motivation) and beliefs (expectancies) on pain and pain disability.24–30 These studies showed that pain-catastrophising, pain anxiety, pain-related fear, helplessness and unsuccessful prior treatment experiences seem to be associated with over-predicting actual pain and under-predicting abilities to perform physical tasks and to adjust pain behaviour, and these patients have low treatment expectancies. By contrast, self-efficacy, better pain coping and control, pain acceptance and readiness to change seem to be associated with decreased pain, improved physical performance and better adjustment to pain. Those patients probably have higher treatment expectancies.
Since the primary outcome measures in our trial were subjective, we assume that pre-treatment attitudes and beliefs may also have influenced the prognosis of patients in this study. We considered the possibility that pre-treatment expectancies and readiness to change in patients recruited by the two strategies in this trial may have been dissimilar. To begin with, patients recruited by the GPs had decided to visit their GPs for their complaints, whereas patients recruited by advertisement did not intend to contact their GPs. Patients recruited by the GPs were asked to participate, whereas those recruited by advertisement reacted spontaneously to an announcement in a local newspaper. Furthermore, we found that patients recruited by the GPs had had more successful experiences with all treatments commonly applied for shoulder complaints. As a result, their expectations about shoulder treatments in general may have been higher, whereas expectations among patients recruited by advertisement were probably lower as regards commonly applied treatments and probably only higher as regards the new behavioural programme. Unfortunately, detailed information on these attitudes and beliefs, as possible prognostic factors, was not available in our dataset.
Eighty-three patients were enrolled by the GPs over a period of 14 months, whereas 93 patients were recruited by advertisement over a period of 4 months. In terms of efficiency, the recruitment rate for advertisements was thus considerably higher. These findings confirm that recruitment strategies in which the GP plays an active role as gatekeeper are less efficient.12,16
Implications for future research
Demographic and disease characteristics are important variables with respect to recruitment rates, as has been reported above. In the present study, however, these factors did not account for differences we found in the prognosis of patients recruited by different strategies. Therefore, we recommend that more specific potential modifiers relating to recruitment strategy (for example, beliefs and attitude) should be included in the design of studies and recruitment strategies (in addition to the above-mentioned demographic and putative disease-specific prognostic factors). We conclude that using two recruitment strategies did not influence the outcomes on clinical effectiveness of our trial. However, this conclusion is based only on analyses of our data and differential effects might be different if other groups have to be adopted. Recruitment strategy should be considered as a putative modifying factor in the design of a study and we recommend further research into this phenomenon.
Acknowledgments
The authors wish to thank all participating general practitioners and physiotherapists for their contributions, Marlies Smeulders for her contribution to the selection process, and Natali de Bruijn and Sjan Heijkers for data collection.
Notes
Funding
This study was funded by the Netherlands Organisation for Scientific Research (NWO-MW, grant number 904-65-901) and by the 'De Drie Lichten Foundation, Hilversum, the Netherlands (00/60)
Ethical approval
The study was approved by the Medical Ethics Committee of the Institute for Rehabilitation Research iRv (METC-01-0012)
Competing interests
The authors have stated that there are none
- Received January 29, 2005.
- Revision received April 29, 2005.
- Accepted July 29, 2005.
- © British Journal of General Practice, 2006.