In reply to the letter of DM Fleming, dated 13 December 2005, in which he raises the issue of selection bias, we would like to point out that the randomisation prevented selection bias, although some residual bias (or confounding) cannot be precluded in small trials such as ours. That, for example, was the reason why we adjusted for a slight, but important imbalance in the age distributions between the groups.1
We reported an adjusted number needed to treat (NNT) of 18.97 (95% CI: NNT [harm] = 8.92 to infinity to NNT [benefit] 5.47). This number is based on the weighted average of subgroup-specific NNTs within our trial. We reported a single NNT under the assumption that the clinically relevant subgroups in our trial have NNTs that are similar to the overall one. Of course, one may question this assumption as Dr Fleming appears to do. Thus, the important issue Dr Fleming raises is that of generalisability of trial results or effect modification. There is a large body of literature on this topic.2–4 Briefly, the question is whether clinically relevant subgroups of patients may be distinguished to which (very) different NNT apply? There are several options to explore this question. First, one may perform (biology-informed) subgroup analyses within a trial. An obvious subgroup analysis, based on the culture results, we performed ourselves. However, too many subgroup analyses are bound to yield false-positive results.5–7 Second, one may perform new trials in homogeneous subgroups of particular interest, in allergic patients for example. Third, one may try to tackle the problem in meta-analyses using meta-regression, but individual patient data meta-analysis is the preferred design.8
In conclusion, the rigorous design and execution is likely to guarantee the internal validity of our findings. Speculations on effects that deviate from the overall mean findings are always possible and cannot be rejected straightaway. For example, we agree that our trial may have under-estimated the effect of fusidic acid if many patients with a red eye based on an allergy only were included, assuming that is in such patients fusidic acid has no beneficial effect. We have no data to assess this hypothesis. However, in a modestly-sized trial such as ours the options to explore subgroup effects are extremely limited. We should welcome new well-designed and rigorously executed trials in this field.
- © British Journal of General Practice, 2006.
