The association between an individual's particular characteristics and the context within which s/he lives has been the subject of research for social and health scientists for some time.1-3 This work has had an impact on research in primary care, with the increasing use of cluster randomised controlled trials throughout the world. Recent examples have included a study evaluating the effects of injury prevention training on the knowledge, attitudes and practices of midwives and health visitors in the UK4; a study that assessed the long-term effectiveness of the ‘green prescription’ on the health of 40–79 year-old patients in New Zealand5; and a study evaluating the effectiveness of a multi-factorial intervention in reducing falls in older people in residential homes in Sweden.6
The essential nature of health service organisation has meant that the use of these trials has become common, replacing the more traditional trials where participants are individually randomised. Proponents of cluster randomised controlled trials argue that in ‘real life’ people are socially grouped in a way that their individual characteristics may be linked to being part of that group. In primary care research these ‘clusters’ may be clinics or general practices, for example. By ignoring the contribution of these groupings on people's knowledge, attitudes and behaviours in designing and analysing studies, an aspect of potential bias is ignored.
Trial design issues
There are some issues that are pertinent particularly to cluster randomised trials. For example, as a result of the loss of power due to the variability between clusters, sample sizes need to be larger than in individually randomised trials. Estimates are needed, usually from previous studies of the ‘intra-cluster correlation coefficient’. If this ‘design effect’ is large then more participants will be needed than in an individually randomised trial.7,8 and if the cluster sizes are unequal this will also impact on the sample size needed. This makes the trial more expensive, some say, unnecessarily so.9
A clustered design also means that contamination between intervention and control arms may be reduced overall, although detractors say that the whole issue of contamination between arms of the trial has been overstated. Cluster trials are only more efficient when contamination exceeds 30% (which is rare) and that it can be overcome by increasing the sample size.9
Previously, there have also been problems of area level deprivation being used to infer something about individual level characteristics. This is known as the ‘ecological fallacy’.10 Inferences from aggregate data cannot be applied to individuals when attempting to look at the effects of contextual differences. Therefore, the usefulness of multilevel modelling techniques for analysing data from randomised trials is their ability to partition the variance between individual and group level characteristics and also to explore the interaction between them.
Multilevel modelling techniques
The procedures used for analysing cluster randomised controlled trials have their origin in educational research and are known as multilevel modelling techniques.11 Many educational studies have examined the effectiveness of schools as compared by their individual performance and the performance of the pupils studying within them. These developments have been accompanied by the development of the appropriate statistical techniques.12,13 The example often given is one of comparing exam marks of pupils in different schools.12 The model looks at the relationship of the entrant reading score as the predictor variable. The average of the school could be taken as an aggregate measure (that is, all pupils' marks together) and then a regression model would be used. We would then have to choose whether to carry out the analysis at the pupil or the school level. Proponents of this model say that these single-level analyses are not satisfactory. If the model ignores the individual student scores then useful information, which would be relevant to causality, is discarded. However, if the school level data is ignored, this leads to inefficient procedures because separate terms for each school would be used and therefore it is not easy to generalise. As schools are not treated as a random sample it provides no useful quantification of the variation among schools in the population more generally. Multilevel models provide the opportunity to look at the different levels of hierarchy in the population and then to see where the effects are occurring. It allows for better estimates of simple questions and also allows for more complex questions to be answered.
The multilevel modelling technique is an extension of multivariate regression. Within the regression equation extra terms are added to take into account the clustered nature of the data (Box 1). The multilevel modelling technique is therefore actually a range of models using either fixed or random effects. The basic tenet of multilevel modelling techniques is that any statistical model should recognise a hierarchical structure where one is present, and if this is not done the consequences of failing to do this must be recognised.
Multilevel modelling can be very powerful and useful for disentangling the effects of different variables in health data,10 but because it is rarely used and relatively complicated, detractors argue there is the chance that it can be used and reviewed incorrectly.14 The facility of complex computer packages for data analysis can lead to unnecessary confusion when more simple techniques would suffice.
In their systematic review for the Health Technology Assessment programme,15 Ukoumunne and colleagues found that of the 56 papers which reported cluster-based interventions from seven health science journals, very few followed the methodological recommendations that they suggest. Therefore, there is a need to improve the quality of conducting and reporting studies that use multilevel modelling techniques.
Gorard,14 in outlining the uses of multilevel modelling, highlights the lack of research capacity available in order to critically analyse studies that have used such techniques. This is clearly quite a problem and highlights the need for further training and capacity building in primary care. Nevertheless, multilevel modelling does offer opportunities to look at health issues in new and different ways. As long as we keep in mind the inherent problems we may discover an exciting ‘new world’ of possibility.
Details of the basic multilevel model
The basic multilevel model is an extension of the ordinary least squares regression demonstrated by the equation:
where y is the outcome variable, a is the intercept, b is the dependent variable, x the coefficient of the slope, and e is the error term, known as the residual. The residual is assumed to be normally distributed with a mean of zero and a constant variance. However, patients in a particular cluster may share some common influences: the health professionals, the treatments they receive, higher consulting rates, overworked staff, for example. If two patients taken together with the same characteristics were compared the predicted outcome may depend on the cluster they are in. Thus, with a simple extension the cluster can be incorporated into the equation. If j refers to the cluster and i refers to the individual patient then the model would be described by the following equation:
where uj is the variation between practices on the outcome of interest. It is assumed to be a random variable with a mean of zero and constant variance. The distribution of the cluster effect (uj) is the estimation of the variance across all of the clusters involved in the study. If the variance is large then the outcome of interest is dependent on the cluster, if the variance is small then the variations in outcome of interest may be explained by the measured characteristics alone. The size and the statistical significance of the effect of the cluster can be measured using a multilevel model.
Questions for Chekhov
You were no man for evangelical discomfort,
no ascetic of the Sermon on the Mount:
so what drove you to the limit, yours and Russia's,
and made you, briefly, prison inspector?
The carriage that took you was unspringed.
Novosibirsk, Yenisey, Baikal, Amur:
dry feet (you commented) were the highest good;
distance in the taiga lived on and on.
Opening the door directly on the Milky Way
you surprised yourself, recognising
desolation's deepest ore in a convict's speech.
(Better said: your body recognised it, craving sleep.)
Was that what it took to be inspector
of the bare-faced? The only voices in the island
were the wind's. Eskimo wisdom
out of Diogenes: vastness far from human.
Iain Bamforth
The centenary of the death of Anton Chekhov falls on July 4 2004.
- © British Journal of General Practice, 2004.