Enhancing ventilation in homes of children with asthma: cost-effectiveness study alongside randomised controlled trial

Study population

The trial took place from 2004 to 2006 in Wrexham, North Wales, UK. Children were recruited through their GPs; 20 out of the 23 practices within Wrexham Local Health Board participated. Households were randomly allocated to the intervention group or to a delayed-intervention control group who received housing modification after the end of the trial.

Intervention

Each child's household was visited by a local authority housing officer, who assessed the improvements needed. Ventilation systems were installed in the roof spaces of houses. Improvements were made to bring central heating systems to a defined standard; new systems were installed if none existed. There was no cost to the families for these improvements. The companion paper by Woodfine et al gives more detail about the study population, inclusion and exclusion criteria, and intervention.11

Measurement of effectiveness

The main outcome measure in this cost-effectiveness study was the parent-reported asthma module of PedsQL™, a validated quality-of-life measure for children with asthma.12 The study used the total score for asthma-related quality of life on a 100-point scale, measured at the final follow-up 12 months after randomisation. This asthma-specific measure was chosen, as there is no generally accepted technique for measuring utility in young children.13 Data were also collected on the frequency of primary and secondary healthcare use and asthma-related prescribing.

Measurement of costs

Cost-effectiveness analysis was undertaken from a multi-agency, public-sector perspective. This combined the perspectives of Wrexham County Borough Council, which bore the cost of housing improvements in this trial, and the NHS, which bears the healthcare costs associated with childhood asthma. Wrexham County Borough Council provided information on the housing modification received by each household in the intervention group, and associated costs. As this study takes a public-sector perspective, it has not included the running costs of ventilation systems, estimated at £15 a year, or costs to families of running central heating.

Parents were asked to use the validated Client Service Receipt Inventory,14 adapted to recall their child's contacts with health services over the study period. A researcher later visited each participating practice to abstract from children's notes a structured record of the type and frequency of consultations, visits to emergency departments or outpatient clinics, inpatient stays, and prescribed drugs. It was found that parents' and practices' reports of healthcare use were not entirely consistent. However, both approaches led to similar conclusions about differences between groups, and thus about cost-effectiveness. After consideration, it was decided to report estimates of resource use from general practice records, because these records included prescribing information.

National costs were obtained from the report by Curtis and Netten for units of health and social care;15 the Department of Health for NHS reference costs;16 and the Health and Social Care Information Centre for prescribing costs.17 All costs are in £ Sterling for 2006. As participants were followed for only 12 months, discounting was not necessary.8

Analysis of effects

Like the primary effectiveness analysis in the companion paper, the economic analysis focused on 177 of the 192 children randomised in the trial.11 Fifteen children who had no follow-up questionnaire were excluded. For eight children who did not complete a 12-month PedsQL questionnaire, scores at 12 months were imputed by regression on scores at baseline and 4months. Baseline PedsQL and complete cost data were available for all 177 children. Analysis of covariance was used to adjust effects for baseline scores.

Analysis of costs

Health-service use and prescribing of asthma drugs were compared between the intervention and control groups. As distributions of both frequencies and costs were skewed, frequencies were compared by non-parametric tests, and costs by bootstrapping.

Costs in the 12 months before randomisation did not differ significantly between groups. The possibility of adjusting costs for the corresponding costs before randomisation, as for effects, was investigated. Although primary-care costs before and after randomisation were related, secondary-care costs were highly skewed, and extreme values before randomisation hardly related to extreme values after randomisation. Hence, adjusting for past history did not improve the precision of estimates. For consistency, therefore, the analysis did not adjust any costs.

Cost-effectiveness analysis

As the trial used a condition-specific outcome rather than a utility measure, cost-effectiveness analysis, rather than cost-utility analysis, was used.8 This analysis compared children's adjusted change over 12 months in asthma-related PedsQL score with their healthcare costs, augmented by the household cost of housing modifications in the intervention group.

Incremental cost-effectiveness ratios (ICERs) were estimated, and bootstrapping was used with 1000 replications to generate confidence intervals (CIs). A cost-effectiveness plane was generated, showing the joint distribution of bootstrapped costs and effects, and a cost-effectiveness acceptability curve was also generated, covering a range of cost-effectiveness thresholds. Where possible, a bootstrapped 95% CI for the ICER was derived from the cost-effectiveness acceptability curve.

Choice of threshold ICER

Cost-effectiveness acceptability curves were used to show the probability that the intervention is cost-effective over the range £0 to £600 per PedsQL point. There is no guidance from the National Institute for Health and Clinical Excellence (NICE) or the literature as to what society should be prepared to pay for improvements in disease-specific quality-of-life measures like the PedsQL.18,19 The results of this study therefore show how this intervention has shifted distributions of PedsQL scores in the intervention and control groups, and relate this to the average cost per child of the intervention to Wrexham County Borough Council.

Sensitivity analyses

Economic evaluation uses sensitivity analysis to investigate how sensitive findings are to basic assumptions, by varying those assumptions. Building costs vary across the UK, so the ICER was recalculated to test whether the study findings depend on building costs in North Wales, which are lower than those in London but higher than those in Northern Ireland.20

Following recent guidance that economic analysis of public-health interventions should give more weight to equity,21 the study tested whether its findings would change if the criterion for housing improvements (asthma severity) were more stringent. The sample of children was subdivided at the median of baseline asthma scores (median = 67), and each half of the sample was analysed separately. As PedsQL does not distinguish between moderate and severe asthma, this equal division maximises the power of the comparison and offers a working definition of ‘severe’ asthma.

Effectiveness

At baseline, the 177 children analysed in this, and the companion, paper were similar in all respects to the 15 children excluded for lack of follow-up data. The distributions of PedsQL scores at baseline were similar in the intervention and control groups, as was the distribution of costs. After 12 months, the PedsQL showed a mean (adjusted) improvement of 8.65 in the intervention group and 1.58 in the control group.11 The resulting difference of 7.07 (95% CI = 2.79 to 11.36) between groups is equivalent to a standardised effect size of 0.42.

Costs of housing modification

The housing officers assessed 38 children (19 intervention, 19 control) as needing both ventilation and heating improvements, and the remaining 139 (69 intervention, 70 control) as needing only ventilation. However, six of the intervention group assessed as needing only ventilation subsequently received both ventilation systems and central heating improvement.

In contrast, two intervention households received neither ventilation systems nor central heating improvement, for family reasons. These were analysed by ‘intention to treat’. Table 1 includes the costs of housing modification to improve ventilation, and heating if necessary, as estimated by Wrexham County Borough Council. These programme-specific building costs include a standard 12% managerial overhead including surveying costs. The mean total cost of housing modification was £1718. The mean cost of improving ventilation and heating (excluding six houses described above) was £3675; the mean cost of improving ventilation alone was £1179 (including these six).

Health-service use by children

Table 2 summarises the costs of health-service use by children in the intervention and control groups over 12 months. There was no consistent difference between the two groups. Primary care costs were similar in the two groups, while the costs relating to secondary care over 12 months were lower, but not significantly lower, in the intervention group, mainly through fewer outpatient consultations. The total NHS costs were £61 lower for the intervention group than the control group, but this difference was also not significant. However, the total costs, including the costs of housing improvements, are inevitably much lower in the control group: £560 compared with £2217. The mean difference is £1657 with bootstrapped 95% CI = £1282 to £2036.

Prescribing

In the year before randomisation, mean prescribing costs in primary care were £106 for the intervention group and £111 for the control group. Over the next year these costs rose to £122 in the intervention group and £136 in the control group; this difference was not significant. Table 2 shows the costs of prescribing over 12 months by category within the British National Formulary. Most costs arise from prescribing bronchodilators and corticosteroids. Although bronchodilators cost more in the intervention group, corticosteroids cost more in the control group; neither difference was statistically significant.

Total NHS costs at baseline and follow-up

In the year before randomisation (baseline), mean NHS costs in the intervention and control groups were £476 and £663 respectively. Although in the year after randomisation (follow-up) these costs rose to £499 in the intervention group and fell to £560 in the control group, there was again no statistically significant difference at baseline, at follow-up, or in the change from baseline to follow-up.

Cost-effectiveness

Using the cost-and-effect data gathered over 12 months for 88 intervention households and 89 control households, an incremental cost-effectiveness ratio (ICER) was estimated. The difference between groups in mean total cost (£1657) was divided by the difference between groups in mean effect (7.07 PedsQL points), to yield an estimated ICER of £234 per unit change in the PedsQL asthma-specific quality-of-life

Analysis of uncertainty

Figure 1A shows the resulting cost-effectiveness plane. All 1000 bootstrapped points fall within the northeast quadrant of this plane, where the intervention is both more costly and more effective than usual treatment.22 Figure 1B shows the corresponding cost-effectiveness acceptability curve, plotting the probability that housing modification is cost-effective against the range of thresholds below which decision-makers are willing to pay for such modifications.23,24 As all points are in the north-east quadrant, the curve intercepts the y axis at 0.0 and approaches 1.0 as the threshold increases. The probability of the intervention being cost-effective is 2.5% at a threshold of £140, and 97.5% at £590. In other words, the bootstrapped 95% CI for the ICER of £234 per unit change in the PedsQL asthma-specific quality-of-life score, runs from £140 to £590.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Cost-effectiveness planes (1A, 1C, 1E) and cost-effectiveness acceptability curves (W, W, 1F) for all children (1A, 1B), children with severe asthma (1C, 1D) and children with moderate asthma (1E, 1F). Figures 1A, 1B: 88 in intervention group and 89 in control group. Figures 1C, 1D: 42 in intervention group and 44 in control group. Figures 1E, 1F: 46 in intervention group and 45 in control group.

Sensitivity analyses

The ICER and estimated probability that housing modifications are effective depends on local building costs.15 The authors estimate that in Wales, where the study took place, the intervention has an ICER of £234 (95% CI=£140 to £590); with the same effectiveness, this intervention in London would have an ICER of £294 (95% CI = £174 to £770), and in Northern Ireland, the ICER would be £166 (95% CI = £98 to £430).

For the 44 children in the control group and 42 children in the intervention group with more severe asthma (that is, baseline PedsQL score below the median of 67), the mean difference in costs was £1590, and the mean difference in effect was 9.67 PedsQL points, yielding an ICER of £165 (bootstrapped 95% CI = £84 to £424; Figures 1C and 1D). For the 45 children in the control group and 46 children in the intervention group with more moderate asthma (that is, baseline PedsQL score at or above the median of 67), the mean difference in costs was £1730, and the mean difference in effect was 4.56 PedsQL points, yielding a higher ICER of £379 (Figures 1E and 1F). Because 5% of bootstrapped points fell in the north-west quadrant of the cost-effectiveness plane, where the intervention is more costly and less effective than usual treatment, it was not possible to derive a two-sided 95% CI for the subsample with more moderate asthma. Instead, a onesided 95% CI was derived: if society were willing to pay only up to £180 (the ‘threshold’) per point improvement on the PedsQL scale, housing modification would be less cost-effective than no action. At a threshold of £300 per unit improvement on the PedsQL score, housing modification has a 93% probability of being cost-effective for children with more severe asthma, compared with 35% for children with more moderate asthma (and 76% for all children). This difference is due mainly to a larger (although not significantly larger) effect in the subsample of children with more severe asthma.

Costs of shifting the distribution of asthma scores in children

Figure 2 shows that tailored heating and ventilation housing modifications led to a 17% shift of children in the intervention group from ‘severe’ to ‘moderate’ asthma, compared with a 3% shift for the control group. The cost to Wrexham County Borough Council was £1718 per child in the intervention group or £12 300 per child shifted from ‘severe’ to ‘moderate’.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Changes in the distribution of asthma scores from baseline to 12 months. All distribution curves are smoothed. Baseline distributions are very similar in the two groups, and the baseline median of both intervention and control groups is 67 (although the proportions are marginally different, as a few score exactly 67), so the same baseline curve is used in both graphs.

Summary

This economic evaluation extends the findings of the CHARISMA trial.11 The analysis estimates the cost of a single-point improvement in the PedsQL asthma-specific quality-of-life score at £234. At baseline, the median reported asthma-related quality-of-life score was 67, with a lower quartile of 56 and an upper quartile of 79; 12 months later, parents of children in the intervention group reported a median score of 76, with a lower quartile of 64 and an upper quartile of 87. In summary, tailored ventilation and heating modifications moved 17% of children in the intervention group from below the original median (‘severe asthma’) to above that median (‘moderate asthma’), while only 3% of controls moved from ‘severe’ to ‘moderate’. Thus, a net 14% of children (or 29% of children with ‘severe’ asthma) became ‘moderate’. The cost to Wrexham County Borough Council was £1718 per child in the intervention group, or £12 300 per child shifted from ‘severe’ to ‘moderate’. Thus, the installation of a ventilation system, and central heating where necessary, in the homes of children with moderate to severe asthma improves their respiratory health and quality of life. These housing modifications are likely to be cost-effective.

Strengths and limitations

The CHARISMA trial shows that there is scope to apply a rigorous economic approach alongside a pragmatic trial to evaluate community-based public-health interventions.25 As resources were limited, however, participating children were only followed for 1 year, and the study did not assess benefits to siblings or parents, measure children's respiratory function, or analyse mould spores in their houses. The measurement of respiratory function is difficult and expensive, and was too variable to contribute to a recent trial of heating alone.26

Comparison with existing literature

A recent Cochrane Review of chemical and physical methods to control asthma concluded that they were ineffective, and recommended that further studies should be more rigorous.27 By using a randomised controlled trial to show that ventilation systems improve respiratory quality of life, CHARISMA has responded to this challenge. Furthermore, it has added the first rigorous cost-effectiveness analysis of housing modifications to address childhood asthma.

Implications for practice and research

Wanless, in his report, previously challenged health economists to produce rigorous evidence of the cost-effectiveness of public health interventions.28 In response, CHARISMA provides evidence that improving ventilation, and heating where necessary, in homes of children with moderate to severe asthma is an effective use of public resources.

Nevertheless, the findings of this study almost certainly underestimate the full value of the effects of housing modification. Given the net benefit of 7 PedsQL points to children in the intervention group after 12 months, it is inconceivable that they gain no benefit after that. Furthermore, several parents with asthma spontaneously reported that they too had benefited. Both observations illustrate the published guidance on the economic analysis of public-health interventions.21 This recommended that analysts should strive, not only to estimate benefits in the long term, but also to consider benefits to individuals not directly targeted by the intervention. Although this guidance cannot be followed directly, the indirect evidence about long-term and parental benefits allows a conclusion that housing modification is likely to be cost-effective.

Despite the clear improvement in the health of children in the intervention group, their healthcare costs, notably repeat prescriptions, continued at their previous level. As the beneficial effects of housing modification become more widely known, the authors hope that the potential for reducing the prescription of asthma drugs will be recognised.

Sensitivity analysis showed that the cost-effectiveness of housing modifications was dependent on local building costs. As CHARISMA almost certainly underestimated the full value of housing modification, the authors see little reason for geographical variation in the uptake of these modifications. Sensitivity analysis also showed that cost-effectiveness differed between children with more severe asthma (whose ICER was £165), and those with less severe asthma (whose ICER was £379).

Although this subgroup analysis had lower power than the main trial, the authors conclude that the case for improving the housing of children with ‘severe’ asthma is even more cogent than for children with ‘moderate’ asthma.

Although the findings of CHARISMA are encouraging, NICE needs research of two kinds before recommending that local authorities install ventilation systems in the houses of children with asthma. First, a multicentre trial is recommended, following children for at least 2 years and measuring a wider range of benefits, including benefits to other members of the family. Second, NICE should commission research to measure utility in children.