Strengths and limitations
There were limitations to the data the authors were able to collect. Information about NBCSP results were extracted from the general practice EHRs, which may be incomplete.25 In addition, the percentage of eligible individuals who completed a FIT were likely to be underestimated, as only 80% of people aged 50–60 years old attend general practice every year,26 and not everyone nominates a GP to receive their NBCSP results. In the authors’ recent trial,27 29.7% (131/441) of people did not have their NBSCP kit results in their patient record when compared with their NBCSP records; this was similar in both arms of the same trial (28.4% in the control arm, 30.8% in the intervention arm) (unpublished data). As such, the authors expect that the underestimation would be similar for both intervention and control practices, and the estimated absolute intervention effect would be unbiased. Nevertheless, the percentage of kits returned in the intervention arm was higher (39.2%) than the national average for 50–60-year-olds during the trial period (33.4% based on complete NBCSP data5).
The de-identified outcome data related to the total number of individuals eligible for the trial (denominator) and whether their FIT results were recorded in the EHR (numerator) were collected separately, so the authors were unable to link the individual-level data between the two data sources, but could link the data at the aggregate level for each practice. This limited the ability to conduct sub-group analyses to explore whether there were differences in the intervention effect by patient characteristics (for example, age groups).
Another limitation was not knowing whether people were receiving the SMS just before receiving their kit. The authors estimated when the kits would be received and timed the SMS for the month before this, according to the NBCSP rules. However, during the trial period, the NBCSP delivered kits up to 6 months after people’s birth dates, not always when they were due; this meant that it was possible that people did not receive the SMS just before they were due to receive the kit, but the authors had no way of assessing this.
The authors are addressing these limitations by using data from the newly established National Cancer Screening Register in a follow-on trial: SMARTERscreen.28 This trial is developing methods for collecting de-identified individual-level data directly from the register to ensure that the SMS is sent at the right time and the data collected are more complete; in this way, more-granular data can be provided about individual responses to the SMS, and it should be possible to ascertain the number of individuals with a FIT result, as recorded in the register.
Another potential limitation was the fact that the 12-month data collection period for each general practice was fixed from the time the practice entered the trial. However, the observation period for individuals varied, ranging from a minimum of 6 months to a maximum of 12 months, depending on the time between the SMS being sent and the end of the 12-month trial data-collection period. NBCSP monitoring data show that, if people are going to return their kit, most do so within 4 months;25 given that there was a minimum timeframe of 6 months in the study reported here, the authors were confident they would capture the bulk of returned kits. This was the same for both trial arms.
Unexpected problems were also encountered because the trial was conducted during the COVID-19 pandemic (2020–2021). However, co-design of the SMS was limited to existing evidence-based resources, expert opinion from the multidisciplinary investigator team, and consultation with PPI representatives. The authors were unable to conduct in-person consultations or focus groups with PPI as planned.
The recruitment of general practices was also challenging during the COVID-19 pandemic, with general practice facing unprecedented demands, including repeated lockdowns. Despite these challenges, the required number of practices were recruited with no attrition, and practice and patient characteristics between trial arms were similar.
The study was conducted in partnership with the WVPHN, and the sample was drawn from within the Western District of Victoria region. People living in that district live in, mostly, rural areas — half the practices were in areas categorised as MMM 4–5 (that is, either medium-sized or small rural towns) — and, as such, the findings might not be generalisable to the entire Australian population, the majority of whom live in metropolitan areas.20 As interventions have been demonstrated to be more effective in under-served groups,29 this result might not translate to other sub-populations with higher baseline screening rates. Also, the study was limited to the practices’ ‘active patients’, who were defined as having attended the practice at least three times within the previous 2 years.21 The authors excluded patients who were not regular attenders, as they assumed that ‘non-active’ patients would be less likely to nominate a GP in the trial when returning their NBCSP kit.
The results demonstrated that the SMARTscreen intervention led to the proportion of patients returning a kit being 16.5% higher; however, the screening participation data could be an underestimate for the reasons highlighted above, which would suggest the health and financial benefits might be even greater than suggested by the findings.
