Measuring the complexity of general practice consultations: a Delphi and cross-sectional study in English primary care

Background The complexity of general practice consultations may be increasing and varies in different settings. A measure of complexity is required to test these hypotheses. Aim To develop a valid measure of general practice consultation complexity applicable to routine medical records. Design and setting Delphi study to select potential indicators of complexity followed by a cross-sectional study in English general practices to develop and validate a complexity measure. Method The online Delphi study over two rounds identified potential indicators of consultation complexity. The cross-sectional study used an age–sex stratified random sample of patients and general practice face-to-face consultations from 2013/2014 in the Clinical Practice Research Datalink. The authors explored independent relationships between each indicator and consultation duration using mixed-effects regression models, and revalidated findings using data from 2017/2018. The proportion of complex consultations in different age–sex groups was assessed. Results A total of 32 GPs participated in the Delphi study. The Delphi panel endorsed 34 of 45 possible complexity indicators after two rounds. After excluding factors because of low prevalence or confounding, 17 indicators were retained in the cross-sectional study. The study used data from 173 130 patients and 725 616 face-to-face GP consultations. On defining complexity as the presence of any of these 17 factors, 308 370 consultations (42.5%) were found to be complex. Mean duration of complex consultations was 10.49 minutes, compared to 9.64 minutes for non-complex consultations. The proportion of complex consultations was similar in males and females but increased with age. Conclusion The present consultation complexity measure has face and construct validity. It may be useful for research, management and policy, and for informing decisions about the range of resources needed in different practices.


INTRODUCTION
GPs in the UK report increasing pressure from a rising workload. 1,2 The number of consultations increased by 14% between 2007 and 2014, and the mean duration of face-to-face consultations increased by 7%. 3 Doctors' perceptions of an increasing workload may reflect an increase in the complexity as well as number of consultations. This may be associated with an ageing population, increasing prevalence of multimorbidity and polypharmacy, transfer of activities from secondary to primary care, increasingly complex clinical guidelines, and growing policy expectations of what can be achieved within each consultation. The increasing delegation of routine tasks to other staff is also likely to increase the proportion of general practice consultations that are complex and intellectually demanding. 1,4 To test this hypothesis, it is necessary to define and measure complexity within general practice consultations. A measure suitable for research and analysis needs to be applicable to routine electronic medical records, making it possible to explore changes in complexity over time and how consultation complexity varies in different practices, areas, and populations. A complexity measure would also be useful for resource allocation formulae, planning staffing needs, and as a case-mix variable within models to predict patient outcomes or the use of hospital and other services.
The aim of this study was to develop a valid and reliable measure of the complexity of general practice consultations that can be applied to routine medical records.

METHOD
In this study, complex consultations are defined as those that are more difficult to conduct, challenging, multi-faceted, intensive, or time-consuming than average. This study was conducted in two stages. First, a Delphi study was conducted to agree characteristics of consultations that were perceived by GPs to be complex. Second, a valid and reliable measure was devised using cross-sectional data from a large sample of routine general practice consultations; the

Abstract Background
The complexity of general practice consultations may be increasing and varies in different settings. A measure of complexity is required to test these hypotheses.

Aim
To develop a valid measure of general practice consultation complexity applicable to routine medical records.

Design and setting
Delphi study to select potential indicators of complexity followed by a cross-sectional study in English general practices to develop and validate a complexity measure.

Method
The online Delphi study over two rounds identified potential indicators of consultation complexity. The cross-sectional study used an age-sex stratified random sample of patients and general practice face-to-face consultations from 2013/2014 in the Clinical Practice Research Datalink. The authors explored independent relationships between each indicator and consultation duration using mixed-effects regression models, and revalidated findings using data from 2017/2018. The proportion of complex consultations in different age-sex groups was assessed.

Results
A total of 32 GPs participated in the Delphi study. The Delphi panel endorsed 34 of 45 possible complexity indicators after two rounds. After excluding factors because of low prevalence or confounding, 17 indicators were retained in the cross-sectional study. The study used data from 173 130 patients and 725 616 face-to-face GP consultations. On defining complexity as the presence of any of these 17 factors, 308 370 consultations (42.5%) were found to be complex. Mean duration of complex consultations was 10.49 minutes, compared to 9.64 minutes for noncomplex consultations. The proportion of complex consultations was similar in males and females but increased with age.

Conclusion
The present consultation complexity measure has face and construct validity. It may be useful for research, management and policy, and for informing decisions about the range of resources needed in different practices.
Keywords cross-sectional studies; Delphi technique; general practice; office visits; risk adjustment. measure was re-validated in a separate dataset of consultations from a different year.

Delphi study
The research team created a list of variables that might increase the complexity of consultations based on previous literature, [5][6][7][8][9][10][11][12][13][14][15] their own experience, and informal discussion with general practice colleagues. Only characteristics that were likely to be coded in routine medical records were included. Demographical factors, such as age, sex, or deprivation, were not included since the intention was to explore how the final complexity measure varied according to these characteristics.
Development and piloting of the Delphi study identified two conceptual issues. First, it was found necessary to distinguish between consultation complexity factors and patient complexity factors. Complex consultation factors were defined as problems addressed within the consultation that made it complex. However, some patients have enduring characteristics that are likely to make most of their consultations complex irrespective of the problems presented -these were defined as complex patient factors. Second, it was found that some practitioners felt that almost all their consultations were complex. Therefore, when designing the Delphi questionnaire, clinicians were asked whether each characteristic made a consultation 'more complex than average'.
Colleagues from eight English universities were asked to each recruit five clinically active GPs to participate in the Delphi study. These doctors were asked to complete an online questionnaire in two rounds. In the first round, they were presented with 14 consultation factors and 19 patient factors and asked to indicate whether or not each factor made consultations more complex than average on a five point scale from 1 (no more complex than average) to 5 (very much more complex than average). Responders to the first-round questionnaire could add comments about individual factors or suggest additional factors that had not been included.
Factors that received strong endorsement by the panel in the first round were accepted as markers of complexity. Scores of 3 to 5 (moderate to extreme complexity) were considered to indicate endorsement of a characteristic; and a score of 1 (no more complex than the average patient) to indicate rejection. Factors that >70% of participants endorsed and <20% rejected were accepted as markers of complexity. Factors that <40% of participants endorsed and >20% rejected were not accepted. All other factors were designated uncertain and were taken forward to a second round of voting. In some cases, the wording of items was revised before the second round to provide greater clarity in the light of responders' comments.
In the second round, participants were sent an individualised report that showed how their responses for each characteristic, and overall, compared with the median and interquartile range from other members of the panel. The report included a summary of comments from participants in round 1 about factors that had been designated uncertain. In round 2 participants were invited to vote again on the uncertain factors and on new factors that had been proposed by participants. Factors were accepted or rejected using the same criteria as for round 1. Factors that remained uncertain were rejected.

Creating and validating a complexity measure
Read code sets were created for each of the patient and consultation complexity factors endorsed following the Delphi study. One of the authors with extensive experience of coding general practice consultations created an initial code set for each characteristic. These code sets were checked independently by two other authors (academic GPs), with discrepancies resolved by discussion or involving another author (also an academic GP). The final code sets are available at https://doi.org/10.5287/bodleian:8gq7zbb8w.
The prevalence of each characteristic was assessed based on an age-sex stratified sample from the Clinical Practice Research Datalink (CPRD) Gold database

How this fits in
Increasing general practice workload owing to rising consultation rates may be compounded by increasing complexity of consultations. Exploring these effects requires a valid and reliable measure of consultation complexity but there are no well-accepted measures. The authors have developed a suitable measure, starting with factors that GPs believe increase complexity and demonstrating those that are associated with longer consultations. The complexity measure presented in this study may be useful for research, management, and policy, for example in allocating resources.
of non-temporary patients in England who were registered for any amount of time between 1 April 2013 and 31 March 2014 and had at least one face-to-face surgery consultation with a GP. Any characteristics that applied to <0.05% of consultations or patients were excluded from further consideration to simplify the measure. Frequency data were used to specify factors that had been described qualitatively in the Delphi process. For example, 'frequent attender' was re-specified as patients with ≥14 GP consultations in the previous year, based on the 95th centile for number of consultations.
To assess construct validity, the authors explored the independent relationship between each complexity factor and consultation duration using mixed-effects regression models of mean general practice consultation duration on patient and consultation complexity factors, with random effects for patient and practice. Consultation and patient factors were considered in separate models. Factors with a prevalence <0.05% or those that appeared to reduce the length of consultations were removed from the initial models. Remaining factors were removed in a backwards stepwise fashion using P<0.05 as the threshold. For a given consultation, the consultation factor applied if the topic was coded within the consultation and the patient factor applied if the consultation was with a patient with this factor.
A complex consultation was defined as one in which ≥1 complexity factors were present. The mean duration of complex and non-complex consultations was compared, and the proportion of complex consultations by age-group was described.
The described analyses of construct validity were repeated as further re-validation in a separate dataset of patients from the CPRD comprising 58 528 patients who consulted at least once between 1 April 2017 and 31 March 2018.

Delphi study
Of 41 GPs sent details of the study, 32 agreed to participate and completed the first round of the survey. Participants included 10 (31%) males and 22 (69%) females with a mean of 11.2 (median 6; range <1 to 29) years' experience in general practice. The potential complexity factors in the first-round survey included 14 consultation factors and 19 patient factors. After the first round of the Delphi process, seven consultation factors were endorsed and none were rejected, while nine patient factors were endorsed and five were rejected. Seven consultation and five patient factors were uncertain and   Tables S2 and S3 show the prevalence of consultation complexity factors and patient complexity factors respectively, along with the final wording used to define each factor.
Factors coded in <0.05% of consultations or patients were omitted. This excluded two consultation factors: consultations about 'medically unexplained symptoms' and those 'resulting in urgent secondary care assessment' and two patient factors: 'paraplegia', and 'medically unexplained symptoms within last year'. Four further factors were excluded as consultation factors but included in the modelling as patient factors: 'palliative care', 'homelessness', 'domestic violence', and 'safeguarding'.
The results of the initial mixed-effects regression models of consultation and patient factors against consultation duration for 2013/2014 are shown in Table 3, with equivalent data for 2017/2018 in Supplementary Table S4. The final models, omitting variables with no significant

Factors suggested by participants and included in round 2
Patient is housebound or a nursing home patient -- relationship with consultation duration, include 17 factors (Table 4). The re-validation analysis used data from consultations between 1 April 2017 and 31 March 2018 and included 234 447 consultations with 58 528 independent patients. In the final model, five factors were no longer significantly associated with consultation duration (Table 4). However, the authors decided to retain these factors in their complexity measure because the factors had all been endorsed by GPs in the Delphi study, and the coefficients for duration were all positive with confidence intervals that overlapped in the development and validation data sets (see Table 3 and  Supplementary Table S4 The proportion of complex consultations was strongly associated with increasing age, and was slightly higher in males than in females, except in patients aged >85 years ( Figure 1).

DISCUSSION Summary
In this article the authors have defined, created, and applied a measure of the complexity of general practice consultations which can be applied to routine electronic medical records. This measure was constructed using characteristics of patients and problems selected by a consensus process involving experienced GPs, demonstrating face validity. The measure has been validated by showing that each of the characteristics in the final selection, and the overall complexity measure, are associated with consultation duration in two independent samples of consultations.

Strengths and limitations
This study has several limitations. The concept of complexity in consultations is nebulous, and though widely recognised by clinicians, it is hard to define. 13,16 The present definition of complexity encompasses intellectual, emotional, and workload demands, but other definitions of complexity would lead to different measurement tools. The choice of complexity factors was based on the experience of the research team and the literature, with additional factors suggested by the GPs in the Delphi panel, but other factors could have been considered. Some factors may add complexity to consultations but are not coded within electronic medical records. In this study some factors were dropped, such as medically unexplained symptoms, which almost certainly generate complexity within consultations but are  rarely coded, so inclusion would add little to the measure when used for analysis at a population level. Two variables (patients with dementia or patients who are housebound) had a statistically significant negative association with consultation duration. In post-hoc analysis it was found that these characteristics were associated with more consulting time over a whole year, resulting from a higher number of consultations that are shorter than average. The development of the complexity measure was conducted in England, and factors that cause consultation complexity may differ in other countries, for example insurance status in the US. 13,14 The complexity measure developed here was based on a sample of consultations taken 6 years ago. This was deliberate to create a baseline against which to assess changes in complexity over time in a subsequent article. However, in this study the authors have revalidated the findings in a more recent dataset (2017/2018) and this analysis largely confirmed the present findings.
The authors recognise that mean duration of consultations is not a gold standard for complexity, since the length of a consultation is only partly related to complexity and not all complex consultations are lengthy. However, it was the best and simplest (while imperfect) proxy available within routine medical records. The purpose of the cross-sectional analysis was not to derive a model to predict consultation duration, but to provide evidence for the construct validity of the present complexity measure by showing a positive association with a variable (duration) that the authors hypothesised would be related to it. The analysis fulfilled the present aims by confirming: that each of the included complexity factors was independently associated with longer consultations; that a measure defined as the presence of ≥1 of these factors was discriminating, with complex consultations being on average 9% longer than non-complex consultations; and that these findings were robust when repeated in a different data sample. Though the complexity measure is useful as a binary 'complex/non-complex' variable, the authors do not propose combining the factors to create a cumulative score (see statistical note in Supplementary Box 1).
The present measure is reliable in that it is based on objective analysis of medical records and defined code sets for complexity factors, unlike measures that require subjective judgements. 4,7,13,17 Basing the measure on the views of practising GPs and assessing the relationship with consultation duration provides evidence of face and construct validity respectively.
Further validation exercises could explore the relationship between the present complexity measure and other variables, such as practitioners' self-assessment of the complexity of a sample of consultations. Future research should also explore the relationship between complexity and risk prediction models for healthcare utilisation. The authors anticipate some, but not complete, overlap. 14 It is likely that different tools will be best at predicting different outcomes and measures should be used in combination to understand population healthcare needs. 18

Ethical approval
The protocol for this study was approved by the Independent Scientific Advisory Committee (ISAC) of the Medicines and Healthcare products Regulatory Agency (MHRA) (ISAC protocol number: 15_120R; available from the authors on request). Ethical approval for observational research using the Clinical Practice Research Datalink (CPRD) with approval from ISAC was granted by a National Research Ethics Service committee (Trent MultiResearch Ethics Committee, REC reference number: 05/MRE04/87).

Provenance
Freely submitted; externally peer reviewed.