Measuring the complexity of general practice consultations: development and validation of a complexity measure. British of General Practice.

Background: The complexity of general practice consultations may be increasing and vary in different settings. Testing these hypotheses requires a measure of complexity. Aim: To develop a valid measure of general practice consultation complexity applicable to routine medical records. Design: Delphi study to select potential indicators of complexity followed by cross-sectional study to develop and validate a complexity measure. Setting: English general practices. Method: An online Delphi study over two rounds involved 32 general practitioners to identify potential indicators of consultation complexity. The cross-sectional study used an age-sex stratified random sample of 173,130 patients and 725,616 general practice face-to-face consultations from 2013/14 in the Clinical Practice Research Datalink. We explored independent relationships between each indicator and consultation duration using mixed effects regression models, and revalidated findings using data from 2017/18. We assessed the proportion of complex consultations in different age-sex groups. Results: After two rounds, the Delphi panel endorsed 34 of 45 possible complexity indicators. In the cross-sectional study, after excluding factors because of low prevalence or confounding, 17 indicators were retained. Defining complexity as the presence of any of these factors, 308,370 consultations (42.5%) were complex. Mean duration of complex consultations was 10.49 minutes, compared to 9.64 minutes for non-complex consultations. The proportion of complex consultations was similar in men and women but increased with age. Conclusion: Our consultation complexity measure has face and construct validity. It may be useful for research, management and policy, informing decisions about the range of resources needed in different practices. problems discussed within consultations. 16 Three studies have asked general practitioners about features that make patients complex, and we build on this by considering aspects of consultations as well as patients. 12,14,15,17 A few previous authors have devised case-mix measures applicable to primary care, but these have either not taken account of clinicians’ perceptions of the complexity of different factors 18-21 or not been designed for analysis of routine medical records. 13 22 There is some overlap 23 24 resource These measures are based information, other psychological factors 11 within consultations 12,15,17,26 and captured our complexity measure.


Introduction
In the United Kingdom, general practitioners report increasing pressure from rising workload. 1,2 The number of consultations increased by 14% between 2007 and 2014, and the mean duration of face-toface consultations increased by 7%. 3 Doctors' perceptions of increasing workload may reflect an increase in the complexity as well as number of consultations. This may be associated with an ageing population, increasing prevalence of multimorbidity and polypharmacy, transfer of activities from secondary to primary care, increasingly complex clinical guidelines, and growing policy expectations of what can be achieved within each consultation. The increasing delegation of routine tasks to other staff is also likely to increase the proportion of general practice consultations which are complex and intellectually demanding. 1,4 To test this hypothesis, it is necessary to define and measure complexity within general practice consultations. A measure suitable for research and analysis needs to be applicable to routine electronic medical records, making it possible to explore changes in complexity over time and how consultation complexity varies in different practices, areas, and populations. A complexity measure would also be useful for resource allocation formulae, planning staffing needs, and as a case-mix variable within models to predict patient outcomes or the use of hospital and other services.
The aim of this study was to develop a valid and reliable measure of the complexity of general practice consultations that can be applied to routine medical records.

Methods
We defined complex consultations as those which are more difficult to conduct, challenging, multifaceted, intensive, or time-consuming than average. This study was conducted in two stages. First, we conducted a Delphi study to agree characteristics of consultations which were perceived by general practitioners to be complex. Second, we devised a valid and reliable measure using cross-sectional data from a large sample of routine general practice consultations, and re-validated the measure in a separate dataset of consultations from a different year.

Delphi study
The research team created a long-list of variables that might increase the complexity of consultations based on previous literature, [5][6][7][8][9][10][11][12][13][14][15] their own experience, and informal discussion with general practice colleagues. We only included characteristics that were likely to be coded in routine medical records. We A c c e p t e d M a n u s c r i p t -B J G P -B J G P . 2 0 2 0 . 0 4 8 6 5 did not include demographic factors such as age, sex or deprivation since the intention was to explore how the final complexity measure varied according to these characteristics. Development and piloting of the Delphi study identified two conceptual issues. First, we found it necessary to distinguish between consultation complexity factors and patient complexity factors. We defined complex consultation factors as problems addressed within the consultation which made it complex. However, some patients have enduring characteristics which are likely to make most of their consultations complex irrespective of the problems presented -these were defined as complex patient factors. Second, we found that some practitioners felt that almost all their consultations were complex. Therefore, when designing the Delphi questionnaire, we asked clinicians whether each characteristic made a consultation 'more complex than average'.
We asked colleagues from eight English universities to each recruit five clinically active general practitioners to participate in the Delphi study. These doctors were asked to complete an online questionnaire in two rounds. In the first round, they were presented with 14 consultation factors and 19 patient factors and asked to indicate whether or not each factor made consultations more complex than average, on a five point scale from 1 (no more complex than average) to 5 (very much more complex than average). Respondents to the first-round questionnaire could add comments about individual factors or suggest additional factors that we had not included.
Factors which received strong endorsement by the panel in the first round were accepted as markers of complexity. We considered scores of 3 to 5 (moderate to extreme complexity) to indicate endorsement of a characteristic and a score of 1 (no more complex than the average patient) to indicate rejection.
Factors which more than 70% of participants endorsed, and fewer than 20% rejected, were accepted as markers of complexity. Factors which fewer than 40% of participants endorsed and more than 20% rejected were not accepted. All other factors were designated uncertain and were taken forward to a second round of voting. In some cases, the wording of items was revised before the second round to provide greater clarity in the light of respondents' comments.
In the second round, participants were sent an individualised report which showed how their responses for each characteristic and overall compared with the median and inter-quartile range from other members of the panel. The report included a summary of comments from participants in round 1 about factors which had been designated uncertain. In round 2, participants were invited to vote again on the 6 uncertain factors and on new factors which had been proposed by participants. Factors were accepted or rejected using the same criteria as for round 1. Factors that remained uncertain were rejected.

Creating and validating a complexity measure
We created Read code sets for each of the patient and consultation complexity factors endorsed following the Delphi study. A researcher with extensive experience of coding general practice consultations (AF) created an initial code set for each characteristic. These code sets were checked independently by two academic general practitioners (BN and CS), with discrepancies resolved by discussion or involving a third academic general practitioner (RH). The final code sets are available at https://doi.org/10.5287/bodleian:8gq7zbb8w.
We assessed the prevalence of each characteristic based on an age-sex stratified sample from the Clinical Practice Research Datalink (CPRD) Gold database of non-temporary patients in England who were registered for any amount of time between 1/4/2013 and 31/3/2014 and had at least one face-toface surgery consultation with a GP. We excluded from further consideration any characteristic which applied to less than 0.05% of consultations or patients, to simplify the measure. We used frequency data to specify factors which had been described qualitatively in the Delphi process. For example, 'frequent attender' was re-specified as patients with ≥14 GP consultations in the previous year, based on the 95 th centile for number of consultations.
To assess construct validity, we explored the independent relationship between each complexity factor and consultation duration using mixed effects regression models of mean general practice consultation duration on patient and consultation complexity factors, with random effects for patient and practice.
Consultation and patient factors were considered in separate models. Factors with a prevalence lower than 0.05% or which appeared to reduce the length of consultations were removed from the initial models. Remaining factors were removed in a backwards stepwise fashion using p<0.05 as the threshold. For a given consultation, the consultation factor applied if the topic was coded within the consultation and the patient factor applied if the consultation was with a patient with this factor.
We defined a complex consultation as one in which one or more complexity factor was present. We compared the mean duration of complex and non-complex consultations, and described the proportion of complex consultations by age-group.
A c c e p t e d M a n u s c r i p t -B J G P -B J G P . 2 0 2 0 . 0 4 8 6 7 As a further re-validation, we repeated the above analyses of construct validity in a separate dataset of patients from the CPRD comprising 67,829 patients who consulted at least once in 2017/18.

Delphi study
Of 41 general practitioners sent details of the study, 32 agreed to participate and completed the first Four further factors were excluded as consultation factors but included in the modelling as patient factors (palliative care, homelessness, domestic violence, safeguarding).
The results of the initial mixed effects regression models of consultation and patient factors against consultation duration for 2013/14 are shown in Table 3, with equivalent data for 2017/18 in 8 Supplementary Table 4. The final models, omitting variables with no significant relationship with consultation duration, include 17 factors (Table 4).
The re-validation analysis used data on consultations between 1/4/2017 and 31/3/2018 and included 234,447 consultations with 58,528 independent patients. In the final model, five factors were no longer significantly associated with consultation duration (Table 4). However, we decided to retain these factors in our complexity measure because they had all been endorsed by GPs in the Delphi study, and the coefficients for duration were all positive with confidence intervals that overlapped in the development and validation data sets. The proportion of complex consultations was strongly associated with increasing age, and was slightly higher in men than in women except in patients aged over 85 years (Figure 1).

Summary
In this paper we have defined, created and applied a measure of the complexity of general practice consultations which can be applied to routine electronic medical records. This measure was constructed using characteristics of patients and problems selected by a consensus process involving experienced general practitioners, demonstrating face validity. The measure has been validated by showing that each of the characteristics in the final selection, and the overall complexity measure, are associated with consultation duration in two independent samples of consultations.

Comparison with existing literature
Our research builds on previous research. Two studies 4,7 and an online survey 2 have asked primary care clinicians to record the complexity of their consultations subjectively, for example using a five-point scale from very simple to very complex, while another study quantified the number and range of 9 problems discussed within consultations. 16 Three studies have asked general practitioners about features that make patients complex, and we build on this by considering aspects of consultations as well as patients. 12,14,15,17 A few previous authors have devised case-mix measures applicable to primary care, but these have either not taken account of clinicians' perceptions of the complexity of different factors [18][19][20][21] or not been designed for analysis of routine medical records. 13 22 There is some overlap between measures of complexity and case-mix measures such as Adjusted Clinical Groups, 23 Rx-Risk 24 and the Charlson score 25 which have been designed to predict health outcomes, resource utilisation or mortality. These case-mix measures are based on combinations of diagnostic information, medication data and/or demographic factors but do not account for social, behavioural or other psychological factors 11 which often create the greatest demands on general practitioners within consultations 12,15,17,26 and are captured by our complexity measure.

Strengths and limitations
Our study has several limitations. The concept of complexity in consultations is nebulous, and although widely recognised by clinicians it is hard to define. 13,26 Our definition of complexity encompasses intellectual, emotional and workload demands, but other definitions of complexity would lead to different measurement tools. The choice of complexity factors was based on the experience of the research team and the literature, with additional factors suggested by the general practitioners in the Delphi panel, but other factors could have been considered. Some factors may add complexity to consultations but are not coded within electronic medical records. We dropped some factors (such as medically unexplained symptoms) which almost certainly generate complexity within consultations but are rarely coded, so inclusion would add little to the measure when used for analysis at a population level. Two variables (patients with dementia or housebound) had a significant negative association with consultation duration. In post-hoc analysis we found that these characteristics were associated with more consulting time over a whole year, resulting from a higher number of consultations which are shorter than average. The development of the complexity measure was conducted in England, and factors which cause consultation complexity may be different in other countries (for example insurance status, in the United States). 13,14 The complexity measure we developed was based on a sample of consultations which is six years old. This was deliberate, to create a baseline against which to assess changes in complexity over 10 time in a subsequent paper. However, we revalidated the findings in a more recent dataset (2017/18) and this analysis largely confirmed our findings.
We recognise that mean duration of consultations is not a gold standard for complexity, since the length of a consultation is only partly related to complexity and not all complex consultations are lengthy.
However, it was the best and simplest (while imperfect) proxy available within routine medical records.
The purpose of the cross-sectional analysis was not to derive a model to predict consultation duration, but to provide evidence for the construct validity of our complexity measure by showing a positive association with a variable (duration) that we hypothesised would be related to it. The analysis fulfilled our aims by confirming (a) that each of the included complexity factors was independently associated with longer consultations, (b) that a measure defined as the presence of one or more of these factors was discriminating, with complex consultations being on average 9% longer than non-complex consultations and (c) that these findings were robust when repeated in a different data sample.
Although our complexity measure is useful as a binary 'complex/non-complex' variable, we do not propose combining the factors to create a cumulative score (see statistical note in appendix).
Our measure is reliable in that it is based on objective analysis of medical records and defined code sets for complexity factors, unlike measures which require subjective judgements. 4,7,13,22 Basing the measure on the views of practicing GPs and assessing the relationship with consultation duration provides evidence of face and construct validity respectively.
Further validation exercises could explore the relationship between our complexity measure and other variables, such as practitioners' self-assessment of the complexity of a sample of consultations. Future research should also explore the relationship between complexity and risk prediction models for health care utilisation. We anticipate some, but not complete, overlap. 14 It is likely that different tools will be best at predicting different outcomes and measures should be used in combination to understand population healthcare needs. 27

Implications for research and practice
This paper describes a valid and reliable measure of the complexity of general practitioners' consultations. In future research we plan to explore the complexity of consultations in different settings and populations, and how complexity has changed over time. This may be relevant to the development of resource allocation formulae. The current UK formula for allocating payments to primary care takes account of the number of expected consultations based on characteristics of the practice population, 11 but not the complexity of those consultations. 28 Practices which have a high proportion of complex consultations may need a different mix of staff from practices with few complex consultations. There is growing interest in creating population health management systems by linking health and social care datasets to understand current and future health and care needs. 29 Use of a complexity measure may support this aim, providing greater nuance and understanding by taking account of the different workforce, workload and resource implications of consultations with different levels of complexity.

Funding
This research was supported by the NIHR School for Primary Care Research grant number 243 and the Nuffield Department of Primary Care Health Sciences, University of Oxford.

Ethical approval
The protocol was approved by the Independent Scientific Advisory Committee (ISAC) of the MHRA (ISAC protocol number 15_120R; available from the authors on request). Ethics approval for observational research using the CPRD with approval from ISAC was granted by a National Research Ethics Service committee (Trent MultiResearch Ethics Committee, REC reference number 05/MRE04/87).