Overview

There were three stages to the method:

• Stage 1: Creating a subset of The Health Improvement Network (THIN) database made up of people with: a diabetes disease code, prescribed therapy used in diabetes, or random or fasting blood glucose results compatible with a diagnosis of diabetes. Those with a definite other type of diabetes (e.g. genetic and steroid induced) were excluded. The codes for diabetes were classified into those that could be directly or partially mapped, or had no clear mapping to a diagnosis of T1DM or T2DM.

• Stage 2: Applying both our original EOMR compatible algorithm and the modified POMR algorithm to this subset. Our principle analysis looked at the number and proportion of people who could be classified into definite, probable and possible T1DM or T2DM cases (Figure 1), or who could not be so classified.

• Stage 3: Comparing the characteristics of patients who could be dual-classified by both algorithms (i.e. EOMR and POMR), only one algorithm (EOMR-only or POMR-only), or neither of the algorithms.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1: Venn diagram illustrating the groups (A) Dual identified as having diabetes by both algorithms, (B) by just the POMR (new), (C) EOMR (old) or (D) Unclassifiable by either algorithm; within the subset of people with possible diabetes

Stage 1: Creating a subset of the THIN database with possible diabetes and code mapping.

The THIN database comprised practices that use InPS Vision, which is one of the most strictly problem orientated CMR systems available in the UK. This POMR system insists that all data entered are linked to a problem.

By way of contrast, EOMRs allow the user to file notes without a problem being entered. They also allow the user to enter multiple near synonyms for the same problem. For example, in an EOMR a patient with diabetes might have definite diagnostic codes combined with more vague codes. (e.g. the Read code C10F: T2DM is a ‘definite code’ for T2DM; whereas C104z: diabetes mellitus with nephropathy NOS is a code stating a complication, and does not allow the type of diabetes to be determined.)

The THIN database comprised approximately 2.5 million records (N = 2,466,364) with an age–sex distribution similar to the English National population (Office for National Statistics 2007 estimate shown in Figure 2). The distribution of men and women was 1,225,008 and 1,241,356, respectively. The age distribution within the THIN cohort followed a similar distribution of the UK national population of 2007. Practices throughout the UK belong to THIN and are broadly representative of the UK population in terms of patients’ demographics and other characteristics.13,14 We performed our analyses on data collected up to 2009. The subset we created included any diagnostic code for diabetes (Read code C10%) or coded records of therapy for diabetes and blood glucose measures that met the WHO diagnostic diagnosis of diabetes.15 Additionally for all patients, we extracted age, gender, latest random or fasting blood glucose and recording of body mass index (BMI). We excluded people with a definite other type of diabetes (n = 393). These 393 excluded individuals comprised 253 people with rare forms of diabetes; steroid induced (n = 123) and autosomal dominant (n = 64) were the commonest. One hundred and forty people had C10 codes for metabolic disorders that are not diabetes. The crude prevalence of T1DM and T2DM diabetes was 4.08% (100,513/2,466,364), based on any code that implied diabetes.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2 Profile of the THIN population compared with the English National Population.

We mapped all the diabetes codes in the subset. Codes were categorised into those which could be definitely or partially mapped to T1DM or T2DM, and those with no clear mapping. Additionally there was a fourth category of codes which could not be mapped to either T1DM or T2DM. For example, the Read codes C10E, C10F and nearly all their child codes directly map onto codes for T1DM and T2DM, respectively. Some codes partially map. For example, the Read code C1001: maturity onset diabetes probably represents T2DM. Other codes have no clear mapping, and it is impossible to tell what type of diabetes, though most of these are likely to represent possible T2DM.16 Our mapping was based on the work completed as part of the NHS Diabetes⁄Royal College of General Practitioners (RCGP) practical classification of diabetes.17,18

Comparing estimated prevalence of T1DM and T2DM using the new POMR – and the original EOMR – algorithms

We compared the prevalence of T1DM, T2DM and unclassified diabetes as estimated using the POMR and EOMR algorithms (Table 1). The POMR algorithm identified around 16% fewer cases of T1DM (7,750/2,466,364 for an estimated prevalence of 0.31%) compared with the EOMR algorithm (9,264 cases, prevalence 0.38%). In contrast, the POMR algorithm identified 13% more cases of T2DM (89,990 cases and prevalence 3.65%) than the EOMR estimate (79,417 cases and prevalence 3.22%). A much smaller proportion (2.4%) of diabetes cases was unclassifiable by the POMR algorithm than by the EOMR algorithm (11.4%). All these differences were highly significant (p < 0.0001).

Comparing the age, BMI and blood glucose levels between dual classified, POMRonly, EOMR-only and unclassifiable patient groups

We compared the age, BMI, and blood glucose (RBG and FBG) for the cases classified by either of the algorithms (Figure 1). The dual classified (group A) and POMR only (group B) patients were very similar in terms of mean (66 years) and median (68 years) age, while the EOMR-only (group C) were on average 7 years older (mean 73 years and median 75 years).

Mean BMI was highest in the dual-classified group (30.8 kg/m2) and the POMR-only group (30.3 kg/m2). Both these groups had significantly higher mean BMI than the EOMRonly group (mean 29.4 kg/m2), which had the lowest mean BMI of all groups (Tables 2 and 3).

RBG was highest in the POMR-only (11.1 mmol/L) and dualclassified groups (10.4 mmol/L), and in both these were significantly higher than the EOMR-only group (6.9 mmol/L) and the unclassifiable group (7.6 mmol/L). FBG levels showed a very similar pattern, being highest in the POMR-only and dual classified groups, and significantly lower in the EOMR-only and unclassifiable groups.

In summary, the higher BMI and blood glucose levels in the cases classified by both and the new POMR algorithm appear to be more compatible with diabetes than the lower BMI and much lower levels of glucose recorded in people not classified by either of the algorithms.

Comparing the therapy and blood glucose testing between dual classified, POMRonly, EOMR-only and unclassifiable patient groups

Almost a quarter (24.5%) of the dual classified group and over a third (34.6%) of the POMR-only group were prescribed insulin (Table 4). By contrast, only 4.8% of the unclassifiable cases and none of the EOMR-only group were prescribed insulin.

As expected, the majority of prescribing overall was for OAD mainly used in T2DM. Some form of OAD was prescribed to 79.9% of the dual-classified and 91% of the POMR-only groups. Only 12.2% of the unclassifiable group were prescribed OADs, and none of the EOMR-only groups were receiving any medical therapy for diabetes.

Summary of Results

Based on their age, BMI and blood glucose profiles, along with their prescribing history, the dual-classified group looks quite typical of a mixed population of T1DM and T2DM patients. The additional group of cases classified by the POMR algorithm alone are very similar to the dual classified group, and their characteristics are also highly consistent with them having a mixture of T1DM and T2DM. In contrast, the group classifiable only using the EOMR algorithm was somewhat atypical: one the one hand they appear to be the best controlled in terms of blood glucose level, while on the other hand, they are receiving no prescriptions for anti-diabetic therapy. On this basis, it is tempting to speculate that many patients in this group either do not have diabetes, or maybe borderline cases. Finally, based on blood glucose, many of the people not classifiable by either of the algorithms appear not to have diabetes, yet approximately 13% of this group are receiving medical treatment for this condition.

Principal findings

Compared with the original EOMR algorithm, the ’new‘ POMR algorithm identified significantly fewer patients with T1DM (7,750 versus 9,264, p < 0.0001, Table 1), and significantly more patients with T2DM (89,990 versus 79,417, p < 0.0001). There are a large number of cases where the type of diabetes is uncertain from the code used. The new POMR algorithm left 2.4% of diabetes diagnosis labelled ’uncertain‘ and the EOMR 11.4%; when run against data from a strictly problemorientated system.

The characteristics of the cases classified by both algorithms and the additional cases just identified by the new POMR algorithm are strongly suggestive of these individuals having diabetes. The cases just identified by the EOMR algorithm may be less likely to have diabetes. However, there are potentially cases not identified by either of the algorithms.

Implications of the findings

Our results suggest that the algorithms which underpin such tools may need to be tailored to the specific CMR system on which they are run. Specifically, we found that the algorithm developed for a more EOMR system performed less well when applied to more a strictly POMR database, than an algorithm that was specifically adopted to that system.

Improving the quality of diabetes related coding has clear benefits in terms of patient management and safety. For example, patients with a non-specific diabetes diagnosis are more likely to have poor glycaemic control.20

The quality of routinely collected electronic information and their fitness for purpose is determined by more than just the GIGO – garbage in garbage out – principle; there needs to be a wider adoption of ontological principles that confirm the compatibility of diagnostic codes with the demographics of the patient, other diagnoses, therapy and investigations.21

Different types of clinical computer system may require different approaches. Algorithms that have performed well in one system may not do so on another. It is also plausible that strict problem orientation, as found in this system, resulted in improved data quality compared with less problem orientated systems. Researchers using primary care databases need to take into account both the limitations of coding within a single CMR system as well as systematic differences in coding between databases derived from different CMR systems.

Comparison with the literature

This study confirms the findings of previous research which identified different types of errors and discrepancies in the recording of information about diabetes.5–7,16,18,19 Practitioners using CMR will increasingly need access to tools, which identify and allow for correction of diabetes related coding errors.

Systematic reviews have found that, despite methodological shortcomings, inconsistent definitions and considerable heterogeneity in interventions, patient populations, processes and outcomes of care,22 integrated care programs can improve the quality of patient care.10 Good quality data collected as part of routine clinical care is required to address this evidence gap cost effectively. Routinely collected electronic health care data, aggregated into large clinical data warehouses, are increasingly being mined, linked and used for audit, continuous quality improvement in clinical care, health service planning, epidemiological study and evaluation research. Managing the increasing amount of routinely collected data is therefore a priority.

The benefit of a POMR system is it encourages users to commit to a previously entered condition as opposed to creating a separate stand-alone entry. This ‘joined up’ coding system has the added advantage of reduced variability in coding for any given patient. By doing so, the chance of vague coding reference from multiple attendances is avoided. This system design assists, therefore, in maintaining data quality. This is the first set of data to critically compare how different data extract approaches might be needed for POMR compared with EOMR.

We have stopped short of saying that POMR are superior to EOMR, though they may be for chronic diseases such as diabetes. This is because we know from multichannel video observation of the clinical consultation that users of more problem orientated systems appeared to code more, and to be more likely to code consistently chronic diseases such as diabetes.11,12 However, the downside of this approach is that the small number of doctors we directly observed using these systems would link minor problems to make common threads. An example of this was a doctor who widely used the code: Had a chat to patient (Read code 8CR), which was of less use in understanding the nature of the consultation than a more problem related new episode title.

Limitations of the methods

We have no gold standard for those who truly have diabetes in the population. This study like much of our other work in this area seeks to infer meaning from routine data.

The limitations of routine data are well described; they are not only rich but also inconsistent. The data quality is poor in about 5% of records in health organisations.23–25 Many studies regularly report a range of deficiencies in the routinely collected electronic information for clinical6,26 or health promotion23 purposes in hospital27 and general practice28 settings.

Larger studies simultaneously comparing data from more than one CMR system are required to elucidate if there are other more subtle differences between the algorithms.

Call for further research

We need to develop more formalised ways of developing and sharing algorithms and ontologies to sort complex datasets.29,30 The aggregation of increasingly large datasets raises issues of semantic interoperability and a need for automated methods to assess and manage data quality.30

Finally, the pros and cons of different levels of problem orientation should be an important part of the design and development of requirements analysis for CMR systems.