METHODS

Study participants

Forty two family physician members of the American Academy of Family Physicians' (AAFP) National Network for Family Practice and Primary Care Research (the National Network) volunteered to participate in the study. They were evenly distributed throughout the US (10 in the north east, 12 in the south, nine in the mid-west, and 11 in the west) and were similar to all 64 714 non-federal US family physicians in mean age (study physicians 46 years, all family physicians 48 years), sex (75.7% v 75.9% male), and mean years in medical practice (16 years v 17 years). All participants were board certified in family practice compared with 86.3% of active AAFP family physician members. Most (73.0%) did not use electronic medical records in their practice and most (83.8%) did not have access to the Internet.

Study design

Data were collected as part of a crossover randomized controlled trial of paper and computer reporting systems in which physicians made anonymous reports by both methods. Paper reports were made using a traditional card based data collection system^16,20 and computer reports were made using an error reporting tool designed for the study by the World Health Network (http://www.healix.com). Paper reports were sent to the National Network office in Kansas where they were stripped of identifiers and assigned a numeric code to signify the physician making the report. Electronic data were sent encrypted from physicians' practices in the US through the secure World Health Network server in London before also being sent to the Kansas office (see box 1 for further technical details). Reports were then sent to the Robert Graham Center: Policy Studies in Family Practice and Primary Care (the Robert Graham Center) in Washington, DC. The process resulted in a set of anonymous reports that could be associated with a unique participating physician.

The Georgetown University Institutional Review Board approved the study. During data collection, all error reports were reviewed in the Robert Graham Center by a family physician (LG) who was not involved in data analysis, and a process was created to contact participating physicians to resolve situations where patients potentially remained at risk of harm. No such situation arose.

Study data

The study was planned to produce sufficient qualitative data to give depth and credibility to the characterization of errors in a preliminary taxonomy. After discussion with National Network physicians, we concluded: (1) that 500 error reports would provide sufficient data to derive a valid characterization of the errors recognized and reported in family practice; (2) that recruitment of approximately 50 physicians would give sufficient diversity of practice locations and settings to ensure that the study data were not unduly influenced by a few prolific reporters; and (3) that 10 error reports per doctor (five collected on paper forms and five on computer) was a manageable burden for participants and a reasonable number to expect in a 20 week data collection period (from 9 May to 26 September 2000). Physicians reported the first five errors they observed in their practices after the starting date. On completion of their first five reports, they made a further five reports using the alternative method of data transfer.

The data consisted of observations of family physicians during their daily clinical practice, whether in ambulatory care clinics, hospitals, patients' homes, nursing homes, or other sites of care delivery.

Definition of “error”

The definition of medical error used in this study was that adopted by the US Institute of Medicine⁶—that is, “safety is defined as freedom from accidental injury” and “error is defined as the failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim”. We made this definition functional for participating family physicians by explaining:

“For this study, please report anything that happened in your own practice that should not have happened, that was not anticipated and that makes you say `that should not happen in my practice, and I don't want it to happen again'. It can be small or large, administrative or clinical—anything that you identify as something to be avoided in the future.”

To be eligible for inclusion in the study, reported events did not require an adverse outcome of care or actual or potential harm to patients.

Collected data included information about the error and, if appropriate, affected patients. A free text description of the error was made with prompts to record what happened and observed consequences. The date, time, and site at which the error occurred and the date, time, and site at which it was recognized by the physician were also collected. If the error related to a particular patient, the report included the patient's age, sex, ethnicity, consequence to the patient and to others, and whether the reporting doctor usually provided the patient's care.

Analysis of data

The analytical plan rested on three key decisions:

1. To “bracket” prior knowledge of other characterizations of medical errors and to conduct a qualitative analysis that allowed the taxonomy to emerge from the study data. The rationale for this decision was that the study base of family physicians' self-reports was different from the data sources used to form other taxonomies, so using them might create traps of preconception that would limit the emergence of new knowledge from the current study.

2. To be guided in the analysis primarily by the words used in the report and to systematically challenge use of the analysts' clinical experience in interpreting these words. Sometimes this clinical experience was essential to understanding the reported event, but clinically informed “reading between the lines” was judiciously applied, debated, and documented in the process of creating the taxonomy. As far as possible, the resulting taxonomy therefore reflects the reporters' descriptions of events they recognized as “errors”, rather than analysts' interpretations of “root causes” of reported events.

3. To assign a single error classification code to each error report. Where it was clear that a number of different errors actually happened in a single reported event, we established a rule to assign the classification code that reflected only the first error in the chronology of the event. Thus we compiled a list of 330 errors from 330 reports.

One researcher (SD) initially made an immersion/crystallization analysis²¹ of free text responses to develop a draft error taxonomy. Using this draft taxonomy but blinded to the error category applied to the report, two family physician researchers (DM and RP) then reviewed every report independently and assigned a category to each. Following independent review, all three reviewers debated the appropriateness of each report's assignment to specific categories and established a revised definition of categories, re-framing and reconstructing the taxonomy through consensus. Several iterations of this process resulted in the taxonomy presented in fig 1. Fourteen reports (4.1% of total) were not used in creating the taxonomy as they were reports of adverse events that the three reviewers agreed did not arise from any error.

As part of a taxonomy useful for research purposes but separate from the description of error types, we compiled a list of contextual factors including the difference between the date, time, and place at which the error was recognized and the date, time, and place at which the doctor reported the error occurring, and patient demographic characteristics. We also reviewed the reports for descriptions of actual harms or costs accruing to patients, doctors, staff, or the health system generally.

RESULTS

Categorization of errors

Qualitative analysis of the 330 error reports produced the four-layer taxonomy shown in fig 1. The taxonomy comprises two primary categories which distinguish between errors attributable to aspects of care delivery systems (“Process” errors; n=284 (86.1% of total errors)) and errors that could only be averted by improving providers' clinical skills and/or knowledge or diverting clinical tasks to clinically trained providers (“Knowledge and skills” errors; n=46 (13.9% of total)). Included in the “Knowledge and skills” category were events such as receptionists failing to make urgent appointments available for acutely ill infants as well as physicians deciding to discharge patients from hospital before they were able to function well at home. “Process” errors included administrative mistakes (102; 30.9%), failure in investigation processes (82; 24.8%), treatment delivery problems (76; 23.0%), miscommunication (19; 5.8%), and payment system mix ups (4; 1.2%). One “Process” error could not be assigned to any subcategory as the reporting physician provided insufficient details. “Knowledge and skills” errors included errors in the execution of a clinical task (19; 5.8%), misdiagnosis (13; 3.9%), and wrong treatment decisions (14; 4.2%). The eight layer II categories were further subdivided into layers III (13 categories) and IV (21 categories), providing further detail.

Context

Analysis of contextual factors showed that physicians recognized reported errors on the day of the event in only 38 cases (11.6%), but 49.8% of errors were recognized within 2 weeks of their occurrence. The greatest reported time lapse between error occurrence and recognition was 141 days. Physicians' offices were the main sites of error occurrence (72.3%) and recognition (72.1%), but many other sites were implicated: 22 reports involved an error event recognized in hospitals and there were 13 reports of errors recognized in emergency departments, 10 reports of errors recognized in nursing homes, and 15 reports of errors recognized in patients' homes, pharmacies, laboratories, and the spaces between these sites (including physicians' cars). Only two error reports (failure of a refrigerator to maintain a temperature that could preserve its contents and inadequately cleaned equipment) were not linked to one unique affected patient.

The demographic characteristics of affected patients are shown in table 1. In 74.5% of reports the event involved a patient whom the reporting physician already knew.

Consequences

Physicians reported a consequence to patients (96 reports; 29.2%) or to someone else, including patients' families, practice staff, and themselves (66 reports; 20.2%) in response to separate prompts. Free text responses were reviewed to construct the consequence categories shown in table 2. Each case may be associated with more than one consequence, but in 184 reports (55.8%) no consequence was recorded or implied. Physicians reported emotional distress or loss of trust by the patient in 25 reports (7.6%). Ten errors precipitated hospital admission; one death resulted.

DISCUSSION

This analysis organizes error reports made by family physicians into a meaningful but preliminary taxonomy. The importance of the taxonomy is that it provides a tool for understanding opportunities to improve patient care and suggests priority areas for remedial attention. The foundation of this tool in primary care is important because of the current dearth of patient safety research in this setting. By virtue of volume alone,^16,22 primary care delivery systems probably contain substantial opportunities for better protection of patients from harm. This study shows that practising family physicians report a spectrum of medical errors that is different from the types of medical errors previously identified in hospital based care.⁴ Most errors reported arose from problems with healthcare processes. Medication errors were reported frequently but they were not such a dominant area of concern as one might infer from the concentration of the medical error literature on this aspect of care.^5,10,11 Both investigations and administrative dysfunction contributed more often to reported errors; errors in clinical knowledge or skills were also reported.

The taxonomy is preliminary but, with further use, its categories will become better defined. The anthropological research approach we took to this analysis is best satisfied when cell “saturation” is achieved.²³ The small number of reports assigned to each category shown in fig 1 suggests that saturation has not been achieved in this study. Given the breadth of scope of the practice of family physicians, it may take some time and repeated use of this tool to develop it to the stage where it truly describes all or most medical errors in primary care.

The definition of error we used in this study deliberately related to true errors or mistakes, without regard to actual or potential consequences. It therefore differs from much of the existing literature on patient safety which tends to focus on “preventable adverse events” (actual harm)^7,13 or “sentinel events” (potential harm).^5,18 Our definition risks attracting reports of trivial events, thereby distracting from the central purpose of patient safety research—namely, to find ways to protect patients from harm. Many reports (55.8%) may seem trivial in that no patient was harmed; however, an event that is trivial on one occasion may be non-trivial on another. The data highlight the important role of administrative systems, particularly medical record systems, as a source of errors that matter to patients and doctors. A single death was the most serious consequence reported in this study and was traced to a failure of message handling. The risk of death was a possibility in other cases—for instance, a physician was unable to contact a patient whose skin biopsy result indicated melanoma because there was no contact information for the patient. Both these examples defeat the notion that message handling and registration processes are unimportant in taking care of patients and securing their safety. Although most undelivered messages or missing data in patient records do not result in harm, these are such common and redeemable system failings that they deserve attention. This attention will save lives.

The definition of “error” used in this study was chosen because of its comprehensibility by participants and on the grounds that so little previous patient safety research has focused on primary care settings that there was a need to determine the type of errors seen by family physicians. Family physicians, as healthcare coordinators for patients, are well positioned to observe health system problems in a wide variety of settings. In this study many errors were observed in sites other than the physician's office and many involved other healthcare professions. By including the full scope of family practice, the study identified errors in both inpatient and outpatient settings and in the transition between care settings. The breadth of the scope of family practice was reflected in the wide variety of error types reported, the spectrum of patient characteristics represented in the data set, and the range of consequences observed. Other healthcare providers, practice staff, and patients are likely to perceive medical errors differently and the research agenda needs to be extended to include their views.

The analytical approach used in this study required key decisions that might be differently addressed by others. There were tensions in our decisions. For instance, our first decision was to eschew other taxonomies and to develop a characterization of primary care errors directly from the data, and our second was to be guided by the words in the report as we analysed each one. On some occasions the reporter showed awareness of existing taxonomies by stating “this is a medication error” or “this is a laboratory error”. When this happened we remained true to our decision to follow the report's words and accepted the influence of other similar work on reporters' language. Our third decision—to assign each report to only one category—was taken to simplify the analysis. We recognize the attraction of coding all perceived errors described in reports but, if this approach is taken, concurrent consideration of chronology may be needed to make sense of the event.

We did not meet the stated goal of 50 participants. Although 54 volunteered for the study, 12 (22.2%) did not send in any reports. We attribute this failure to difficulty with managing unfamiliar computer systems, but equally it might be because those doctors did not observe any errors or they feared to contribute because of concern about litigation. If we had collected the intended 500 reports rather than the actual 344 (330 errors and 14 non-error adverse events), the taxonomy might contain more categories, especially at finer categorization layers. In this early stage of understanding the nature of primary care medical errors, this preliminary taxonomy must be regarded as a work in progress.

CONCLUSION

There are important differences between care in hospitals (where most effort has been focused to improve patient safety) and primary care where most people receive care for most of their health problems. It is therefore necessary to develop a relevant taxonomy of errors as a framework for understanding threats to patient safety in primary care and as a basis for proposing remedies. This study provides a preliminary tool that may be used and developed by further primary care research. It also confirms that errors reported by practising family physicians are not the same as those recorded from hospital based datasets and that they are important to patients.