You are here

Article Text

Article menu
Narrative review
Team-training in healthcare: a narrative synthesis of the literature
  1. Sallie J Weaver1,
  2. Sydney M Dy2,
  3. Michael A Rosen1
  1. 1Department of Anesthesiology and Critical Care Medicine, Johns Hopkins University School of Medicine, Armstrong Institute for Patient Safety & Quality, Baltimore, Maryland, USA
  2. 2Department of Health Policy & Management, Oncology, and Medicine, Johns Hopkins University, Baltimore, Maryland, USA
  1. Correspondence to Sallie J Weaver, Department of Anesthesiology and Critical Care Medicine, Johns Hopkins University School of Medicine, Armstrong Institute for Patient Safety & Quality, 750 E. Pratt St., 15th Floor, #1544, Baltimore, MD 21202, USA; sjweaver{at}jhu.edu

Abstract

Background Patients are safer and receive higher quality care when providers work as a highly effective team. Investment in optimising healthcare teamwork has swelled in the last 10 years. Consequently, evidence regarding the effectiveness for these interventions has also grown rapidly. We provide an updated review concerning the current state of team-training science and practice in acute care settings.

Methods A PubMed search for review articles examining team-training interventions in acute care settings published between 2000 and 2012 was conducted. Following identification of relevant reviews with searches terminating in 2008 and 2010, PubMed and PSNet were searched for additional primary studies published in 2011 and 2012. Primary outcomes included patient outcomes and quality indices. Secondary outcomes included teamwork behaviours, knowledge and attitudes.

Results Both simulation and classroom-based team-training interventions can improve teamwork processes (eg, communication, coordination and cooperation), and implementation has been associated with improvements in patient safety outcomes. Thirteen studies published between 2011 and 2012 reported statistically significant changes in teamwork behaviours, processes or emergent states and 10 reported significant improvement in clinical care processes or patient outcomes, including mortality and morbidity. Effects were reported across a range of clinical contexts. Larger effect sizes were reported for bundled team-training interventions that included tools and organisational changes to support sustainment and transfer of teamwork competencies into daily practice.

Conclusions Overall, moderate-to-high-quality evidence suggests team-training can positively impact healthcare team processes and patient outcomes. Additionally, toolkits are available to support intervention development and implementation. Evidence suggests bundled team-training interventions and implementation strategies that embed effective teamwork as a foundation for other improvement efforts may offer greatest impact on patient outcomes.

  • Team training
  • Teamwork
  • Patient safety

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/

https://doi.org/10.1136/bmjqs-2013-001848

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    …It has become necessary to develop medicine as a cooperative science; the clinician, the specialist, and the laboratory workers uniting for the good of the patient, each assisting in elucidation of the problem at hand, and each dependent upon the other for support. –William J. Mayo, Commencement speech at Rush Medical College, 1910

    Deficiencies in communication and teamwork have long been cited as a frequent contributor to adverse events. Precise estimates of the extent of the problem are difficult to make, given definitional, reporting and measurement inconsistencies. However, a variety of studies support the notion that teamwork and communication are critical components of safe healthcare systems. Previous reviews report linkages between various aspects of teamwork (eg, situational monitoring, communication, leadership, trust, shared mental models) and clinical performance.1–3 Meta-analytic results suggest the relationship between team processes and clinical performance indicators has generally been characterised by medium to large effect sizes.4 ,5 For example, studies in surgery have shown increased odds of complications and death (OR 4.82; 95% CI 1.30 to 17.87) when surgical teams exhibit less frequent teamwork behaviours (eg, less information sharing during intraoperative and handoff phases, and less briefing).6 Reviews of malpractice claims further underscore that communication problems are major contributing factors in 24% of cases.7 Other studies found teamwork and communication issues cited as root causes in 52–70% of adverse events.8 ,9 Additionally, teamwork and communication dimensions of safety culture have been significantly related to adverse clinical events.10 ,11

    The 2001 Making Health Care Safer report12 provided one of the early reviews concerning the topic of healthcare team-training in a chapter entitled ‘Crew Resource Management and Its Application in Medicine’. This review discussed early conceptualisations of team-training in other high-reliability industries such as aviation and summarised early studies attempting to translate team-training principles developed elsewhere into healthcare settings. The development and implementation of team-training programmes in acute care settings has grown dramatically in the last decade with improvements in content, methods and evaluation desgins.13–15 Advances in training content, implementation and evaluation have increasingly drawn on over 30 years of evidence examining team performance processes and team-training across a wide variety of high-risk environments.16 While previous reviews described the state of team-training in healthcare through 2009,13 ,14 the prevalence of peer-reviewed literature evaluating team-training interventions continues to grow. Publication trends suggest that in the last 5 years, over 448 papers examining teamwork topics in a variety of languages, across a variety of care settings, have appeared in PubMed (see figure 1). Additionally, team-training was formally identified in 2013 as a top patient safety strategy encouraged for adoption in acute care settings.17 Therefore, our objective is to provide an updated narrative synthesis of the body of evidence evaluating team-training in acute care settings, including implementation trends, evidence for effectiveness and potential boundary conditions (ie, factors that may enhance or inhibit effectiveness). In providing this updated review, we hope to also highlight fruitful areas for future research and innovative practice.

    Figure 1
    Figure 1

    PubMed publication trends from 2000 through October 2013 for team-training and related concepts.

    Types of team-training strategies

    Team-training is defined as a constellation of content (ie, the specific knowledge, skills and attitudes that underlie targeted teamwork competencies), tools (ie, team task analysis, performance measures) and delivery methods (ie, information, demonstration and practice-based learning methods) that together form an instructional strategy.18 In this sense, team-training is a systematic methodology for optimising the communication, coordination and collaboration of healthcare teams that combines specific content with opportunities for practice, formative feedback and tools to support transfer of training to the daily care environment.

    As described by the National Quality Forum19: Health care organizations must establish a proactive, systematic, organization-wide approach to developing team-based care through teamwork training, skill building, and team-led performance improvement interventions that reduce preventable harm to patients…training programs should systematically address and apply the principles of effective team leadership, team formation [and team processes].

    Borrowing from other high-reliability communities, team-training in healthcare originated largely in the form of Crew Resource Management (CRM), a specific team-training strategy focused on developing a subset of teamwork competencies including hazard identification, assertive communication and collective management of available resources.20–23 However, the practice of team-training is much broader. Today, team-training is an overarching term that encompasses a broad range of learning and development strategies, methods and teamwork competencies. Table 1 outlines several types of team-training strategies.

    View this table:
    Table 1

    Team-training strategies

    The critical element defining team-training is that the learning activity focuses on developing, refining and reinforcing knowledge, skills or attitudes that underlie effective teamwork behaviours such as communication, coordination and collaboration. Prior reviews found that the most commonly targeted teamwork competencies include communication, situational awareness, leadership, role clarity and coordination.14 ,24–27 To this end team-training activities are often designed to develop generalisable, transportable teamwork competencies that learners can apply across different settings and teams. This differentiates team-training from learning activities focused on technical clinical skills (eg, differential diagnosis and procedural skills), as well as team building exercises focused on developing emergent states such as trust or cohesion among members of an intact team.

    Methods

    A PubMed search for key review articles examining team-training interventions in acute care settings published between January 2000 and December 2012 was conducted. Key search terms included ‘team-training’, ‘teamwork training’, ‘teamwork training interventions’, ‘crew resource management’, ‘TeamSTEPPS’ and related terms. Following identification of relevant reviews with searches terminating in 2008,13 200912 ,21 and 2010,24 ,25 a search of PubMed and PSNet was completed to identify additional primary studies published between 1 January 2011 and 31 December 2012. Studies were excluded if they were only descriptive in nature, if conducted in non-English-speaking populations or if primarily targeting students or trainees. Narrative summaries of individual studies and shared themes are presented. This review was supported by Agency for Healthcare Research and Quality (AHRQ), which had no role in the selection or review of the evidence or the decision to submit this manuscript for publication.

    Results

    Several previous reviews have examined the effectiveness and implementation of healthcare team-training.2 ,13–15 23–33 For example, one systematic review of interventions to improve team effectiveness in healthcare found that the majority involved some form of team-training (42 of 48 reviewed studies).13 Other reviews investigated the effectiveness of team-training for obstetric emergencies,33 for enhancing communication in surgery and anaesthesia,20 ,25 ,31 classroom-based interventions27 and simulation-based interventions.15 ,24 ,30 Additionally, several reviews investigated the content, design and delivery of team-training and the impact of team processes on clinical processes and outcomes.2 ,14 ,24 ,26 ,29 ,31 ,32 ,34 We draw on results from these reviews and 26 articles published since their searches terminated in order to describe the current body of evidence regarding the effectiveness of team-training in acute care settings, the contexts in which it has been evaluated and effects of different design and delivery methods. Findings from 26 studies published after previous reviews concluded their searches are summarised in table 2 and discussed in greater detail in subsequent sections (see online supplementary appendix for detailed evidence summary).

    View this table:
    Table 2

    Studies examining team-training effects on clinical process or patient outcomes

    What is the context for team-training?

    Previous reviews highlight that team-training has been implemented across a broad range of acute care contexts.2 ,13 ,14 ,27 ,30 This includes academic hospitals35 and community-based hospitals,36 ,37 as well as medical centres affiliated with the Veterans Administration (VA) and the Military Health System.38–40 Though our focus is on current practitioners,41–44 team-training programmes have focused on a variety of audiences including students and clinical trainees.45 ,46 The majority of studies have focused on evaluating team-training efforts among frontline clinicians working in obstetric or perioperative care areas; however, studies have also been conducted in emergency and critical care, procedural areas (eg, endoscopy) and administration. For example, table 2 demonstrates that 9 studies of the 26 published between 2011 and 2012 were conducted in surgical work areas, 5 were conducted in critical care or inpatient nursing units and 4 were conducted in obstetrical or emergency/trauma settings, respectively. Four studies were conducted in other areas, including procedural areas or administration, or were conducted across multiple types of care areas.

    Team-training programmes for hospital leaders have also demonstrated positive effects. Individual executive leadership development programmes fall outside the scope of traditional team-training initiatives; however, team-training programmes for executive leaders is an area ripe for further research as many healthcare organisations move towards enterprise affiliations and Accountable Care Organization (ACO) models. In this way administrators, service line leaders, division leaders and frontline providers increasingly manage and work not only in teams, but as part of teams-of-teams, or multiteam systems (MTS).47 ,48 MTSs are defined as two or more teams that work interdependently towards at least one common goal.49 Initial work has begun to examine the leadership competencies that matter most in these contexts and to test interventions to cultivate them.50 ,51 However, further development and evaluation of team-training interventions targeting MTSs within the healthcare context are needed.

    Team type

    Teamwork in healthcare is often characterised by highly dynamic team membership, participation on multiple teams and rapid team formation. As opposed to intact teams who have a history and future working together, some types of healthcare teams, particularly in acute care settings, may not have ever worked together before, form under stressful conditions (eg, a code) and may not have great likelihood of working together closely in the future. Such teams are known as ad hoc teams or action teams and are defined by rapid formation, an abbreviated lifespan and often limited experience working together previously. Given this context, can team-training be effective for such ad hoc teams? Previous reviews indicate that healthcare team-training has been conducted with intact (ie, teams who work together currently or on a regular basis) and ad hoc teams (ie, teams formed quickly for brief periods of performance or for training purposes only). For example, Weaver14 found eight studies that reported training intact teams and five studies that reported training in ad hoc teams. One meta-analysis that examined team-training effectiveness across a variety of industries including healthcare found similar effect sizes for teams who worked together on a regular basis (intact teams ρ=0.48) and teams who did not (ad hoc teams ρ=0.44).4 A second meta-analysis that also examined the effects of team-training on overall team performance across a range of settings found that team-training had a greater positive impact on the performance of ad hoc teams (dIntact=0.62; dAdhoc=0.92); however, the difference was not statistically significant.52 Within healthcare, several of the most robust studies have used training strategies allowing teams to train together (eg, closing the operating room for 1 day so that team members could attend training together); however, this is not a boundary condition for training effectiveness.

    What have we learned about team-training design and delivery?

    Several reviews of team-training and team processes in healthcare have also examined training design and delivery.13 ,14 ,24 ,27 ,31 These reviews find variation among programmes in the teamwork competencies targeted, curriculum used, how much time learners spend in training, how often clinicians and staff are participating, and other details regarding content, delivery strategies and evaluation efforts.

    Training content

    As described earlier, many different types of training strategies fall under the umbrella of team-training. Previous reviews found communication, situational awareness, leadership and situation monitoring as the most common teamwork competencies targeted.14 ,27 Table 2 emphasises that similar competencies have also been the focus of more recent primary studies, particularly as the adoption of teamwork training curriculums built around these competencies (eg, TeamSTEPPS, CRM, VA Medical Team Training (VA MTT)) has grown. For example, 9 of the 26 studies reviewed used some form of CRM, 7 reported using components of the TeamSTEPPS curriculum and 3 studies used VA MTT. Seven studies reported using other team-training curriculums.

    More narrowly defined programmes targeting deep development of specific teamwork skills (eg, leadership) in the healthcare context have also been evaluated. Two53 ,54 of the twenty-six studies reviewed specifically evaluated an intervention designed to develop teamwork competencies among hospital leaders and administrators. Such programmes may differ from those focused on frontline providers in the level of emphasis on certain teamwork competencies (eg, transformational leadership, conflict management and mediation, broader internal and external environment awareness, boundary spanning activities that facilitate coordination with other internal or external teams, etc., may be of greater emphasis),55 ,56 as well as design (eg, greater use of table-top simulations or game-based approaches).57

    Training modality and methods

    The evidence to date demonstrates that healthcare team-training programmes vary in the instructional modalities and methods used. Instructional methods can be conceptualised in terms of three broad categories: (1) information-based methods (eg, didactic lecture), (2) demonstration-based methods (eg, behavioural modelling, videos) and (3) practice-based methods (eg, simulation, role playing). Previous reviews found that the majority (83%) of team-training programmes integrated both information and practice-based methods, while only 35% reported incorporated demonstration-based activities.14

    Simulation-based versus classroom-based training. Simulation is a broad training modality that may include physical simulation of clinical care environments, standardised patients, cognitive simulations and role play. It includes both centre-based learning activities and in situ activities that occur in the actual environment where care is provided.

    Simulation is a common modality for team-training in healthcare. Previous reviews found 68% of programmes using simulation-based activities.14 Several reviews have examined simulation-based team-training exclusively.15 ,24 ,30 In terms of outcomes, previous reviews examining simulation-based team-training15 ,24 ,30 and classroom-based team-training27 separately reported similar positive effects on learner reactions, knowledge or skills, and clinical practices. In terms of patient outcomes, both simulation studies and classroom-based studies have shown a positive impact on patient outcomes. Though simulation is powerful learning activity, particularly for teamwork competencies, some of the most robust studies published since these previous reviews were completed did not use elaborate simulation methods, but still achieved significant reductions in both mortality and morbidity.38 ,58 Direct comparison studies also provide insight. Riley36 compared perinatal work areas in three different hospitals; one participated in a didactic team-training programme, one participated in an in situ simulation programme and one served as a control. Results indicated that significantly greater reductions in patient harm were achieved by the group that participated in the in situ simulation programme compared with both the didactic and control groups (37% reduction vs 1% reduction vs 43% increase in harm, respectively).

    Training duration. Programme duration is an area in which team-training programmes in healthcare vary. A review of 18 studies evaluating classroom-based team-training interventions found course duration varied from 4 h to 3 days with several studies describing longer train-the-trainer programmes.27 Another review found that 53% of 40 reviewed team-training programmes were designed to last less than 1 day.14

    Intervention design and implementation strategy

    Overall, no comprehensive meta-analysis to date has directly examined variations in design or implementation strategy as boundary conditions influencing the effectiveness of healthcare team-training programmes. However, current evidence supports the notion that the most effective team-training programmes leverage a bundled intervention design that pairs learning activities and practice, with tools designed to support effective teamwork during daily clinical care (eg, checklists, reminders, peer coaches). This bundled design aligns with a systems-based approach to training that emphasises a comprehensive strategy for developing and sustaining expert team performance.32 These strategies view training as one piece of the puzzle that will support effective team performance under stressful conditions over time. For example, Buljac-Samardzic's13 descriptive systematic review of strategies to optimise team effectiveness found that the majority involved some form of team-training; however, other interventions also included tools to support team performance during care (eg, checklists, goal lists; 8 studies), as well as broader organisational interventions (eg, redesign of care processes or team structures, 8 studies).

    Implementation strategy. In terms of implementation strategy, both train-the-trainer and direct train-the-staff strategies have been used. For example, a train-the-trainer model formed the foundation for the National Implementation of TeamSTEPPS Project,59 a collaborative effort by the U.S. Department of Defense (DoD), the AHRQ and the American Institute for Research (AIR) designed to create a national training and support infrastructure for healthcare entities implementing team-training. Through a national network of team resource centres, individuals interested in leading the implementation of team-training within their organisation could become TeamSTEPPS Master Trainers by participating in an intensive 3-day training session. Master Trainers then return to train administrators and frontline personnel within their own organisation using the customisable TeamSTEPPS curriculum. A slightly different approach was used in the large-scale implementation of team-training throughout the VA. As part of the VA National Center for Patient Safety Medical Team-training (MTT) Programme, learning sessions for participating VA medical centres were facilitated directly by an interdisciplinary team of dedicated MTT faculty.38 Both strategies, however, include local facility change teams, implementation of on-the-job tools (eg, process checklists, scripts) to support training transfer, as well as measurement and evaluation processes as integral implementation components.

    Transfer of training. Overall, systems-based models of team-training share an implementation approach that strategically considers what happens before, during and after training in order to facilitate transfer of effective teamwork into daily care processes. These models are rooted in classic transfer of training models, which stipulate that transfer, generalisation of teamwork skills developed in training and maintenance of these skills over time is the product of the work environment (ie, support for using teamwork skills, opportunities to use these skills in practice), training design and individual trainee characteristics.60 This suggests that the most effective team-training efforts include readiness assessment and preparation, learning activities that bring together multidisciplinary practitioners, deployment of performance support tools (eg, checklists or structured communication tools) and post-training support through coaching, reinforcement and recognition for improved teamwork, and ongoing measurement paired with constructive feedback concerning teamwork processes.61 For example, the VA MTT programme was led by a national coordinating entity (the VA National Center for Patient Safety) who worked with local implementation teams through 2 months of preplanning and readiness preparation prior to implementation.38 Transfer of training was also supported by structured communication tools and quarterly follow-up coaching. The TeamSTEPPS programme similarly offers readiness assessment and evaluation planning, and facilitator training as part of the TeamSTEPPS Master Trainer curriculum, as well as coaching and deployment support.

    Organisational policies and procedures have also been changed to facilitate transfer of trained teamwork skills. For example, one study of team-training in surgery found significant improvements in on-time first case starts and operating room turnover times following training.42 To sustain these improvements, the organisation implemented policy changes and incentives around these two processes. The combination of team-training and these organisational changes resulted in continued improvement in both processes over the 4 years following training implementation. Specifically, on-time starts increased 42% across the total 5-year period reported (69–98%) and there was a 38% decrease in OR turnover time (mean2006=43 min to mean2010=27 min). Such examples reinforce the notion that effective team-training initiatives go beyond training alone to proactively address aspects of the organisational work environment that support team members in sustaining effective teamwork practices over time, as well as generalisation of skills to new teams and new situations.62

    What have we learned about team-training effectiveness?

    Team-training has demonstrated the capacity to improve both how teams do their work and resulting outcomes across a wide variety of industries. For example, one meta-analysis of team-training that included 93 effect sizes across a broad range of industries found that participation in team-training can account for nearly 20% of the variance in team processes (ρ=0.44) and outcomes (ρ=0.39).4

    Several reviews also support the notion that team-training can meaningfully improve participant knowledge or attitudes, teamwork processes, clinical care processes and even patient outcomes.5 ,13 ,14 ,27 ,31 Though local customisation of team-training curriculum is common and evaluation metrics are heterogeneous, we attempt to summarise findings in the remainder of this section in terms of participant reactions, changes in knowledge, attitudes or skill, teamwork processes or behaviours in clinical environments, and clinical processes or patient outcomes. This organising structure is based upon Kirkpatrick's framework for training evaluation that organises training outcomes across the four levels described below.63

    Level 1: learner reactions to training

    Previous reviews demonstrate that learner reactions are one of the more common, yet least robust, evaluation metrics reported. One review found that 60% of 40 reviewed articles reported learner reactions to training (eg, perceptions of training utility, applicability and general satisfaction with training).14 The sample of 26 studies summarised in table 2 demonstrates a shift towards more comprehensive evaluation of team-training programmes, however. Ten studies explicitly reported trainee reactions; while 7 studies examined trainee knowledge, attitudes or self-efficacy; 16 studies evaluated team behaviours, processes or emergent states (eg, teamwork climate, safety climate, trust); and 13 studies evaluated clinical processes or patient outcomes (see online supplementary appendix).

    Overall, learner reactions to team-training have been shown to be positive, often with 80% or more of training participants, indicating that the topics covered during training were applicable to their work setting (see online supplementary appendix). Reactions can be important factors in shaping learner motivation to use effective teamwork strategies on the job and should be considered for that reason; however, they are not highly correlated with other training outcomes64 and are not generally considered strong indicators of training effectiveness. Increasingly evaluation efforts have moved away from reporting participant reactions as primary outcomes, focusing instead on evaluating effects on provider behaviours and patient outcomes.

    Level 2: learner knowledge, self-efficacy and attitudes

    This second level of evaluation examines the impact of training on learner knowledge, self-efficacy and attitudes. Previous reviews indicate that such outcomes were reported in 50%13 to 78%27 of studies. Self-efficacy or confidence in one's ability to engage in effective teamwork behaviours during future clinical events was the most commonly reported outcome at this level among the studies reviewed in table 2. Self-efficacy is an important antecedent that shapes intentions to use effective teamwork skills during clinical practice and has been related to increased use of effective teamwork behaviours.65 Overall, previous reviews and the additional seven primary articles that evaluated these outcomes provide evidence that team-training can positively impact these types of outcomes (see online supplementary appendix). Of the seven articles, five reported significant improvements in teamwork attitudes, self-efficacy or knowledge following training and the majority of these used CRM or TeamSTEPPS training curriculums. Several studies have also investigated the duration of these effects over time or differential effects among various types of clinicians. For example, Figueroa65 found that increases in leadership and clinical skills confidence achieved immediately post-training were sustained at a 3-month post-training evaluation. Similarly Patterson66 found gains in teamwork knowledge sustained at follow-up conducted 8–10 months post-training.

    Level 3: teamwork behaviours, processes and emergent states

    Changes in teamwork behaviours and processes are arguably the primary criteria by which the effectiveness of team-training programmes in healthcare has been evaluated. Previous reviews found that 44%28 to 63%14 of studies reported evaluating the impact of team-training on team behaviour or performance. Sixteen of the twenty-six studies (62%) reviewed (see online supplementary appendix) evaluated the impact of team-training at this level, of which 12 reported significant improvements (ie, statistically significant improvement on one or more process measures or at least one dimension of safety culture/climate, or statistically significant differences between participating teams and controls). The methods used to assess team performance range from team self-ratings to blinded observational ratings using validated measurement tools. A number of observational tools and behavioural marker systems have been developed to evaluate teamwork in both simulated and live practice environments (eg, the Oxford Non-Technical Skills (NOTECHS) scale,67 the Anesthetists’ Non-Technical Skills (ANTS) system68), which have greatly improved the strength and validity of findings in studies that have used them. For example, one high-quality study that used a modified version of the NOTECHS adapted for the trauma care setting found significant improvements in observed teamwork processes both during simulated cases and live cases following an in situ simulation-based team-training programme that included both a brief web-based didactic learning session and multidisciplinary clinical simulations.

    In addition to team behaviours, evaluations of team-training have also often included measures of emergent, affective aspects of team performance such as teamwork climate or safety climate. Teamwork climate and safety climate reflect shared perceptions regarding communication openness, mutual support and other aspects of team functioning. Previous reviews also indicate that such measures may be used as part of pretraining needs analysis.27They are often measured with a safety culture or climate survey; however, reporting of results is highly heterogeneous. Some studies report composite scores that aggregate several survey items into one domain or dimension (eg, Carney,69 Phipps70) while others report item-level results (eg, Stevens71). Overall, there is some moderate evidence that team-training may positively impact teamwork climate or safety climate outcomes; however, conclusions are limited by variation in reporting.

    Level 4: clinical processes and patient outcomes

    More recently, studies demonstrate the impact of team-training programmes on clinical processes, as well as indices of care safety and quality. Previous reviews found that only a few studies had attempted to evaluate the impact of team-training on such outcomes.5 ,13 ,14 ,27 ,72 However, effect sizes have tended to fall in the medium to large range (eg, d=0.63,73 as cited in5), indicating that team-training has been related to meaningful improvements in clinical process and patient outcomes. Thirteen of the twenty-six studies summarised in table 2 evaluated the impact of team-training at this level, of which ten reported statistically significant improvement. Additionally, several robust studies offer high-quality evidence that comprehensive team-training programmes can improve both clinical processes and patient outcomes. For example, an evaluation of the VA MTT programme demonstrated significant and sustained decreases in preoperative delays (from 16% to 7% of cases, p=0.004), increased antibiotic prophylaxis compliance (from 85% to 97%, p<0.0001), decreases in equipment issues/case delays (from 24% to 7% of cases, p<0.0001), decreased handoff issues (from 5.4% to 0.3% of cases, p<0.0001) and most notably significantly greater reductions in both mortality (RR=1.49) and morbidity (RR=1.20) for facilities that implemented team-training.38 ,58 Furthermore, a dose–response relationship was established such that for each quarter the programme was in place at a facility, a decrease of 0.5 deaths per 1000 procedures (p=0.001) was observed. Implementation of TeamSTEPPS has been associated with increased efficiency in clinical processes for multidisciplinary trauma teams (eg, decreased times from arrival to surgery from 130.1 to 94.5 min (p<0.05), endotracheal intubation from 10.1 to 6.6 min (n.s.) and CT scan from 26.4 to 22.1 min (p<0.01),74 as well as an 83% reduction in medication and transfusion errors (p<0.001) and a 70% reduction in needlestick injuries and exposures (p<0.05) in a U.S. Combat Support Hospital deployed in Iraq.39 Other studies have also reported significant reductions in clinical decision time (p<0.05)40 associated with team-training, as well as one study showing a reduction in adverse clinical events and a 50% reduction in high-severity malpractice claims (pretraining 11 high-severity claims, post-training 5 high-severity claims, no statistics reported).75

    Strength of evidence

    Overall, the body of evidence concerning the effectiveness of team-training in healthcare spans the continuum of low-quality, medium-quality and high-quality studies. The majority of previous reviews did not systematically rate study quality. However, Buljac-Samardzic13 concluded that the majority of studies reviewed were of low-level to moderate-level quality even though eight of the reviewed team-training studies were categorised as high or moderate quality (ie, RCT or high-quality pre-post study). Similar conclusions were drawn by McCulloch.28 Significant homogeneity among studies, both in training content and strategy, as well as reported outcomes limited conclusions. In the review by Rabøl27 of classroom-based team-training interventions, 15 of the 18 reviewed studies were uncontrolled and 17 studies were rated at a moderate or high risk for bias.

    A number of high-quality primary studies have been published since these reviews were completed. These include the VA studies38 ,58 ,69 and several studies of the TeamSTEPPS curriculum.35 ,39 ,42 These studies used high-quality retrospective controlled designs or high-quality pre-post designs that examined the effects of training across multiple outcomes including teamwork processes and patient outcomes. Several very recent studies published after our search window ended have also employed strong quasi-experimental designs, larger sample sizes, robust analytical methods (eg, multilevel analyses) and different outcomes (eg, care coordination practices across teams, communication with patients and families) to further strengthen conclusions regarding team-training effectiveness.76–79

    Are the effects of team-training sustained over time?

    Sustainability of team-training effects over time is a common question among both improvement scientists and practitioners. Many early evaluations of team-training limited follow-up evaluations to 6 months post-training or less.27 ,28 Recent studies have included longer follow-up evaluation periods with multiple measurement points. Of the 26 papers summarised in table 2, 11 included follow-up evaluations examining at least one of the Kirkpatrick levels more than 6 months after the training activity. For example, Mayer and colleagues35 found significant improvements in observed team processes 1 month post-training that were sustained at 6 months and 12 months post-training in a sample of critical care providers. Findings regarding changes in survey measures of patient safety culture over time were mixed, however. In studies examining the implementation of the VA's team-training programme, results indicated both significant improvements in perceptions of teamwork climate 9–11 months following implementation69 and statistically significant reductions in surgical mortality 1 year post.38 ,58 Several studies also demonstrated that variability in aggregate indices of patient harm were reduced following implementation of a team-training programme delivered using a combination of both didactic learning activities and in situ simulation.36 ,70 Overall, there is evidence for improvements in teamwork knowledge, attitudes and team processes during shorter term evaluation windows (eg, less than 6 months) and increasingly more robust evidence for longer term impact on aggregate indices of real or potential patient harm. There are also indications of skill decay over time, however, with several studies demonstrating that all short-term improvements in knowledge, skills and practices were not maintained over longer time periods.66 ,80 While a great deal of work has been dedicated to examining optimal refresher training intervals for clinical procedural skills (eg, resuscitation or intubation skills81 ,82), relatively minimal work has been done to understand optimal refresher intervals for teamwork competencies.

    Discussion

    In summary, previous reviews of team-training in healthcare and recent studies have found that these programmes can improve teamwork processes and patient outcomes (eg, reductions in adverse events, reductions in mortality and morbidity). Current evidence also offers detailed insight into team-training design and implementation strategies relevant for researchers and practitioners alike.13 ,14 ,24 ,27–32 Additionally, over three decades of research examining team-training in a wide variety of contexts exists that both researchers and practitioners can draw from.

    In terms of the strength of evidence, previous systematic reviews included several studies that used RCT or controlled pre-post designs. Several robust studies examining team-training strategies have been published since previous reviews concluded. However, it is important to note that previous reviews and recent studies reflect a wide range in quality of evidence—with several studies limited by small sample sizes, weak study design and limited detail regarding the team-training curriculum or implementation strategy. Overall, our synthesis suggests that there is moderate to high-quality evidence that team-training can positively impact healthcare team processes and, in turn, clinical processes and patient outcomes. These effects have been demonstrated across a variety of acute care areas, including critical care, surgery and emergency care. A key finding is that the studies demonstrating the most robust evidence for effectiveness35 ,38 ,58 have implemented team-training as a bundled intervention that includes preplanning, readiness assessment, interdisciplinary learning activities and tools to support active transfer and sustained use of effective teamwork practices into daily care. Additionally, evaluation efforts undertaken in the most robust studies leveraged large care systems and strong quasi-experimental evaluation designs. This is not to say that every attempt to implement team-training should be compelled to follow similar evaluation efforts. These findings do underscore the value of implementing bundled interventions that match learning activities with activities to actively practice effective teamwork strategies and tools to support effective teamwork in daily practice. This underscores that team-training is most effective when paired with other methods to improve teamwork and also that team-training is one of several interventions that has proven effective at reducing patient harm and improving patient safety.62 ,83

    Additionally, our synthesis underscores that there is meaningful heterogeneity and local adaptation in both training design and implementation across studies. While the studies reviewed used similar general training strategies (eg, CRM), the details of specific competency areas targeted and delivery methods were highly heterogeneous. As noted in previous reviews13 ,14 ,28 and in the more recent work reviewed here, training programmes varied in duration, activities and modality.

    The importance of local customisation is also reflected in the toolkits available to support team-training implementation. For example, a customisable version of the TeamSTEPPS curriculum is available publically through AHRQ (http://teamstepps.ahrq.gov/). Additionally, the VA MTT programme is available to VA Medical Centers through the National Center for Patient Safety (http://www.patientsafety.gov/mtt/). Both programmes provide customisable content while emphasising local adaptation of learning materials and implementation strategies. Best practices for team-training development and evaluation61 ,84 can also provide guiderails for implementation and evaluation design.

    To continue building this evidence base, future work should continue to evaluate team-training efforts across time and outcomes. This includes continuing to examine the impact on patient safety outcomes using robust implementation designs, evaluating team-training effects in multiteam system contexts (eg, chronic disease management, comprehensive cancer care), examining the comparative effectiveness of different methods of training implementation and examining implementation methods to support sustainment of behaviour changes achieved through training. For example, there is currently limited evidence that provides insight into the frequency of retraining or dedicated practice needed to develop and maintain effective teamwork skills. Additionally, there is a need to examine how dynamic team composition (ie, changes in team membership) moderate team processes and the effects of team-training. Methodologically, robust validation studies can continue to strengthen the evidence regarding team-training effectiveness and the factors or conditions that influence effectiveness. Finally, longitudinal studies and studies addressing the integration of team-training concepts throughout the career development of healthcare professionals are needed to continue building our understanding of team-training in healthcare.

    References

    1. ↵
      1. Baker DP,
      2. Gustafson S,
      3. Beaubien J,
      4. et al
      . Medical teamwork and patient safety: the evidence-based relation. Rockville, MD: Agency for Healthcare Research and Quality, 2005. http://www.ahrq.gov/research/findings/final-reports/medteam/index.html
    2. ↵
      1. Manser T
      . Teamwork and patient safety in dynamic domains of healthcare: a review of the literature. Acta Anaesthesiol Scand 2009;53:143–51.
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Sorbero M,
      2. Farley D,
      3. Mattke S,
      4. et al
      . Outcome measures for effective teamwork in inpatient care: final report. Santa Monica, CA: RAND Corporation, 2008. http://www.rand.org/content/dam/rand/pubs/technical_reports/2008/RAND_TR462.pdf
    4. ↵
      1. Salas E,
      2. DiazGranados D,
      3. Klein C,
      4. et al
      . Does team training improve team performance? A meta-analysis. Hum Factors 2008;50:903–33.
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Schmutz J,
      2. Manser T
      . Do team processes really have an effect on clinical performance? A systematic literature review. Br J Anaesth 2013;110:529–44.
      OpenUrlAbstract/FREE Full Text
    6. ↵
      1. Mazzocco K,
      2. Petitti DB,
      3. Fong KT,
      4. et al
      . Surgical team behaviors and patient outcomes. Am J Surg 2009;197:678–85.
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Rogers SO,
      2. Gawande AA,
      3. Kwaan M,
      4. et al
      . Analysis of surgical errors in closed malpractice claims at 4 liability insurers. Surgery 2006;140:25–33.
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Rabøl LI,
      2. Andersen ML,
      3. Østergaard D,
      4. et al
      . Descriptions of verbal communication errors between staff. An analysis of 84 root cause analysis-reports from Danish hospitals. BMJ Qual Saf 2011;20:268–74.
      OpenUrlAbstract/FREE Full Text
    9. ↵
      1. Singh H,
      2. Thomas EJ,
      3. Petersen LA,
      4. et al
      . Medical errors involving trainees: a study of closed malpractice claims from 5 insurers. Arch Intern Med 2007;167:2030–6.
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      1. Mardon RE,
      2. Khanna K,
      3. Sorra J,
      4. et al
      . Exploring relationships between hospital patient safety culture and adverse events. J Patient Saf 2010;6:226–32.
      OpenUrlCrossRefPubMed
    11. ↵
      1. Singer S,
      2. Lin S,
      3. Falwell A,
      4. et al
      . Relationship of safety climate and safety performance in hospitals. Health Serv Res 2009;44:399–421.
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      1. Shojania KG,
      2. Duncan BW,
      3. McDonald KM,
      4. et al
      . Making Health Care Safer: Summary. 2001. In: AHRQ Evidence Report Summaries. Rockville (MD): Agency for Healthcare Research and Quality (US); 1998-2005. http://www.ncbi.nlm.nih.gov/books/NBK11945/
    13. ↵
      1. Buljac-Samardzic M,
      2. Dekker-van Doorn CM,
      3. van Wijngaarden JDH,
      4. et al
      . Interventions to improve team effectiveness: a systematic review. Health Policy 2010;94:183–95.
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      1. Weaver SJ,
      2. Lyons R,
      3. DiazGranados D,
      4. et al
      . The anatomy of health care team training and the state of practice: a critical review. Acad Med 2010;85:1746–60.
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      1. Cumin D,
      2. Boyd MJ,
      3. Webster CS,
      4. et al
      . A Systematic review of simulation for multidisciplinary team training in operating rooms. Simul Healthc 2013;8:171–9.
      OpenUrlCrossRefPubMed
    16. ↵
      1. Salas E,
      2. Cooke NJ,
      3. Gorman JC
      . The science of team performance: progress and the need for more…. Hum Factors 2010;52:344–6.
      OpenUrlFREE Full Text
    17. ↵
      1. Shekelle PG,
      2. Pronovost PJ,
      3. Wachter RM,
      4. et al
      . The top patient safety strategies that can be encouraged for adoption now. Ann Intern Med 2013;158:365–8.
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Cannon-Bowers J,
      2. Tannenbaum S,
      3. Salas E,
      4. et al
      . Defining competencies and establishing team training requirements. In: Guzzo R, Salas E, eds. Team effectiveness and decision making in organizations. San Francisco, CA: Jossey-Bass Publishers, 1995:333–80.
    19. ↵
      National Quality Forum. Safe practices for better healthcare 2010 update: A consensus report. 2010.
    20. ↵
      1. Gaba DM
      . Crisis resource management and teamwork training in anaesthesia. Br J Anaesth 2010;105:3–6.
      OpenUrlFREE Full Text
    21. ↵
      1. Salas E,
      2. Burke CS,
      3. Bowers CA,
      4. et al
      . Team training in the skies: does crew resource management (CRM) training work? Hum Factors 2001;43:641–74.
      OpenUrlAbstract/FREE Full Text
    22. ↵
      1. Reznek M,
      2. Smith-Coggins R,
      3. Howard S,
      4. et al
      . Emergency medicine crisis resource management (EMCRM): pilot study of a simulation-based crisis management course for emergency medicine. Acad Emerg Med 2003;10:386–9.
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      1. Maynard MT,
      2. Marshall D,
      3. Dean MD
      . Crew resource management and teamwork training in health care: a review of the literature and recommendations for how to leverage such interventions to enhance patient safety. Adv Health Care Manag 2012;13:59–91.
      OpenUrlCrossRefPubMed
    24. ↵
      1. Weaver SJ,
      2. Salas E,
      3. Lyons R,
      4. et al
      . Simulation-based team training at the sharp end: a qualitative study of simulation-based team training design, implementation, and evaluation in healthcare. J Emerg Trauma Shock 2010;3:369–77.
      OpenUrlCrossRefPubMed
    25. ↵
      1. Gillespie BM,
      2. Chaboyer W,
      3. Murray P
      . Enhancing communication in surgery through team training interventions: a systematic literature review. AORN J 2010;92:642–57.
      OpenUrlCrossRefPubMed
    26. ↵
      1. Baker DP,
      2. Gustafson S,
      3. Beaubein JM,
      4. et al
      . Medical team training programs in health care. In: Henriksen K, Battles JB, Marks ES, et al. eds. Advances in patient safety: from research to implementation (volume 2: concepts and methodology). Rockville,MD: Agency for Healthcare Research and Quality (US), 2005.
    27. ↵
      1. Rabøl LI,
      2. McPhail M,
      3. Bjørn B,
      4. et al
      . Outcomes of a classroom-based team training intervention for multi-professional hospital staff. Med Teach 2012;34:868–9.
      OpenUrlCrossRefPubMed
    28. ↵
      1. McCulloch P,
      2. Rathbone J,
      3. Catchpole K
      . Interventions to improve teamwork and communications among healthcare staff. Br J Surg 2011;98:469–79.
      OpenUrlCrossRefPubMed
    29. ↵
      1. Thomas EJ
      . Improving teamwork in healthcare: current approaches and the path forward. BMJ Qual Saf 2011;20:647–50.
      OpenUrlFREE Full Text
    30. ↵
      1. Eppich W,
      2. Howard V,
      3. Vozenilek J,
      4. et al
      . Simulation-based team training in healthcare. Simul Healthc 2011;6(Suppl):S14–19.
      OpenUrlCrossRefPubMed
    31. ↵
      1. Sevdalis N,
      2. Hull L,
      3. Birnbach DJ
      . Improving patient safety in the operating theatre and perioperative care: obstacles, interventions, and priorities for accelerating progress. Br J Anaesth 2012;109(Suppl):i3–i16.
      OpenUrlAbstract/FREE Full Text
    32. ↵
      1. Salas E,
      2. Weaver SJ,
      3. Gregory ME
      . Team training for patient safety. In: Carayon P. ed Handbook of human factors and ergonomics in health care and patient safety. Boca Raton, FL: CRC Press, 2012:627–48.
    33. ↵
      1. Merién AER,
      2. van de Ven J,
      3. Mol BW,
      4. et al
      . Multidisciplinary team training in a simulation setting for acute obstetric emergencies: a systematic review. Obstet Gynecol 2010;115:1021–31.
      OpenUrlCrossRefPubMedWeb of Science
    34. ↵
      1. Salas E,
      2. Gregory ME,
      3. King HB
      . Team training can enhance patient safety—the data, the challenge ahead. Jt Comm J Qual Patient Saf 2011;37:339–40.
      OpenUrlPubMed
    35. ↵
      1. Mayer CM,
      2. Cluff L,
      3. Lin W-T,
      4. et al
      . Evaluating efforts to optimize TeamSTEPPS implementation in surgical and pediatric intensive care units. Jt Comm J Qual Patient Saf 2011;37:365–74.
      OpenUrlPubMed
    36. ↵
      1. Riley W,
      2. Davis S,
      3. Miller K,
      4. et al
      . Didactic and simulation nontechnical skills team training to improve perinatal patient outcomes in a community hospital. Jt Comm J Qual Patient Saf 2011;37:357–64.
      OpenUrlPubMed
    37. ↵
      1. Weaver SJ,
      2. Rosen MA,
      3. DiazGranados D,
      4. et al
      . Does teamwork improve performance in the operating room? A multilevel evaluation. Jt Comm J Qual Patient Saf 2010;36:133–42.
      OpenUrlPubMed
    38. ↵
      1. Neily J,
      2. Mills PD,
      3. Young-Xu Y,
      4. et al
      . Association between implementation of a medical team training program and surgical mortality. JAMA 2010;304:1693–700.
      OpenUrlCrossRefPubMedWeb of Science
    39. ↵
      1. Deering S,
      2. Rosen MA,
      3. Ludi V,
      4. et al
      . On the front lines of patient safety: implementation and evaluation of team training in Iraq. Jt Comm J Qual Patient Saf 2011;37:350–6.
      OpenUrlPubMed
    40. ↵
      1. Nielsen PE,
      2. Goldman MB,
      3. Mann S,
      4. et al
      . Effects of teamwork training on adverse outcomes and process of care in labor and delivery: a randomized controlled trial. Obstet Gynecol 2007;109:48–55.
      OpenUrlCrossRefPubMedWeb of Science
    41. ↵
      1. Andreatta P,
      2. Frankel J,
      3. Boblick Smith S,
      4. et al
      . Interdisciplinary team training identifies discrepancies in institutional policies and practices. Am J Obstet Gynecol 2011;205:298–301.
      OpenUrlCrossRefPubMed
    42. ↵
      1. Armour Forse R,
      2. Bramble JD,
      3. McQuillan R
      . Team training can improve operating room performance. Surgery 2011;150:771–8.
      OpenUrlCrossRefPubMedWeb of Science
    43. ↵
      1. Fransen AF,
      2. van de Ven J,
      3. Merién AER,
      4. et al
      . Effect of obstetric team training on team performance and medical technical skills: a randomised controlled trial. BJOG 2012;119:1387–93.
      OpenUrlCrossRefPubMed
    44. ↵
      1. McLaughlin T,
      2. Hennecke P,
      3. Garraway NR,
      4. et al
      . A predeployment trauma team training course creates confidence in teamwork and clinical skills: a post-Afghanistan deployment validation study of Canadian Forces healthcare personnel. J Trauma 2011;71:S487–93.
      OpenUrlCrossRefPubMed
    45. ↵
      1. Thomas EJ,
      2. Williams AL,
      3. Reichman EF,
      4. et al
      . Team training in the neonatal resuscitation program for interns: teamwork and quality of resuscitations. Pediatrics 2010;125:539–46.
      OpenUrlAbstract/FREE Full Text
    46. ↵
      1. Chakraborti C,
      2. Boonyasai RT,
      3. Wright SM,
      4. et al
      . A systematic review of teamwork training interventions in medical student and resident education. J Gen Intern Med 2008;23:846–53.
      OpenUrlCrossRefPubMedWeb of Science
    47. ↵
      1. Zaccaro SJ,
      2. Marks MA,
      3. DeChurch LA
      . Multiteam systems: an introduction. In: Zaccaro SJ, Marks MA, DeChurch LA. eds Multiteam systems: an organizational form for dynamic and complex environments. New York, NY: Taylor & Francis, 2011:3–32.
    48. ↵
      1. Marks MA,
      2. DeChurch LA,
      3. Mathieu JE,
      4. et al
      . Teamwork in multiteam systems. J Appl Psychol 2005;90:964–71.
      OpenUrlCrossRefPubMedWeb of Science
    49. ↵
      1. Mathieu JE,
      2. Marks MA,
      3. Zaccaro SJ
      . Multi-team systems. In: Anderson N, Ones DS, Sinangil H, et al. eds. Handbook of industrial, work, and organizational psychology, vol 2: organizational psychology. London, UK: Sage, 2001:289–313.
    50. ↵
      1. DeChurch LA,
      2. Marks MA
      . Leadership in multiteam systems. J Appl Psychol 2006;91:311–29.
      OpenUrlCrossRefPubMedWeb of Science
    51. ↵
      1. Asencio R,
      2. Carter DR,
      3. Dechurch LA,
      4. et al
      . Charting a course for collaboration: a multiteam perspective. Transl Behav Med 2012;2:487–94.
      OpenUrlCrossRefPubMed
    52. ↵
      1. Delise LA,
      2. Allen Gorman C,
      3. Brooks AM,
      4. et al
      . The effects of team training on team outcomes: a meta-analysis. Perform Improv Q 2010;22:53–80.
      OpenUrlCrossRef
    53. ↵
      1. Cooper JB,
      2. Singer SJ,
      3. Hayes J,
      4. et al
      . Design and evaluation of simulation scenarios for a program introducing patient safety, teamwork, safety leadership, and simulation to healthcare leaders and managers. Simul Healthc 2011;6:231–8.
      OpenUrlCrossRefPubMed
    54. ↵
      1. Singer SJ,
      2. Hayes J,
      3. Cooper JB,
      4. et al
      . A case for safety leadership team training of hospital managers. Health Care Manage Rev 2011;36:188–200.
      OpenUrlCrossRefPubMedWeb of Science
    55. ↵
      1. Calhoun JG,
      2. Dollett L,
      3. Sinioris ME,
      4. et al
      . Development of an interprofessional competency model for healthcare leadership. J Healthc Manag 2008;53:375–89; discussion 390–1.
      OpenUrlPubMed
    56. ↵
      1. Storey J,
      2. Holti R
      . Towards a new model of leadership for the NHS.
    57. ↵
      1. Hemmasi M,
      2. Graf LA
      . Managerial skills acquisition: a case for using business policy simulations. Simul Gaming 1992;23:298–310.
      OpenUrlAbstract
    58. ↵
      1. Young-Xu Y,
      2. Neily J,
      3. Mills PD,
      4. et al
      . Association between implementation of a medical team training program and surgical morbidity. Arch Surg 2011;146:1368–73.
      OpenUrlCrossRefPubMedWeb of Science
    59. ↵
      Agency for Healthcare Research and Quality. TeamSTEPPS: National Implementation. http://teamstepps.ahrq.gov/aboutnationalIP.htm (accessed 30 Nov 2011).
    60. ↵
      1. Baldwin TT,
      2. Ford KJ
      . Transfer of training: a review and directions for future research. Pers Psychol 1988;41:63–105.
      OpenUrlCrossRef
    61. ↵
      1. Salas E,
      2. Almeida SA,
      3. Salisbury M,
      4. et al
      . What are the critical success factors for team training in health care? Jt Comm J Qual Patient Saf 2009;35:398–405.
      OpenUrlPubMed
    62. ↵
      1. Salas E,
      2. Rosen MA
      . Building high reliability teams: progress and some reflections on teamwork training. BMJ Qual Saf 2013;22:369–73.
      OpenUrlAbstract/FREE Full Text
    63. ↵
      1. Kirkpatrick DL,
      2. Kirkpatrick JD
      . Evaluating training programs: the four levels. Third edn. San Francisco, CA: Berrett-Koehler, 2006.
    64. ↵
      1. Alliger GM,
      2. Tannenbaum SI,
      3. Bennett W,
      4. et al
      . A meta—analysis of the relations among training criteria. Pers Psychol 2006;50:341–58.
      OpenUrl
    65. ↵
      1. Figueroa MI,
      2. Sepanski R,
      3. Goldberg SP,
      4. et al
      . Improving teamwork, confidence, and collaboration among members of a pediatric cardiovascular intensive care unit multidisciplinary team using simulation-based team training. Pediatr Cardiol 2013;34:612–19.
      OpenUrlCrossRefPubMed
    66. ↵
      1. Patterson MD,
      2. Geis GL,
      3. LeMaster T,
      4. et al
      . Impact of multidisciplinary simulation-based training on patient safety in a paediatric emergency department. BMJ Qual Saf 2013;22:383–93.
      OpenUrlAbstract/FREE Full Text
    67. ↵
      1. Mishra A,
      2. Catchpole K,
      3. McCulloch P
      . The Oxford NOTECHS System: reliability and validity of a tool for measuring teamwork behaviour in the operating theatre. Qual Saf Health Care 2009;18:104–8.
      OpenUrlAbstract/FREE Full Text
    68. ↵
      1. Fletcher G,
      2. Flin R,
      3. McGeorge P,
      4. et al
      . Anaesthetists’ Non-Technical Skills (ANTS): evaluation of a behavioural marker system. Br J Anaesth 2003;90:580–8.
      OpenUrlAbstract/FREE Full Text
    69. ↵
      1. Carney BT,
      2. West P,
      3. Neily JB,
      4. et al
      . Improving perceptions of teamwork climate with the Veterans Health Administration medical team training program. Am J Med Qual 2011;26:480–4.
      OpenUrlAbstract/FREE Full Text
    70. ↵
      1. Phipps MG,
      2. Lindquist DG,
      3. McConaughey E,
      4. et al
      . Outcomes from a labor and delivery team training program with simulation component. Am J Obstet Gynecol 2012;206:3–9.
      OpenUrlCrossRefPubMed
    71. ↵
      1. Stevens L-M,
      2. Cooper JB,
      3. Raemer DB,
      4. et al
      . Educational program in crisis management for cardiac surgery teams including high realism simulation. J Thorac Cardiovasc Surg 2012;144:17–24.
      OpenUrlCrossRefPubMedWeb of Science
    72. ↵
      1. McCulloch P,
      2. Mishra A,
      3. Handa A,
      4. et al
      . The effects of aviation-style non-technical skills training on technical performance and outcome in the operating theatre. Qual Saf Health Care 2009;18:109–15.
      OpenUrlAbstract/FREE Full Text
    73. ↵
      1. Morey JC,
      2. Simon R,
      3. Jay GD,
      4. et al
      . Error reduction and performance improvement in the emergency department through formal teamwork training: evaluation results of the MedTeams project. Health Serv Res 2002;37:1553–81.
      OpenUrlCrossRefPubMedWeb of Science
    74. ↵
      1. Capella J,
      2. Smith S,
      3. Philp A,
      4. et al
      . Teamwork training improves the clinical care of trauma patients. J Surg Educ 2010;67:439–43.
      OpenUrlCrossRefPubMedWeb of Science
    75. ↵
      1. Mann S,
      2. Pratt S,
      3. Gluck P,
      4. et al
      . Assessing quality obstetrical care: development of standardized measures. Jt Comm J Qual Patient Saf 2006;32:497–505.
      OpenUrlPubMed
    76. ↵
      1. Jones KJ,
      2. Skinner AM,
      3. High R,
      4. et al
      . A theory-driven, longitudinal evaluation of the impact of team training on safety culture in 24 hospitals. BMJ Qual Saf 2013;22:394–404.
      OpenUrlAbstract/FREE Full Text
    77. ↵
      1. Bunnell CA,
      2. Gross AH,
      3. Weingart SN,
      4. et al
      . High performance teamwork training and systems redesign in outpatient oncology. BMJ Qual Saf 2013;22:405–13.
      OpenUrlAbstract/FREE Full Text
    78. ↵
      1. Fouilloux V,
      2. Gsell T,
      3. Lebel S,
      4. et al
      . Assessment of team training in management of adverse acute events occurring during cardiopulmonary bypass procedure: a pilot study based on an animal simulation model (Fouilloux, Team training in cardiac surgery). Perfusion 2014;29:44–52m.
      OpenUrlAbstract/FREE Full Text
    79. ↵
      1. Shaw DJ,
      2. Davidson JE,
      3. Smilde RI,
      4. et al
      . Multidisciplinary team training to enhance family communication in the ICU. Crit Care Med 2014;42:265–71.
      OpenUrlCrossRefPubMedWeb of Science
    80. ↵
      1. Tapson VF,
      2. Karcher RB,
      3. Weeks R
      . Crew resource management and VTE prophylaxis in surgery: a quality improvement initiative. Am J Med Qual 2011;26:423–32.
      OpenUrlAbstract/FREE Full Text
    81. ↵
      1. Mancini ME,
      2. Soar J,
      3. Bhanji F,
      4. et al
      . Part 12: education, implementation, and teams: 2010 International consensus on cardiopulmonary resuscitation and emergency cardiovascular care science with treatment recommendations. Circulation 2010;122:S539–81.
      OpenUrlFREE Full Text
    82. ↵
      Effect of Recent Refresher Training on in Situ Simulated Pediatric Tracheal Intubation Psychomotor Skill Performance. 2008.
    83. ↵
      1. Weaver SJ,
      2. Lubomksi LH,
      3. Wilson RF,
      4. et al
      . Promoting a culture of safety as a patient safety strategy: a systematic review. Ann Intern Med 2013;158:369–74.
      OpenUrlCrossRefPubMedWeb of Science
    84. ↵
      1. Weaver SJ,
      2. Salas E,
      3. King HB
      . Twelve best practices for team training evaluation in health care. Jt Comm J Qual Patient Saf 2011;37:341–9.
      OpenUrlPubMed
      1. Castner J,
      2. Foltz-Ramos K,
      3. Schwartz DG,
      4. et al
      . A leadership challenge: staff nurse perceptions after an organizational TeamSTEPPS initiative. J Nurs Adm 2012;42:467–72.
      OpenUrlCrossRefPubMed
      1. Frengley RW,
      2. Weller JM,
      3. Torrie J,
      4. et al
      . The effect of a simulation-based training intervention on the performance of established critical care unit teams. Crit Care Med 2011;39:2605–11.
      OpenUrlPubMedWeb of Science
      1. Heard LA,
      2. Fredette ME,
      3. Atmadja ML,
      4. et al
      . Perceptions of simulation-based training in crisis resource management in the endoscopy unit. Gastroenterol Nurs 2011;34:42–8.
      OpenUrlCrossRefPubMed
      1. Kirschbaum KA,
      2. Rask JP,
      3. Brennan M,
      4. et al
      . Improved climate, culture, and communication through multidisciplinary training and instruction. Am J Obstet Gynecol 2012;207:200.e1–7.
      OpenUrlPubMed
      1. Maxson PM,
      2. Dozois EJ,
      3. Holubar SD,
      4. et al
      . Enhancing nurse and physician collaboration in clinical decision making through high-fidelity interdisciplinary simulation training. Mayo Clin Proc 2011;86:31–6.
      OpenUrlCrossRefPubMedWeb of Science
      1. Steinemann S,
      2. Berg B,
      3. Skinner A,
      4. et al
      . In situ, multidisciplinary, simulation-based teamwork training improves early trauma care. J Surg Educ 2000;68:472–7.
      OpenUrl
      1. Stocker M,
      2. Allen M,
      3. Pool N,
      4. et al
      . Impact of an embedded simulation team training programme in a paediatric intensive care unit: a prospective, single-centre, longitudinal study. Intensive Care Med 2012;38:99–104.
      OpenUrlCrossRefPubMedWeb of Science
      1. Van Schaik SM,
      2. Plant J,
      3. Diane S,
      4. et al
      . Interprofessional team training in pediatric resuscitation: a low-cost, in situ simulation program that enhances self-efficacy among participants. Clin Pediatr (Phila) 2011;50:807–15.
      OpenUrlAbstract/FREE Full Text
      1. Volk MS,
      2. Ward J,
      3. Irias N,
      4. et al
      . Using medical simulation to teach crisis resource management and decision-making skills to otolaryngology housestaff. Otolaryngol Head Neck Surg 2011;145:35–42.
      OpenUrlAbstract/FREE Full Text
    View Abstract

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Contributors SJW and MAR were responsible for study concept, design, analysis, interpretation and preparation of the manuscript. SMDy was responsible for study concept, interpretation and preparation of the manuscript.

    • Funding This work was supported by funding from the Agency for Healthcare Research and Quality (AHRQ), U.S. Department of Health and Human Services (contract no. HHSA-290-2007-10062I). AHRQ reviewed contract deliverables to ensure adherence to contract requirements and quality, and a copyright release was obtained from AHRQ prior to submission of this manuscript. The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the AHRQ or the U.S. Department of Health and Human Services.

    • Competing interests None.

    • Provenance and peer review Not commissioned; externally peer reviewed.