METHODS

This study (conducted in 1998–9) was based on a quality assessment project using previously published quality measures which are detailed below.¹⁶ The sample consisted of a stratified random sample of 60 English general practices from six health authorities, selected to be nationally representative for rurality and deprivation. Within each authority, 10 practices were selected randomly to be representative of their health authority for practice size, training status, and deprivation payments. Where a practice refused participation, another with similar characteristics was selected and invited to participate; 60 out of 75 practices approached (80%) agreed. Practice staff were requested to complete measures of team climate and effectiveness. The measures used in the study are shown in fig 1.

Measures of structure

The structural variables were singlehanded status (binary variable for singlehanded or partnership); team size (number of employed staff); existence of deprivation payments to the practice (binary variable); training status of the practice (binary variable); mean length of employment of staff at the practice; routine booking intervals for patient consultations (categorical variable with three categories for 5, 7.5, and 10 minutes); and variables representing skill mix (see below). Data were collected from practice managers and through observation during visits by researchers to the practices.

Skill mix variables were created based on team composition. At present there is no clear definition of skill mix which can refer to the mix of skills, grades, and disciplines within a team. Because no data were available on individual skills, the focus in the current study was on disciplinary mix. There are no validated measures of disciplinary skill mix available, so three exploratory measures were calculated to examine the concept:

• the ratio of doctors to nurses (SM1);

• the ratio of doctors to non-medical clinical staff (SM2);

• the ratio of clinical to administrative staff (SM3).

Measures of process (team climate)

The TCI is a 65 item measure with six subscales rated on 5 point scales (from “strongly agree” to “strongly disagree”):

• participation (the “safety” of the decision making environment): items concern issues such as sharing information, influence of staff on each other, feelings of being understood and accepted;

• support for innovation (team support for new ideas): items concern issues such as openness to new ideas and sharing resources;

• reflexivity (team discussion and review of procedures): items concern issues such as review of objectives, communications, and decisions;

• task orientation (team emphasis on monitoring quality): items concern issues such as monitoring each others’ work, appraisal of weaknesses, and provision of practical ideas and help;

• clarity of objectives (team understanding of objectives): items concern issues such as agreement about objectives and their perceived usefulness;

• teamworking (degree to which teamworking is valued): items concern issues such as interdependence and perceived liking for teamworking.

The latest version of the questionnaire is based on earlier versions which have demonstrated construct, predictive, and discriminant validity.⁸ In order to avoid excessive hypothesis testing, an overall score on the TCI was computed based on the summed subscales; this score ranged from 6 to 30.

Quantification of team climate

Team climate measures are applied at an individual level, yet the definition of climate requires that the perceptions are shared. Thus, a measure of consensus is required to provide a justification for the aggregation of individual scores and evidence for the construct validity of the team level means.¹⁷ The scores of individual members were aggregated to provide an overall team climate score based on the mean of the individual team members. The r_wg(j) measure of agreement was used,^18,19 which is an index of agreement among judges concerning ratings of single items or homogenous scales. Although the use of the index has been criticised,²⁰ it was used in the present study to ensure comparability with previous analyses of the TCI. Scores of above 0.7 demonstrate acceptable agreement among respondents.

Measures of outcome

Health Care Team Effectiveness (HCTE) scale

The HCTE is a self-report measure of team effectiveness with 21 items measured on 7 point scales (from “not at all” to “to a great extent”) completed by the health professional. The development of the scale has been described elsewhere,^11,21 although validity is largely restricted to face validity at present. The items are combined into three factors—professional practice (audit, setting protocols, use of research evidence); teamworking (professional development, equal opportunities); and patient centred care (information provided to patients, provision of complaints procedure)—and an overall measure of team effectiveness (the mean of all items). One additional factor (perceived team innovation) was measured using five additional items measured on 5 point scales (from “highly stable” to “highly innovative”). Again, to restrict multiple hypothesis testing, only the overall score on the HCTE was used in the analyses (range 1–7), together with the separate measure of innovation (range 1–5).

Chronic disease management

Up to 20 patients with adult asthma, angina, and type 2 diabetes were selected randomly from disease registers. The mean number per practice was 18 (range 6–20) for angina, 19 (range 13–20) for asthma, and 18 (range 9–20) for diabetes. Data were extracted from records by researchers to identify processes defined by experts as “necessary” to provide high quality care.²² An example of the criteria used is shown in box 1. The reliability of data extraction was tested²³ and only reliable variables were included. Data items for chronic disease management were scored on a 0/1 basis depending on whether or not necessary care was provided and recorded for individual patients. These binary variables were analysed using an item response model within a multilevel framework using the procedure GLLAMM6 in Stata. Patient scores were obtained for each condition from the rescaled residuals of the item response model and rescaled to range from 0 to 100. Practice scores were computed for each condition using a multilevel model. These are equivalent to a mean score for each practice adjusted for different pools of patients within practices and the fact that many items were conditional variables.

General Practice Assessment Survey (GPAS)²⁴

The GPAS is a 53 item self-report questionnaire which assesses multiple dimensions of primary care from the perspective of the patient including access, technical care, communication, interpersonal care, trust, knowledge of the patient, nursing care, receptionists, continuity of care, referral, coordination of care, patient recommendation, and overall satisfaction. GPAS measures some constructs with report assessment pairs—for example, “in general, how often do you see your usual doctor?” then “how do you rate this?”. Only assessment items are used in the calculation of scale scores and are measured on 6 point scales. Summed scale scores are rescaled to range from 0 to 100. Factor analysis suggests that three dimensions underlie responses to the GPAS: (1) access (includes all the “access” items as well as assessments of “receptionists” and “continuity of care”); (2) patient centredness (includes items from the “communication”, “interpersonal”, and “knowledge of the patient” scales), and (3) nursing (includes only those items that relate specifically to nursing care).²⁵ To limit multiple hypothesis testing, scores were calculated for the two main dimensions of “access” and “patient centredness” based on the sum of three component scales (ranging from 0–300) and used in conjunction with the single item “overall satisfaction” scale (range 0–100).

GPAS has received preliminary validation in the UK.^26,27 Two hundred adult patients were randomly selected from health authority lists for each general practice in the project and sent a copy of GPAS and two reminders (except in one health authority where no reminders were sent).

Analysis of data

The objectives of the study were to examine the degree to which practice structure predicted climate, and the degree to which structure and climate together accounted for variation in outcomes (fig 1). Multiple regression (using Stata) was used to examine these multivariate relationships. The first regression examined the influence of team structure (independent variables) on team climate (dependent variable). The second group of regressions used team structure and climate as independent variables and the team outcomes as the dependent variables. Because individual responses to the TCI were aggregated at the level of the practice, the number of cases in the multiple regression was far lower than the total number of individual respondents, and both the absolute number of cases and cases per estimated parameter were below that considered optimal.²⁸

To examine the relative predictive power of variables, they were all entered into the equation in the first instance and non-significant variables were then removed sequentially (backward selection) using a criterion of p>0.10. This more liberal criterion was chosen because of the small sample size. Because of the relatively small numbers of cases in the analysis, outliers which had a significant influence were identified using Cook’s distance (a measure of the change in residuals when a particular case is omitted), removed from the analysis, and the model re-run. A Cook’s distance of 8/N was used to identify outliers rather than the conventional 4/N because of the small sample size.

Team scores were aggregated at the practice level because they represent the overall team view of effectiveness. Chronic disease management scores were derived from individual patients, but an overall score was computed for each practice. GPAS data were available at the level of the patient. Analyses of GPAS data with panel data techniques and ordinary least squares at the level of the patient, taking into account clustering, were similar to the analysis at the practice level and the practice level analyses are reported.

RESULTS

Response rate

Of the 60 practices that took part in the main quality assessment project, 46 (77%) provided data for the teamwork analysis. The others were not included because of practical reasons (such as lack of resources and a desire to minimise burden in practices recruited late to the project). There were no major differences between participating and non-participating practices in training status or practice size, but practices in receipt of deprivation payments were less likely to participate in the teamwork evaluation.

Questionnaires were sent to all staff employed by the practices (n=652) as well as attached staff such as health visitors and community psychiatric nurses (n=152). However, data on attached staff were variable and analysis was restricted to employed staff. The final response rate was 387/652 (59%). The response rates from individual practices ranged from 5% to 100%, with a mean of 65 (SD 26)%. Practices with a response rate of less than 30% (n=4) were removed in line with previous teamwork analyses (West, personal communication), leaving 42 practices for analysis. The characteristics of the practices are shown in table 1.

Responses to the GPAS questionnaire in the main quality project were received from 4493 patients, a response rate of 38%. The sample size in the practices included in the teamwork analyses was 3106.

Skill mix

Preliminary analysis showed that the skill mix variables SM1 and SM2 were highly correlated, and only SM1 and SM3 were used in further analyses (Pearson correlation between SM1 and SM3 0.13).

Team climate scores

All scales had satisfactory internal consistency (alpha) and agreement indices (r_wg(j)), with mean scores on the agreement index ranging from 0.84 to 0.96 and only a small number of practices with scores of <0.7 on any scale. TCI subscales were intercorrelated (r=0.48–0.76) and all were highly associated with the overall score, replicating previous work.¹²

Outcome

The intercorrelations between outcome measures are shown in table 2. All three measures of chronic disease management were highly correlated, as were self-reported team effectiveness and innovation. Patient evaluations were more highly associated with self-reported outcomes than chronic disease management. “Access” and “patient centredness” dimensions of the GPAS were highly correlated with the overall evaluation but only moderately correlated with each other.

Multivariate analyses: structure-process relationships

Two outliers were removed. The final regression model showed that the only predictor of team climate was singlehanded status (regression coefficient 2.38, 95% CI 1.47 to 3.29, n=40). Team climate was superior in singlehanded practices, and the model explained approximately 41% of the variance.

Multivariate analyses: structure, process, and outcome relationships

Longer mean booking interval was associated with higher quality management in all three chronic diseases. Higher TCI scores were associated with higher overall patient evaluations of the practice, higher quality diabetes management, and higher self-reported innovation and effectiveness. Larger team size was associated with better quality diabetes management, training status was associated with self-reported innovation, and singlehanded practices were associated with better access and lower self-reported effectiveness. The percentage of variance explained by the models ranged from 11% to 51% (table 3).

DISCUSSION

The study set out to determine whether practice structure predicts process and whether structure and process predict outcomes (fig 1). Broadly, the data suggest that significant relationships do exist between these variables, which has important implications for developments in primary care which may alter structural organisation and thus influence the climate in primary care teams and eventual outcomes.

Strengths and weaknesses of the study

Key strengths are the use of multiple, validated, conceptually independent outcome measures, and the involvement of a more representative sample of practices than previous work. However, the direction of causality cannot be determined in an observational study. It is possible that the associations between team climate and outcomes reflect either the effects of outcomes on climate or the influence of a third unmeasured variable.

Power was a fundamental problem. A post hoc power analysis²⁹ indicated that regression with 42 cases, six predictors, and α=0.10 has reasonable power (85%) to detect only “large” effects (effect size 0.35); with “medium” effects (effect size 0.15) the power is only 49%. The study may therefore be vulnerable to type II errors. Using the more liberal criterion (p<0.10) would also increase the likelihood of type I errors. The results must therefore be seen as preliminary and require replication. The removal of outliers further reduced the sample size. Removal was based on statistical criteria only (for example, to improve model fit), and not on any independent means which has potential for bias.

Conventional regression cannot capture the full relationships in fig 1 where structure affects outcome directly and is also mediated through team process. For example, a large team may have a wider range of clinical skills available, but may not benefit if the climate does not encourage effective sharing of clinical tasks. Examination of these issues would require more complex methods such as simultaneous equation approaches, but these methods have significant sample size requirements.

Clearly, the results require replication in a larger sample, although the difficulties of collecting data from busy primary care teams may make such replication difficult. The aggregation of data at the team level does make such data collection somewhat inefficient: in the present study data from 387 practice staff provided only 42 cases at the level of the team. Furthermore, the exclusion of practices with a response rate of less than 30% may bias results if response rate itself relates to team climate.

Although a large number of outcome variables was used, these variables are not the only ones that might be influenced by team climate. Team climate might have a stronger relationship with other outcomes which are more obviously influenced by interpersonal and professional functioning—for example, team morale, stress, and professional satisfaction.³ The present outcome measures were not chosen on the basis of any explicit theoretical link with team climate.

Finally, the results are dependent on the validity of the outcome measures used. Although the measures of chronic disease management were created using validated consensus group methods,²² data on patient evaluations are weakened by low response rate.

Interpretation of the results

The study found that practice structure predicts process and that structure and process predict outcomes (fig 1).

Of the structural variables entered into the analysis, only singlehanded status was related to a significantly better team climate. The reasons for this are unclear, although team climate was not related to skill mix (which might implicate interdisciplinary issues) or overall staff numbers in the practice (which might suggest an effect of team size). Conflict with GP colleagues has been identified as a stressor for GPs³⁰ and a reason for GPs moving to singlehanded status,³¹ and might explain the observed associations. However, doctors working in partnerships also report benefits of teamworking and support.³¹ In addition, team climate scores are based on responses from the entire practice, so relationships among GPs cannot be an entirely satisfactory explanation unless they are a fundamental determinant of climate for all staff.

Length of booking interval was an important predictor of the quality of care in all three chronic conditions, and the advantages of longer consultations have been discussed frequently.^32–35 Team size was an additional predictor of higher quality diabetes care, which may reflect the positive effects of multidisciplinary working or may be influenced by resource issues as larger practices are better able to find resources and generate the case load needed to support specialist provision—for example, chronic disease management clinics.

The relationship between training status and innovation makes intuitive sense, although the fact that innovation is self-reported means that this association must be treated with caution. Singlehanded status predicted access scores on the GPAS, which also confirms previous work.³⁶

The relationships between team climate, innovation, and effectiveness confirm previous research with the TCI¹² and are vulnerable to the same criticism about common method (self-report) variance.³⁷ The fact that team climate also predicts diabetes management and overall patient evaluation of the practice strengthens the case for its importance.

The mechanisms by which team climate influences chronic disease management remain to be determined. It may involve a non-specific effect whereby climate scores reflect high morale, low stress and professional satisfaction which, in turn, increases staff motivation and effectiveness. This hypothesis is supported by the finding that climate and stress (as measured by the GHQ) are related in primary care teams.³⁷ However, the content of the TCI is concerned with specific facets of team climate and more specific mechanisms may be at work. For example, “clarity of objectives” about chronic disease management may be reflected in more effective care provision. It is not immediately clear why team climate impacts on diabetes management and not the other forms of chronic disease management, although this may reflect the fact that diabetes management was most likely to be delivered through specialist clinics at the time of the survey.

The final issue concerns the nature of the link between team climate and patient evaluations. Again, it may be that team climate reflects high morale, and that patients perceive this in terms of the ways in which staff relate to them in face to face encounters (although there was no correlation with GPAS patient centredness). However, other mechanisms cannot be discounted. Team climate may foster innovation and high quality care which, in turn, affects patients’ views, although the ability of patients to accurately discern technical aspects of care is unclear.³⁸ Of course, an observational study such as this cannot discount the operation of a confounding variable which may explain the observed relationship—for example, the location of the practice and the sociodemographic nature of the practice population—although it should be noted that a crude measure of deprivation (that is, the existence of deprivation payments to the practice) was included in the analysis. It should also be noted that climate predicted patients’ overall evaluation of the practice but not the access and patient centredness dimensions. This may reflect, in part, the fact that the overall satisfaction measure relates to the entire practice whereas the patient centredness dimension relates to individual doctors. However, the single item satisfaction scale was highly correlated with the access and patient centredness dimensions.

Determination of the causal mechanisms that underlie these associations remains crucial to increasing conceptual understanding of the basic model shown in fig 1 and allowing the planning of effective team climate interventions. For example, it is important to determine whether singlehanded status is a key determinant of climate or whether the variable represents a proxy for other factors of relevance.³⁹ The present analysis used the overall team climate score as the measure of team process, whereas previous analyses¹² examined individual climate factors. However, it was felt that analysis of multiple process factors would increase the likelihood of type I errors unless corrections were made for multiple testing, and such corrections would be inappropriate given the modest sample size. More precise specification of the causal mechanisms may require additional survey work, although qualitative work (such as interviews with staff and patients from the practices in this analysis, or participant observation) may be equally useful in this regard. As research has indicated that clinical governance initiatives are increasingly focusing on the primary care team as a whole,⁴⁰ an understanding of the influence of climate on team functioning and effective quality of care is of heightened importance.