If you pay any attention to the primary care literature (as you are reading the BJGP, I assume you do), you may have noticed a new bandwagon snaking its way through our research teams and outputs in recent years. Yes, ‘mixed methods’ approaches are on the move and heading to an academic paper near you. Interviews, focus groups, surveys, observation, a sprinkling of statistics, a touch of thematic analysis: it’s all in there, strengthening inference and adding depth. Or is it? In truth, much of what is described as ‘mixed methods’ has rather fallen off the integrative path, and should more accurately be described as ‘multi-method’. So, here’s my brief road map for how to really follow the mixed methods trail.
WHAT IS MIXED METHODS RESEARCH?
Let’s set off with a look at definitions. There are many, but a pretty common sense (and commonly-used) one is John Cresswell’s:
‘Mixed methods research is a methodology for conducting research that involves collecting, analyzing, and integrating (or mixing) quantitative and qualitative research (and data) in a single study or a longitudinal program of inquiry.’ 1
That looks clear: first, we need both quantitative and qualitative approaches, and, second, these need to be integrated or mixed in some way. However, although we’re pretty good at meeting the first requirement, we’re on a steep learning curve with the second. I’ll return to this later.
Moving on, why might we want to do this? Greene, Caracelli, and Graham 2 formulated an influential scheme summarising different purposes for mixed methods research, including:
triangulation — looking for convergence or corroboration by using different methods;
complementarity — using results from one method to elaborate or clarify results from another method; and
development — using results from one method to develop or inform the other method.
These are not necessarily mutually exclusive: a single study or programme of research may draw on more than one.
Now, before we get too far along the road, I’m going to make a brief but important pitstop to consider the philosophical underpinning for mixed methods research. It will be brief as the risk of stranding our wagon in an epistemological quagmire is fairly high; but it is important, as debate continues about the most appropriate worldviews to draw on in conducting mixed methods research. The legitimacy of mixed methods depends on arguing that quantitative and qualitative approaches and methods can be logically combined. The basis for this is often located within the tradition of pragmatism;3,4 however, other paradigms, such as critical realism, are increasingly entering the arena.5 Practitioners should be aware of this debate and where their sympathies lie before concerning themselves with the technical aspects of mixed methods.
Travelling on now, we reach our critical crossroads: the nature of the integration that will take place between your qualitative and quantitative approaches to make your study truly mixed. Pat Bazeley, who has written extensively in this field, argues that:
‘Integration can be said to occur to the extent that different data elements and various strategies for analysis of those elements are combined throughout a study in such a way as to become interdependent in reaching a common theoretical or research goal, thereby producing findings that are greater than the sum of the parts.’ 6
What does this mean in practice? Broadly, as Bazeley argues:
you can integrate data from different sources in many ways;
you need to integrate data before you draw conclusions;
you must ensure the nature and depth of integration is appropriate to the aims and purpose of your study; and
your end product should be something that would not have been available without integration.
For many of us, integration at the analytical stage appears to be a particular barricade to journeying properly into mixed methods territory. Many published ‘mixed methods’ studies present findings separately; for example, outlining results from a survey of GPs followed by results from interviews with GPs. Some studies may get as far as a bit of narrative integration in the conclusions, drawing on results from both methods to make a few overarching points. Note to ourselves from earlier: this is multi-method not mixed methods. Far fewer studies outline how a survey of GPs was used to sample participants for additional interviews, the topic guide of which was derived from survey findings. I also could not find any primary care studies that presented an integrative matrix whereby quantitative survey findings were cross-tabulated with qualitative analysis of interview data to present within- and between-case data syntheses, although such approaches facilitate innovative and useful insights.7
MIXED METHODS STUDIES AND MULTI-METHOD PROGRAMMES
At this point, I think it is helpful to retrace our steps to the distinction between ‘single study’ mixed methods (for example, interviews and a survey addressing facets of the same question) and ‘multiple study’ mixed methods (for example, an entire multi-method programme of work that includes integration of data to inform conclusions). The former should ideally aspire to publish an integrated analysis within a single paper. For the latter, most published papers remain single-method, and often for good reason: however, delve into the final reports and you can see how the various methodological facets of these programmes are brought together to integrate learning. For example, a recent NIHR-funded programme — ‘Improving Access to Mental Health in Primary Care’ — used a combination of evidence synthesis, interviews with service users and carers, focus groups, ethnographic observations of GP practices, and an exploratory randomised trial to inform and implement a model to increase equity of access for older people and those from minority ethnic communities.8 The researchers used key findings from multiple sources of evidence (both primary and secondary) to guide the form and content of the psychosocial intervention for service users.9,10 As a result of the empirical evidence provided by this work, and ongoing collaboration with local service providers, the programme had a direct influence on changes to the local (then) primary care trust’s model of primary mental health care. Perhaps entirely unwittingly, the programme team concluded by directly paraphrasing Bazeley: ‘our multilevel intervention was greater than the sum of its parts’.8
So, we’re good at employing multiple methods across really big research areas, and this is something that is very much encouraged by research funders. We’re less good at the really specific mixed methods single studies. Now have a final rest, travellers, while we ponder the last enticing diversion on our journey: the use of qualitative methods within randomised controlled trials. This is increasingly done to assist with the interpretation of trial findings (especially negative ones) and the implementation of these into clinical practice, and a good thing too.11,12 However, this is a fairly specialised form of mixed methods that lags even further behind, and one that less obviously lends itself to reporting findings simultaneously. Some new adventurers are needed to take this path.
THE ‘THIRD EFFORT’
Where does our trail go from here? To achieve true integration in single studies, we need to first analyse quantitative and qualitative data separately before moving on to the assimilation of these to develop new insights. This ‘third effort’ requires substantial additional time and commitment, often unrecognised and unplanned for by funders and researchers themselves.13 Few of us have received formal training in mixed methods approaches, particularly in hands-on methods for integrating findings. The mixed methods literature, too, has seen a plethora of theoretical and conceptual articles that are not yet balanced by high-quality empirical papers. Although specialist journals exist, such as the Journal of Mixed Methods Research, it can be difficult to publish truly mixed methods approaches in the primary care literature. These are still new and unfamiliar, and journal editors too have a role to play in encouraging the development of this field, particularly seeking out peer reviewers who are up to speed with the latest developments. As we often struggle to find good peer reviewers even for qualitative work, this is an additional challenge; recognition of this gap is the first important step. And while I do not believe integrative analyses are difficult to present in the limited time slots of conferences, the silo reporting of different methods remains the norm at present.
As general practice evolves to take on increasingly complex activities, our research needs to ask the right questions about the effectiveness, efficiency, safety, equity, and appropriateness of primary care. We must also seek to better understand the adoption, implementation, and dissemination of interventions and innovations. To do this, we need to draw on many different methodological traditions. Where questions require us to examine the phenomenon of interest through more than one lens, the integrative approach of mixed methods can lead to greater insights. There is certainly sufficient interest and commitment to develop the practice, and reap the benefits of mixed methods to answer important research questions in primary care. We just need to keep those wagon wheels rolling in the right direction.
Notes
Provenance
Freely submitted; not externally peer reviewed.
Competing interests
The author has declared no competing interests.
- © British Journal of General Practice 2015