the biggest limitation of case study evidence is that

10 Case Study Advantages and Disadvantages

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

Learn about our Editorial Process

A case study in academic research is a detailed and in-depth examination of a specific instance or event, generally conducted through a qualitative approach to data.

The most common case study definition that I come across is is Robert K. Yin’s (2003, p. 13) quote provided below:

“An empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident.”

Researchers conduct case studies for a number of reasons, such as to explore complex phenomena within their real-life context, to look at a particularly interesting instance of a situation, or to dig deeper into something of interest identified in a wider-scale project.

While case studies render extremely interesting data, they have many limitations and are not suitable for all studies. One key limitation is that a case study’s findings are not usually generalizable to broader populations because one instance cannot be used to infer trends across populations.

Case Study Advantages and Disadvantages

1. in-depth analysis of complex phenomena.

Case study design allows researchers to delve deeply into intricate issues and situations.

By focusing on a specific instance or event, researchers can uncover nuanced details and layers of understanding that might be missed with other research methods, especially large-scale survey studies.

As Lee and Saunders (2017) argue,

“It allows that particular event to be studies in detail so that its unique qualities may be identified.”

This depth of analysis can provide rich insights into the underlying factors and dynamics of the studied phenomenon.

2. Holistic Understanding

Building on the above point, case studies can help us to understand a topic holistically and from multiple angles.

This means the researcher isn’t restricted to just examining a topic by using a pre-determined set of questions, as with questionnaires. Instead, researchers can use qualitative methods to delve into the many different angles, perspectives, and contextual factors related to the case study.

We can turn to Lee and Saunders (2017) again, who notes that case study researchers “develop a deep, holistic understanding of a particular phenomenon” with the intent of deeply understanding the phenomenon.

3. Examination of rare and Unusual Phenomena

We need to use case study methods when we stumble upon “rare and unusual” (Lee & Saunders, 2017) phenomena that would tend to be seen as mere outliers in population studies.

Take, for example, a child genius. A population study of all children of that child’s age would merely see this child as an outlier in the dataset, and this child may even be removed in order to predict overall trends.

So, to truly come to an understanding of this child and get insights into the environmental conditions that led to this child’s remarkable cognitive development, we need to do an in-depth study of this child specifically – so, we’d use a case study.

4. Helps Reveal the Experiences of Marginalzied Groups

Just as rare and unsual cases can be overlooked in population studies, so too can the experiences, beliefs, and perspectives of marginalized groups.

As Lee and Saunders (2017) argue, “case studies are also extremely useful in helping the expression of the voices of people whose interests are often ignored.”

Take, for example, the experiences of minority populations as they navigate healthcare systems. This was for many years a “hidden” phenomenon, not examined by researchers. It took case study designs to truly reveal this phenomenon, which helped to raise practitioners’ awareness of the importance of cultural sensitivity in medicine.

5. Ideal in Situations where Researchers cannot Control the Variables

Experimental designs – where a study takes place in a lab or controlled environment – are excellent for determining cause and effect . But not all studies can take place in controlled environments (Tetnowski, 2015).

When we’re out in the field doing observational studies or similar fieldwork, we don’t have the freedom to isolate dependent and independent variables. We need to use alternate methods.

Case studies are ideal in such situations.

A case study design will allow researchers to deeply immerse themselves in a setting (potentially combining it with methods such as ethnography or researcher observation) in order to see how phenomena take place in real-life settings.

6. Supports the generation of new theories or hypotheses

While large-scale quantitative studies such as cross-sectional designs and population surveys are excellent at testing theories and hypotheses on a large scale, they need a hypothesis to start off with!

This is where case studies – in the form of grounded research – come in. Often, a case study doesn’t start with a hypothesis. Instead, it ends with a hypothesis based upon the findings within a singular setting.

The deep analysis allows for hypotheses to emerge, which can then be taken to larger-scale studies in order to conduct further, more generalizable, testing of the hypothesis or theory.

7. Reveals the Unexpected

When a largescale quantitative research project has a clear hypothesis that it will test, it often becomes very rigid and has tunnel-vision on just exploring the hypothesis.

Of course, a structured scientific examination of the effects of specific interventions targeted at specific variables is extermely valuable.

But narrowly-focused studies often fail to shine a spotlight on unexpected and emergent data. Here, case studies come in very useful. Oftentimes, researchers set their eyes on a phenomenon and, when examining it closely with case studies, identify data and come to conclusions that are unprecedented, unforeseen, and outright surprising.

As Lars Meier (2009, p. 975) marvels, “where else can we become a part of foreign social worlds and have the chance to become aware of the unexpected?”

Disadvantages

1. not usually generalizable.

Case studies are not generalizable because they tend not to look at a broad enough corpus of data to be able to infer that there is a trend across a population.

As Yang (2022) argues, “by definition, case studies can make no claims to be typical.”

Case studies focus on one specific instance of a phenomenon. They explore the context, nuances, and situational factors that have come to bear on the case study. This is really useful for bringing to light important, new, and surprising information, as I’ve already covered.

But , it’s not often useful for generating data that has validity beyond the specific case study being examined.

2. Subjectivity in interpretation

Case studies usually (but not always) use qualitative data which helps to get deep into a topic and explain it in human terms, finding insights unattainable by quantitative data.

But qualitative data in case studies relies heavily on researcher interpretation. While researchers can be trained and work hard to focus on minimizing subjectivity (through methods like triangulation), it often emerges – some might argue it’s innevitable in qualitative studies.

So, a criticism of case studies could be that they’re more prone to subjectivity – and researchers need to take strides to address this in their studies.

3. Difficulty in replicating results

Case study research is often non-replicable because the study takes place in complex real-world settings where variables are not controlled.

So, when returning to a setting to re-do or attempt to replicate a study, we often find that the variables have changed to such an extent that replication is difficult. Furthermore, new researchers (with new subjective eyes) may catch things that the other readers overlooked.

Replication is even harder when researchers attempt to replicate a case study design in a new setting or with different participants.

Comprehension Quiz for Students

Question 1: What benefit do case studies offer when exploring the experiences of marginalized groups?

a) They provide generalizable data. b) They help express the voices of often-ignored individuals. c) They control all variables for the study. d) They always start with a clear hypothesis.

Question 2: Why might case studies be considered ideal for situations where researchers cannot control all variables?

a) They provide a structured scientific examination. b) They allow for generalizability across populations. c) They focus on one specific instance of a phenomenon. d) They allow for deep immersion in real-life settings.

Question 3: What is a primary disadvantage of case studies in terms of data applicability?

a) They always focus on the unexpected. b) They are not usually generalizable. c) They support the generation of new theories. d) They provide a holistic understanding.

Question 4: Why might case studies be considered more prone to subjectivity?

a) They always use quantitative data. b) They heavily rely on researcher interpretation, especially with qualitative data. c) They are always replicable. d) They look at a broad corpus of data.

Question 5: In what situations are experimental designs, such as those conducted in labs, most valuable?

a) When there’s a need to study rare and unusual phenomena. b) When a holistic understanding is required. c) When determining cause-and-effect relationships. d) When the study focuses on marginalized groups.

Question 6: Why is replication challenging in case study research?

a) Because they always use qualitative data. b) Because they tend to focus on a broad corpus of data. c) Due to the changing variables in complex real-world settings. d) Because they always start with a hypothesis.

Lee, B., & Saunders, M. N. K. (2017). Conducting Case Study Research for Business and Management Students. SAGE Publications.

Meir, L. (2009). Feasting on the Benefits of Case Study Research. In Mills, A. J., Wiebe, E., & Durepos, G. (Eds.). Encyclopedia of Case Study Research (Vol. 2). London: SAGE Publications.

Tetnowski, J. (2015). Qualitative case study research design.  Perspectives on fluency and fluency disorders ,  25 (1), 39-45. ( Source )

Yang, S. L. (2022). The War on Corruption in China: Local Reform and Innovation . Taylor & Francis.

Yin, R. (2003). Case Study research. Thousand Oaks, CA: Sage.

• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 101 Hidden Talents Examples
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 15 Green Flags in a Relationship
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 15 Signs you're Burnt Out, Not Lazy
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 15 Toxic Things Parents Say to their Children

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

• About This Site
• What is social theory?
• Habermas/Parsons
• Frankfurt School
• Inequalities
• Research Students
• Dirty Looks
• Latest Posts
• Pedagogy & Curriculum
• Contributors
• Publications

Select Page

What are the benefits and drawbacks of case study research?

Posted by Mark Murphy | May 24, 2014 | Method , Research Students | 0

There should be no doubt that with case studies what you gain in depth you lose in breadth – this is the unavoidable compromise that needs to be understood from the beginning of the research process. So this is neither an advantage nor a disadvantage as one aspect cancels out the benefits/drawbacks of the other – there are other benefits and drawbacks that need attention however …

• Their flexibility: case studies are popular for a number of reasons, one being that they can be conducted at various points in the research process. Researchers are known to favour them as a way to develop ideas for more extensive research in the future – pilot studies often take the form of case studies. They are also effective conduits for a broad range of research methods; in that sense they are non-prejudicial against any particular type of research – focus groups are just as welcome in case study research as are questionnaires or participant observation.
• Capturing reality: One of their key benefits is their ability to capture what Hodkinson and Hodkinson call ‘lived reality’ (2001: 3). As they put it, case studies have the potential, when applied successfully, to ‘retain more of the “noise” of real life than many other types of research’ (Hodkinson and Hodkinson, 2001: 3). The importance of ‘noise’ and its place in research is especially important in contexts such as education, for example in schools where background noise is unavoidable. Educational contexts are always complex, and as a result it is difficult to exclude other unwanted variables, ‘some of which may only have real significance for one of their students’ (Hodkinson and Hodkinson, 2001, 4).
• The challenge of generality: At the same time, given their specificity, care needs to be taken when attempting to generalise from the findings. While there’s no inherent flaw in case study design that precludes its broader application, it is preferable that researchers choose their case study sites carefully, while also basing their analysis within existing research findings that have been generated via other research designs. No design is infallible but so often has the claim against case studies been made, that some of the criticism (unwarranted and unfair in many cases) has stuck.
• Suspicion of amateurism: Less partisan researchers might wonder whether the case study offers the time and finance-strapped researcher a convenient and pragmatic source of data, providing findings and recommendations that, given the nature of case studies, can neither be confirmed nor denied, in terms of utility or veracity. Who is to say that case studies offer anything more than a story to tell, and nothing more than that?
• But alongside this suspicion is another more insiduous one – a notion that ‘stories’ are not what social science research is about. This can be a concern for those who favour  case study research, as the political consequences can be hard to ignore. That said, so much research is based either on peoples’ lives or the impact of other issues (poverty, institutional policy) on their lives, so the stories of what actually occurs in their lives or in professional environments tend to be an invaluable source of evidence. The fact is that stories (individual, collective, institutional) have a vital role to play in the world of research. And to play the specific v. general card against case study design suggests a tendency towards forms of research fundamentalism as opposed to any kind of rational and objective take on case study’s strengths and limitations.
• Preciousness: Having said that, researchers should not fall into the trap (surprising how often this happens) of assuming that case study data speaks for itself – rarely is this ever the case, an assumption that is as patronising to research subjects as it is false. The role of the researcher is both to describe social phenomena and also to explain – i.e., interpret. Without interpretation the research findings lack meaningful presentation – they present themselves as fact when of course the reality of ‘facts’ is one of the reasons why such research is carried out.
• Conflation of political/research objectives: Another trap that case study researchers sometimes fall into is presenting research findings as if they were self-evidently true, as if the stories were beyond criticism. This is often accompanied by a vague attachment to the notion that research is a political process – one that is performed as a form of liberation against for example policies that seek to ignore the stories of those who ‘suffer’ at the hands of overbearing political or economic imperatives. Case study design should not be viewed as a mechanism for providing a ‘local’ bulwark against the ‘global’ – bur rather as a mechanism for checking the veracity of universalist claims (at least one of its objectives). The valorisation of particularism can only get you so far in social research.

[This post is adapted from material in ‘Research and Education’ (Curtis, Murphy and Shields , Routledge 2014), pp. 80-82].

Reference: Hodkinson, P. and H. Hodkinson (2001). The strengths and limitations of case study research. Paper presented to the Learning and Skills Development Agency conference, Making an impact on policy and practice , Cambridge, 5-7 December 2001, downloaded from h ttp://education.exeter.ac.uk/tlc/docs/publications/LE_PH_PUB_05.12.01.rtf.26.01.2013

About The Author

Mark Murphy

Mark Murphy is a Reader in Education and Public Policy at the University of Glasgow. He previously worked as an academic at King’s College, London, University of Chester, University of Stirling, National University of Ireland, Maynooth, University College Dublin and Northern Illinois University. Mark is an active researcher in the fields of education and public policy. His research interests include educational sociology, critical theory, accountability in higher education, and public sector reform.

Related Posts

Should mobile phones be banned from Scottish Schools?

August 16, 2017

Foucault and academic leadership in Taiwan: A BERA2103 blog post

May 29, 2013

Positionality and my PhD

April 15, 2016

Dramatic encounters: My methodological engagement with Foucault [PART 2]

January 30, 2016

Recent Posts

• First Online: 27 October 2022

Cite this chapter

• R. M. Channaveer 4 &
• Rajendra Baikady 5  

3225 Accesses

1 Citations

This chapter reviews the strengths and limitations of case study as a research method in social sciences. It provides an account of an evidence base to justify why a case study is best suitable for some research questions and why not for some other research questions. Case study designing around the research context, defining the structure and modality, conducting the study, collecting the data through triangulation mode, analysing the data, and interpreting the data and theory building at the end give a holistic view of it. In addition, the chapter also focuses on the types of case study and when and where to use case study as a research method in social science research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Case Study Research

Ang, C. S., Lee, K. F., & Dipolog-Ubanan, G. F. (2019). Determinants of first-year student identity and satisfaction in higher education: A quantitative case study. SAGE Open, 9 (2), 215824401984668. https://doi.org/10.1177/2158244019846689

Baxter, P., & Jack, S. (2015). Qualitative case study methodology: Study design and implementation for novice researchers. The Qualitative Report . Published. https://doi.org/10.46743/2160-3715/2008.1573

Bhatta, T. P. (2018). Case study research, philosophical position and theory building: A methodological discussion. Dhaulagiri Journal of Sociology and Anthropology, 12 , 72–79. https://doi.org/10.3126/dsaj.v12i0.22182

Article   Google Scholar  

Bromley, P. D. (1990). Academic contributions to psychological counselling. A philosophy of science for the study of individual cases. Counselling Psychology Quarterly , 3 (3), 299–307.

Google Scholar  

Crowe, S., Cresswell, K., Robertson, A., Huby, G., Avery, A., & Sheikh, A. (2011). The case study approach. BMC Medical Research Methodology, 11 (1), 1–9.

Grässel, E., & Schirmer, B. (2006). The use of volunteers to support family carers of dementia patients: Results of a prospective longitudinal study investigating expectations towards and experience with training and professional support. Zeitschrift Fur Gerontologie Und Geriatrie, 39 (3), 217–226.

Greenwood, D., & Lowenthal, D. (2005). Case study as a means of researching social work and improving practitioner education. Journal of Social Work Practice, 19 (2), 181–193. https://doi.org/10.1080/02650530500144782

Gülseçen, S., & Kubat, A. (2006). Teaching ICT to teacher candidates using PBL: A qualitative and quantitative evaluation. Journal of Educational Technology & Society, 9 (2), 96–106.

Gomm, R., Hammersley, M., & Foster, P. (2000). Case study and generalization. Case study method , 98–115.

Hamera, J., Denzin, N. K., & Lincoln, Y. S. (2011). Performance ethnography . SAGE.

Hayes, N. (2000). Doing psychological research (p. 133). Open University Press.

Harrison, H., Birks, M., Franklin, R., & Mills, J. (2017). Case study research: Foundations and methodological orientations. In Forum qualitative sozialforschung/forum: Qualitative social research (Vol. 18, No. 1).

Iwakabe, S., & Gazzola, N. (2009). From single-case studies to practice-based knowledge: Aggregating and synthesizing case studies. Psychotherapy Research, 19 (4–5), 601–611. https://doi.org/10.1080/10503300802688494

Johnson, M. P. (2006). Decision models for the location of community corrections centers. Environment and Planning b: Planning and Design, 33 (3), 393–412. https://doi.org/10.1068/b3125

Kaarbo, J., & Beasley, R. K. (1999). A practical guide to the comparative case study method in political psychology. Political Psychology, 20 (2), 369–391. https://doi.org/10.1111/0162-895x.00149

Lovell, G. I. (2006). Justice excused: The deployment of law in everyday political encounters. Law Society Review, 40 (2), 283–324. https://doi.org/10.1111/j.1540-5893.2006.00265.x

McDonough, S., & McDonough, S. (1997). Research methods as part of English language teacher education. English Language Teacher Education and Development, 3 (1), 84–96.

Meredith, J. (1998). Building operations management theory through case and field research. Journal of Operations Management, 16 (4), 441–454. https://doi.org/10.1016/s0272-6963(98)00023-0

Mills, A. J., Durepos, G., & Wiebe, E. (Eds.). (2009). Encyclopedia of case study research . Sage Publications.

Ochieng, P. A. (2009). An analysis of the strengths and limitation of qualitative and quantitative research paradigms. Problems of Education in the 21st Century , 13 , 13.

Page, E. B., Webb, E. J., Campell, D. T., Schwart, R. D., & Sechrest, L. (1966). Unobtrusive measures: Nonreactive research in the social sciences. American Educational Research Journal, 3 (4), 317. https://doi.org/10.2307/1162043

Rashid, Y., Rashid, A., Warraich, M. A., Sabir, S. S., & Waseem, A. (2019). Case study method: A step-by-step guide for business researchers. International Journal of Qualitative Methods, 18 , 160940691986242. https://doi.org/10.1177/1609406919862424

Ridder, H. G. (2017). The theory contribution of case study research designs. Business Research, 10 (2), 281–305. https://doi.org/10.1007/s40685-017-0045-z

Sadeghi Moghadam, M. R., Ghasemnia Arabi, N., & Khoshsima, G. (2021). A Review of case study method in operations management research. International Journal of Qualitative Methods, 20 , 160940692110100. https://doi.org/10.1177/16094069211010088

Sommer, B. B., & Sommer, R. (1997). A practical guide to behavioral research: Tools and techniques . Oxford University Press.

Stake, R. E. (2010). Qualitative research: Studying how things work .

Stake, R. E. (1995). The Art of Case Study Research . Sage Publications.

Stoecker, R. (1991). Evaluating and rethinking the case study. The Sociological Review, 39 (1), 88–112.

Suryani, A. (2013). Comparing case study and ethnography as qualitative research approaches .

Taylor, S., & Berridge, V. (2006). Medicinal plants and malaria: An historical case study of research at the London School of Hygiene and Tropical Medicine in the twentieth century. Transactions of the Royal Society of Tropical Medicine and Hygiene, 100 (8), 707–714. https://doi.org/10.1016/j.trstmh.2005.11.017

Tellis, W. (1997). Introduction to case study. The Qualitative Report . Published. https://doi.org/10.46743/2160-3715/1997.2024

Towne, L., & Shavelson, R. J. (2002). Scientific research in education . National Academy Press Publications Sales Office.

Widdowson, M. D. J. (2011). Case study research methodology. International Journal of Transactional Analysis Research, 2 (1), 25–34.

Yin, R. K. (2004). The case study anthology . Sage.

Yin, R. K. (2003). Design and methods. Case Study Research , 3 (9.2).

Yin, R. K. (1994). Case study research: Design and methods (2nd ed.). Sage Publishing.

Yin, R. (1984). Case study research: Design and methods . Sage Publications Beverly Hills.

Yin, R. (1993). Applications of case study research . Sage Publishing.

Zainal, Z. (2003). An investigation into the effects of discipline-specific knowledge, proficiency and genre on reading comprehension and strategies of Malaysia ESP Students. Unpublished Ph. D. Thesis. University of Reading , 1 (1).

Zeisel, J. (1984). Inquiry by design: Tools for environment-behaviour research (No. 5). CUP archive.

Download references

Author information

Authors and affiliations.

Department of Social Work, Central University of Karnataka, Kadaganchi, India

R. M. Channaveer

Department of Social Work, University of Johannesburg, Johannesburg, South Africa

Rajendra Baikady

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to R. M. Channaveer .

Editor information

Editors and affiliations.

Centre for Family and Child Studies, Research Institute of Humanities and Social Sciences, University of Sharjah, Sharjah, United Arab Emirates

M. Rezaul Islam

Department of Development Studies, University of Dhaka, Dhaka, Bangladesh

Niaz Ahmed Khan

Department of Social Work, School of Humanities, University of Johannesburg, Johannesburg, South Africa

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Channaveer, R.M., Baikady, R. (2022). Case Study. In: Islam, M.R., Khan, N.A., Baikady, R. (eds) Principles of Social Research Methodology. Springer, Singapore. https://doi.org/10.1007/978-981-19-5441-2_21

Download citation

DOI : https://doi.org/10.1007/978-981-19-5441-2_21

Published : 27 October 2022

Publisher Name : Springer, Singapore

Print ISBN : 978-981-19-5219-7

Online ISBN : 978-981-19-5441-2

eBook Packages : Social Sciences Social Sciences (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• BMC Med Res Methodol

Case study research and causal inference

Judith green.

1 Wellcome Centre for Cultures & Environments of Health, University of Exeter, Exeter, UK

Benjamin Hanckel

2 Institute for Culture and Society, Western Sydney University, Sydney, Australia

Mark Petticrew

3 Department of Public Health, Environments & Society, London School of Hygiene & Tropical Medicine, London, UK

Sara Paparini

4 Wolfson Institute of Population Health, Queen Mary University of London, London, UK

5 Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK

Associated Data

Not applicable; no new data generated in this study.

For the purpose of open access, the author has applied a ‘Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising.

Case study methodology is widely used in health research, but has had a marginal role in evaluative studies, given it is often assumed that case studies offer little for making causal inferences. We undertook a narrative review of examples of case study research from public health and health services evaluations, with a focus on interventions addressing health inequalities. We identified five types of contribution these case studies made to evidence for causal relationships. These contributions relate to: (1) evidence about system actors’ own theories of causality; (2) demonstrative examples of causal relationships; (3) evidence about causal mechanisms; (4) evidence about the conditions under which causal mechanisms operate; and (5) inference about causality in complex systems. Case studies can and do contribute to understanding causal relationships. More transparency in the reporting of case studies would enhance their discoverability, and aid the development of a robust and pluralistic evidence base for public health and health services interventions. To strengthen the contribution that case studies make to that evidence base, researchers could: draw on wider methods from the political and social sciences, in particular on methods for robust analysis; carefully consider what population their case is a case ‘of’; and explicate the rationale used for making causal inferences.

Case study research is widely used in studies of context in public health and health services research to make sense of implementation and service delivery as enacted across complex systems. A recent meta-narrative review identified four broad, overlapping traditions in this body of work: developing and testing complex interventions; analysing change in organisations; undertaking realist evaluations; and studying complex change naturalistically [ 1 ]. Case studies can provide essential thick description of interventions, context and systems; qualitative understanding of the mechanisms of interventions; and evidence of how interventions are adapted in the ‘real’ world [ 2 , 3 ].

However, in evaluative health research, case study designs remain relegated to a minor, supporting role [ 4 , 5 ], typically at the bottom of evidence hierarchies. This relegation is largely due to assumptions that they offer little for making the kinds of causal claims that are essential to evaluating the effects of interventions. The strengths of deep, thick studies of specific cases are conventionally set against the benefits of ‘variable-based’ designs, with the former positioned as descriptive, exploratory or illustrative, and the latter as providing the strongest evidence for making causal claims about the links between interventions and outcomes. In conventional hierarchies of evidence, the primary evidence for making causal claims comes from randomised controlled trials (RCTs), in which the linear relationship between a change in one phenomenon and a later change in another can be delineated from other causal factors. The classic account of causality drawn on in epidemiology requires identifying that the relationship between two phenomena is characterised by co-variation; time order; a plausible relationship; and a lack of competing explanations [ 6 ]. The theoretical and pragmatic limitations of RCT designs for robust and generalizable evaluation of interventions in complex systems are now well-rehearsed [ 2 , 7 – 10 ]. In theory, though, random selection from a population to intervention exposure maximises ability to make causal claims: randomisation minimises risks of confounding, and enables both an unbiased estimate of the effect size of the intervention and extrapolation to the larger population [ 6 ]. Guidance for evaluations in which the intervention cannot be manipulated, such as in natural experiments, therefore typically focuses on methods for addressing threats to validity from non-random allocation in order to strengthen the credibility of probabilistic causal effect estimates [ 4 , 11 ].

This is, however, not the only kind of causal logic. Case study research typically draws on other logics for understanding causation and making causal inferences. We illustrate some of the contributions made by case studies, drawing on a narrative review of research relating to one particularly enduring and complex problem: inequalities in health. The causal chains linking interventions to equity outcomes are long and complex, with recognised limitations in the evidence base for ‘what works’ [ 12 ]. Case study research, we argue, has a critical role to play in making claims about whether, how and why interventions reduce, mitigate, or exacerbate inequalities. Our examples are drawn from a broader review of case study research [ 1 ] and supporting literature reviews [ 5 ], from which we focused on cases which had an explanatory aim, and which shed light on how interventions in public health or health services might reduce, create or sustain inequality. In this paper, we: i) outline some different kinds of evidence relevant to causal relationships that can be  derived from case study research; ii) outline what is needed for case study research to contribute to explanatory, as well as exploratory claims; and iii) advocate for greater clarity in reporting case study research to foster discoverability.

Cases and causes

There are considerable challenges in defining case study designs or approaches in ways that adequately delineate them from other research designs. Yin [ 13 ], for instance, one of the most highly cited source texts on case studies in health research [ 1 ], resists providing a definition, instead suggesting case study research is more a strategy for doing empirical research. Gerring [ 14 ] defines case study research as: “ an intensive study of a single unit for the purpose of understanding a larger class of (similar) units ” (p342, emphasis in original). This definition is useful in suggesting the basis for the inferences drawn from cases, and the need to consider the relationships between the ‘case’ (and phenomena observed within it) and the population from which it is drawn. Gerring notes that studies of single cases may have a greater “affinity” for descriptive aims, but that they can furnish “evidence for causal propositions” ( [ 14 ], p347). Case studies are, he suggests, more likely to be useful in elucidating deterministic causes: those conditions that are necessary and/or sufficient for an outcome, whereas variable based designs have advantages for demonstrating probabilistic causation, where the aim is to estimate the likelihood of two phenomena being causally related. Case studies provide evidence for the mechanisms of causal relationships (e.g. through process tracing, through observing two variables interacting in the real world) and corroboration of causal relationships (for instance, through pattern matching).

Gerring’s argument, drawing on political science examples, is that there is nothing epistemologically distinct about research using the case study: rather, it has particular affinities with certain styles of causal modelling. We take this as a point of departure to consider not whether case studies can furnish evidence to help with causal inference in health research, but rather how they have done this. From our examples on case study research on inequalities in health, we identify the kinds of claims that relate to causality that were made. We note that some relate to (1) Actors’ accounts of causality : that is, the theories of those studied about if, how and why interventions work. Other types of claim use various kinds of comparative analytic logic to elucidate evidence of causal relationships between phenomena. These claims include: (2) Demonstrations of causal relationships – in which evidence from one case is sufficient for identifying a plausible causal relationship; (3) Mechanisms – evidence of the mechanisms through which causal relationships work; (4) Conditions —evidence of the conditions under which such mechanisms operate; and (5) Complex causality —evidence for outcomes that arise from complex causality within a system. This list is neither mutually exclusive, nor exhaustive: many case studies aim to do several of these (and some more). It is also a pragmatic rather than theoretical list, focusing on the kinds of evidence claimed by researchers rather than the formal methodological underpinnings of causal claims (for a discussion of the latter, see Rohlfing [ 15 ]).

What kinds of causal evidence do case studies provide?

Actors’ accounts of causality.

This is perhaps the most common kind of evidence provided by case study research. Case studies, through in-depth research on the actors within systems, can generate evidence about how those actors themselves account for causal relationships between interventions and outcomes. This is an overt aim of many realist evaluation studies, which focus on real forces or processes that exist in the world that can provide insight into causal mechanisms for change.

Ford and colleagues [ 16 ], for example, used a series of five case studies of local health systems to explore socio-economic inequalities in unplanned hospital admission. Cases were selected on the basis of either narrowing or widening inequalities in admission, with a realist evaluation focused on delineating the context-mechanisms-outcome (CMO) configurations in each setting, to develop a broader theory of change for addressing inequalities. The case study approach used a mix of methods, including drawing on documentary data to assess the credibility of mechanisms proposed by health providers. The authors identified 17 distinct CMO configurations; and five factors that were related to trends for inequalities in emergency admissions, including health service factors (primary care workforce challenges, case finding and proactive case management) and those external to the health service (e.g., financial constraints on public services, residential gentrification). Ford and colleagues noted that none of the CMO configurations were clearly associated with improved or worsening trends in inequalities in admission.

Clearly, actors’ accounts of causality are not in themselves evidence of causality. Ford and colleagues noted that they interrogated accounts for plausibility (e.g. that interventions mentioned were prior to effects claimed) and triangulated these accounts with other sources of data, but that inability to empirically corroborate the hypothesized CMO links limited their ability to make claims about causal inference. This is crucial: actors in a system may be aware of the forces and processes shaping change but unaware of counterfactuals, and they are unlikely to have any privileged insight into whether factors are causal or simply co-occurring (see, for instance, Milton et. al. [ 17 ] on how commonly cited ‘barriers’ in accounts of not doing evaluations are also evident in actors’ accounts of doing successful evaluations). Over-interpretation of qualitative accounts of insiders’ claims about causal relationships as if they provide conclusive evidence of causal relationships is poor methodology.

This does not mean that actors’ accounts are not of value. First, in realist evaluation, as in Ford and colleagues’ study [ 16 ], these accounts provide the initial theories of change for thinking about the potential causal pathways in logic models of interventions. Second, insiders’ accounts of causality are part of the system that is being explained. An example comes from Mead and colleagues [ 18 ], who used a case study drawing largely on qualitative interviews to explore “how local actors from public health, and the wider workforce, make sense of and work on social inequalities in health” ( [ 18 ] p168). This used a case study of a partnership in northwest England to address an enduring challenge in inequalities policy: the tendency for policies that address upstream health determinants to transform, in practice, to focus more on behavioural and individual level factors . Local public health actors in the partnership recognised the structural causes of unequal health outcomes, yet discourses of policy action tended to focus only on the downstream, more individualising levels of health, and on personal choice and agency as targets for intervention. Professionals conceptualised action on inequality as relating only to the health of the poorest, rather than as a problem of a gradient in health outcomes across society. There was a geographical localism in their approach, which framed particular places as constellations of health and social problems. Drawing on theory from figurational sociology, Mead and colleagues note that actors’ own accounts are the starting point of an analysis, which then puts those accounts into play with theory about how such discourses are reproduced. The researchers suggest that partnership working itself exacerbated the individualising frameworks used to orient action, as it became a hegemonic framing, reducing the possibilities for partnerships to transform health inequalities. Here, then, a case study approach is used to shed light on the causes of a common failure in policies addressing inequalities. The authors take seriously the divergence of actors’ own accounts of causality and those of other sources, and analyse these as part of the system.

Finally, insider accounts should be taken seriously as contributing to evidence about causal inference through shedding light on the complex looping effects of theoretical models of causality and public accounts. For instance, Smith and Anderson [ 19 ], drawing on a meta-ethnographic literature review of ‘lay theorising’ about health inequalities, note that, counter to common assumptions, public understanding of the structural causes of health inequalities is sophisticated: but that it may be disavowed to avoid stigma and shame and to reassert some agency. This is an important finding for informing knowledge exchange, suggesting that further ‘awareness raising’ may be unnecessary for policy change, and counter-productive in needlessly increasing stigma and shame.

Demonstrations of causal relationships

When strategically sampled, and rooted in a sound theoretical framework, studies of single cases can provide evidence for generalizable causal inferences. The strongest examples are perhaps those that operate as ‘black swans’ for deterministic claims, in that one case may be all that is needed to show that a commonly held assumption is not generalizable. That is, a case study can demonstrate unequivocally that one phenomenon is not inevitably related to another. These can come from cases sampled because they are extreme or unusual. Prior’s [ 20 ] study of a single man in a psychiatric institution in Northern Ireland, for instance, showed that, counter to Goffman’s [ 21 ] original theory of how ‘total institutions’ lead to stigmatisation and depersonalisation, the effects of institutionalisation depended on context—in this case, how the institution related to the local community and the availability of alternative sources of self-worth available to residents.

Strategically sampled typical cases can also provide demonstrative evidence of causal relationships. To take the enduring health services challenge of inequalities in self-referral to emergency care, Hudgins and Rising’s [ 22 ] case study of a single patient is used to debunk a common assumption that high use of emergency care is related to inappropriate care-seeking by low-income patients. They look in detail at the case of “a 51-year-old low-income, recently insured, African American man in Philadelphia (USA) who had two recent ED [emergency department] visits for evaluation of frequent headaches and described fear of being at risk for a stroke.” ( [ 22 ] p50). Drawing on theories of structural violence and patient subjectivity, they use this single case to shed light on why emergency department use may appear inappropriate to providers. They analyse the interplay of gender roles, employment, and insurance status in generating competing drivers of health seeking, and point to the ways in which current policies deterring self-referral do not align well with micro- and macro-level determinants of service use. The study authors also note that because their methods generate data on ‘why’ as well ‘what’ people do, they can “lay the groundwork” ( [ 22 ], p54] for developing future interventions. Here, again, a single case is sufficient. In understanding the causal pathways that led to this patient’s use of emergency care, it is clear why policies addressing inequalities through deterring low-income users would be unlikely to work.

Mechanisms: how causal relationships operate

A strength of case study approaches compared with variable-based designs is furnishing evidence of how causal relationships operate, deriving from both direct observations of causal processes and from analysis of comparisons within and between cases. All cases contain multiple observations; variations can be observed over time and space, across or within cases [ 14 ]. Observing regularities, co-variation and deviant or surprising findings, and then using processes of analytic induction [ 23 ] or abductive logic [ 24 ] to derive, develop and test causal theories using observations from the case, can build a picture of causal pathways.

Process tracing is one formal qualitative methodology for doing this. Widely used in political and policy studies, but less in health evaluations [ 25 ], process tracing links outcomes with their causes, focusing on the mechanisms that link events on causal pathways, and on the strength of evidence for making connections on that causal chain. This requires sound theoretical knowledge (such that credible hypotheses can be developed), well described cases (ideally at different time points), observed causal processes (the activities that transfer causes to effects), and careful assessment of evidence against tests of varying strength for the necessity and sufficiency for accepting or rejecting a candidate hypothesis [ 26 , 27 ]. In health policy, process tracing methods have been combined to good effect with quantitative measures to examine casual processes leading to outcomes of interest. Campbell et. al. [ 28 ], for instance, used process tracing to look at four case studies of countries that had made progress towards universal health coverage (measured through routine data on maternal and neonatal health indicators), to identify key causal factors related to health care workforce.

An example of the use of process tracing in evaluation comes from Lohmann and colleagues’ [ 25 ] case study of a single country, Burkina Faso, to examine why performance based financing (PBF) fails to improve equity. PBF, coupled with interventions to improve health care take up among the poor, aims to improve health equity in low and middle-income countries, yet impact evaluations suggest that these benefits are typically not realised. This case study drew on data from the quantitative impact assessment; programme documentation; the intervention process evaluation; and primary qualitative research for the process tracing, in the light of the theory of change of the intervention. Lohmann and colleagues [ 25 ] identified that a number of conditions that would have been necessary for the intervention to work had not been met (such as eligible patients not receiving the card needed to access health care or providers not receiving timely reimbursement). A key finding was that although implementation challenges were a partial cause of policy failure, other causal conditions were external to the intervention, such as lack of attention to the non-health care costs incurred by the poorest to access care. Again, a single case, if there are good grounds for extrapolating to similar contexts (i.e., those in which transport is required to access health care), is enough to demonstrate a necessary part of the causal pathway between PBF and intended equity outcomes.

Conditions under which causal mechanisms operate

The example of ‘transport access’ as a necessary condition for PBF interventions to ‘work’ also illustrates a fourth type of causal evidence: that relating to the transferability of interventions. Transferable causal claims are essential for useful evidence: “(f)or policy and practice we do not need to know ‘it works somewhere’. We need evidence for ‘it-will-work-for-us’ claims: the treatment will produce the desired outcome in our situation as implemented there” ( [ 8 ] p1401). Some causal mechanisms operate widely (using a parachute will reduce injury from jumping from a plane; taking aspirin will relieve pain); others less so. In the context of health services and public health research, few interventions are likely to be widely generalizable, as the mechanisms will operate differently across contexts [ 7 ]. This context dependency is at the heart of realist evaluations, with the assumption that underlying causal mechanisms require particular contexts in order to operate, hence the focus on ‘how, where, and for whom’ interventions work [ 29 ]. Making useful claims therefore requires other kinds of evidence, relating to what Cartwright and Munro [ 30 ] call the ‘capacities’ of the intervention: what power it has to work reliably, what stops it working, what other conditions are needed for it to work. This evidence is critical for assessing whether an intervention is likely to work in a given context and to assess the intended and unintended consequences of intervention adoption and implementation. Cartwright and Munro’s recommendation is therefore to study causal powers rather than causes. That is, as well as interrogating whether the intervention ‘causes’ a particular outcome, it is also necessary to address the potential for and stability of that causal effect. To do that entails addressing a broader range of questions about the causal relationship, such as how the intervention operates in order to bring about changes in outcomes; what other conditions need to be present; what might constrain this effect; what other factors within the system also promote or constrain those effects; and what happens when different capacities interact? [ 30 ]. Case study research can be vital in providing this kind of evidence on the capacities of interventions [ 31 ].

One example is from Gibson and colleagues [ 32 ], who use within-case comparisons to shed light on why a ‘social prescribing’ intervention may have different effects across socioeconomic classes. These interventions, typically entailing link workers who connect people with complex health care needs to local services and resources, are often framed as a way to address enduring health inequalities. Drawing on sociological theory on how social class is reproduced through socially structured and unequal distribution of resources (‘capitals’), and through how these shape people’s practices and dispositions, Gibson and colleagues [ 32 ] explicate how capitals and dispositions shaped encounters with the intervention. Their analysis of similarities and differences within their case (of different clients) in the context of theory enables them to abstract inferences from the case. Drawing out the ways in which more advantaged clients mobilised capital in their pursuit of health, with dispositions more closely aligned to the intervention, they unravel classed differences in ability to benefit from the intervention, with less advantaged clients inevitably having ‘shorter horizons’ focused on day to day challenges: “This challenges the claim that social prescribing can reduce inequalities, instead suggesting it has the potential to exacerbate existing inequalities” ( [ 32 ], p6).

Case studies can shed light on the capacities of interventions to improve or exacerbate inequalities, including identifying unforeseen consequences. Hanckel and colleagues [ 33 , 34 ], for example, used a case study approach to explore implementation of a physical health intervention involving whole classes of children running for 15 min each day in the playground in schools in south London, UK. This documented considerable adaption of the intervention at the level of school, class and pupil, and identified different pathways through which the intervention might impact on inequalities. In terms of access, the intervention appeared to be equitable, in that there was no evidence of disproportionate roll out to schools with more affluent pupils or to those with fewer minority ethnic pupils [ 33 ]. However, identifying the ‘capacities’ of the intervention also identified other pathways through which it could have negative equity effects. The authors found that in practice, the intervention emphasised body weight rather than physical activity, and intervention roll-out reinforced class and ethnicity-based stigmatising discourses about lower income neighbourhoods [ 34 ].

Complex causality

There is increasing recognition that the systems that reproduce unequal health outcomes are complex: that is, that they consist of multiple interacting components that cannot be studied in isolation, and that change is likely to be non-linear, characterised by, for instance, phase shifts or feedback loops [ 35 ]. This has two rather different implications. First, case study designs can be particularly beneficial for taking a system perspective on interventions. Case studies enable a focus on aspects that are not well explicated through other designs, such as how context interacts with interventions within systems [ 7 ], or on how multiple conditional pathways might link interventions and outcomes [ 36 ]. Second, when causation is not linear, but ‘emergent’, in that it is not reducible to the accumulated changes at lower levels, evaluation designs focused on only one outcome at one level (such as weight loss in individuals) may fail to identify important effects. Case studies have an invaluable role here in unpacking and surfacing these effects at different levels within the systems within which interventions and services are delivered. One example is transport systems, which have been the focus of considerable public health interest to encourage more ‘active’ modes, in which more of the population walk or cycle, and fewer drive. However, more simplistic evaluations looking at one part of a causal chain (such as that between traffic calming interventions and local mode shift) may fail to appreciate how systems are dynamic, and that causation might be emergent. This is evident in a case study of transport policy impacts from Sheller [ 37 ], who takes the case of Philadelphia, USA, to reveal how this post-car trend has racialized effects that can exacerbate inequality. Weaving in data from participant observations, historical documentary sources and statistical evidence of declining car use, Sheller documents the racialized impacts of transport policies which may have reduced car use and encouraged active modes overall, but which have largely prioritised ‘young white’ mobility in the context of local gentrification and neglect of public transit.

One approach to synthesising evidence from multiple case studies to make claims about complex causation is Qualitative Comparative Analysis (QCA), which combines quantitative methods (based on Boolean algebra) with detailed qualitative understanding of a small to medium N sample of cases. This has strengths for identifying multiple pathways to outcomes, asymmetrical sets of conditions which lead to success or failure, or ‘conjunctural causation’, whereby some conditions are only causally linked to outcomes in relation to others [ 38 ]. There is growing interest in using these approaches in evaluative health studies [ 39 ]. One example relating to the effectiveness of interventions addressing inequalities in health comes from Blackman and colleagues [ 36 ], who explored configurations of conditions which did or did not lead to narrowing inequalities in teenage conception rates across a series of local areas as cases. This identified some surprising findings, including that ‘basic’ rather than good or exemplary standards of commissioning were associated with narrowing the equity gap, and that the proportion of minority ethnic people in the population was a key condition.

Not all case study research aims to contribute to causal inference, and neither should it [ 1 , 5 , 40 ]. However, it can. We have identified five ways in which case study evidence has contributed to causal explanations in relation to a particularly intractable challenge: inequalities in health. It is therefore time to stop claiming that case study designs have only a supporting role to play in evaluative health research. To develop a theoretical evidence base on ‘what works’, and how, in health services and public health, particularly around complex issues such as addressing unequal health outcomes, we need to draw on a greater range of evidential resources for informing decisions than is currently used. Best explanations are unlikely to be made from single studies based on one kind of causality, but instead will demand some kind of evidential pluralism [ 41 ]. That is, one single study, of any design, is unlikely to generate evidence for all links in complex causal chains between an intervention and health outcomes. We need a bricolage of evidence from a diverse range of designs [ 42 ] to make robust and credible cases for what will improve health and health equity. This will include evidence from case studies, both from single and small N studies, and from syntheses of findings from multiple cases.

Our focus on case studies that shed light on interventions for health inequalities identified the critical role that case studies can play in theorising, illuminating and making sense of: system actors’ own causal reasoning; whether there are causal links between intervention and outcome; what mechanism(s) might link them; when, where and for whom these causal relationships operate; and how unequal outcomes can be generated from the operation of complex systems. These examples draw on a range of different theoretical and methodological approaches, often from the wider political and social sciences. The approaches illustrated are rooted in very different, even incompatible, philosophical traditions: what researchers understand by ‘causality’ is diverse [ 43 ]. However, there are two commonalities across this diversity that suggest some conditions for producing good case studies that can generate evidence to support causal inferences. The first is the need for theoretically informed and comparative analysis. As Gerring [ 14 ] notes, causal inferences rely on comparisons – across units or time within a case, or between cases. It is comparison that drives the ability to make claims about the potential of interventions to produce change in outcomes of interest, and under what conditions. There are a range of approaches to qualitative data analysis, and choice of method has to be appropriate for the kinds of causal logics being explicated, and the availability of data on particular phenomena within the case. Typically, though, this will require analysis that goes beyond descriptive thematic analysis [ 31 ]. Approaches such as process tracing or analytic induction require both fine-grained and rigorous comparative analysis, and a sound theoretical underpinning that provides a framework for making credible inferences about the relationships between phenomena within the case and to the wider population from which the case is selected.

This leads to the second commonality: the need to clarify what the case is a case ‘of’, and how it relates to other candidate cases. What constitutes a ‘case’ is inevitably study specific. The examples we have drawn on include: PBF in a country [ 25 ], transport systems in a city [ 37 ], and a social prescribing intervention in primary care [ 32 ]. Clearly, in other contexts, each of these ‘cases’ could be sampling units within variable based studies (of financing systems, or countries; of infrastructures systems, or cities in a state; of particular kinds of service intervention, or primary care systems). Conversely, these cases could be populations within which lower level phenomena (districts, neighbourhoods, patients) are studied. What leads to appropriate generalisations about causal claims is a sound theorisation of the similarities and particularities of the case compared with other candidate cases: how Burkina Faso has commonalities with, or differences from, other settings in which PBF has failed to improve equity; or the contexts of gentrification and residential churn that make Philadelphia similar to other cities in the US; or the ways in which class-based dispositions and practices intersect with similar types of service provisions.

A critical question remains: How can well-conducted case study evidence be better integrated into the evidence base? Calls for greater recognition for case study designs within health research are hardly new: Flyvberg’s advocacy for a greater role for case studies in the social sciences [ 44 ] has now been cited around 20,000 times, and calls for methodological pluralism in health research go back decades [ 42 , 45 , 46 ]. Yet, case studies remain somewhat neglected, with ongoing misconceptions about their limited role, despite calls for evidence based medicine to incorporate evidence for mechanisms as complementary to evidence of correlation, rather than as inferior [ 47 ]. Even where the value of case studies for contributing to causal inference is recognised, searching for good evidence is not straightforward. Case studies are neither consistently defined nor necessarily well reported. Some of the examples in this paper do not use the term ‘case study’ in the title or abstract, although they meet our definition. Conversely, many small scale qualitative studies describe themselves as ‘case studies’, but focus on thick description rather than generalisability, and are not aiming to contribute to evaluative evidence. It is therefore challenging, currently, to undertake a more systematic review of empirical material. Forthcoming guidance on reporting case studies of context in complex systems aims to aid discoverability and transparency of reporting (Shaw S, et al: TRIPLE C Reporting Principles for Case study evaluations of the role of Context in Complex interventions, under review). This recommends including ‘case study’ in the title, clarifying how terms are used, and explicating the philosophical base of the study. To further advance the usefulness of case study evidence, we suggest that where an aim is to contribute to causal explanations, researchers should, in addition, specify their rationales for making causal inferences, and identify what broader class of phenomena their case is a case ‘of’.

Conclusions

Case study research can and does contribute to evidence for causal inferences. On challenging issues such as addressing health inequalities, we have shown how case studies provide more than detailed description of context or process. Contributions include: describing actors’ accounts of causal relationships; demonstrating theoretically plausible causal relationships; identifying mechanisms which link cause and effect; identifying the conditions under which causal relationships hold; and researching complex causation.

Acknowledgements

The research underpinning this paper was conducted as part of the Triple C study. We gratefully acknowledge the input of the wider study team, and that of the participants at a workshop held to discuss forthcoming guidance on reporting case study research.

Abbreviations

Authors’ contributions

BH, JG and MP drafted the first version of the paper, which was revised with theoretical input from SS and SP. All authors contributed to the paper and have reviewed and approved the final manuscript.

The research was funded by the Medical Research Council (MR/S014632/1). JG is supported with funding from the Wellcome Trust (WT203109/Z/16/Z). Additional funding for SP and SS salaries over the course of the study was provided by the UK National Institute for Health Research Oxford Biomedical Research Centre (BRC-1215–20008), Wellcome Trust (WT104830MA; 221457/Z/20/Z) and the University of Oxford's Higher Education Innovation Fund.

The views and opinions expressed herein are those of the authors. Funding bodies had no input to the design of the study and collection, analysis, and interpretation of data or preparation of this paper.

Availability of data and materials

Declarations.

Not applicable.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The Ultimate Guide to Qualitative Research - Part 1: The Basics

• Introduction and overview
• What is qualitative research?
• What is qualitative data?
• Examples of qualitative data
• Qualitative vs. quantitative research
• Mixed methods
• Qualitative research preparation
• Theoretical perspective
• Theoretical framework
• Literature reviews

Research question

• Conceptual framework
• Conceptual vs. theoretical framework

Data collection

• Qualitative research methods
• Focus groups
• Observational research

What is a case study?

Applications for case study research, what is a good case study, process of case study design, benefits and limitations of case studies.

• Ethnographical research
• Ethical considerations
• Confidentiality and privacy
• Power dynamics
• Reflexivity

Case studies

Case studies are essential to qualitative research , offering a lens through which researchers can investigate complex phenomena within their real-life contexts. This chapter explores the concept, purpose, applications, examples, and types of case studies and provides guidance on how to conduct case study research effectively.

Whereas quantitative methods look at phenomena at scale, case study research looks at a concept or phenomenon in considerable detail. While analyzing a single case can help understand one perspective regarding the object of research inquiry, analyzing multiple cases can help obtain a more holistic sense of the topic or issue. Let's provide a basic definition of a case study, then explore its characteristics and role in the qualitative research process.

Definition of a case study

A case study in qualitative research is a strategy of inquiry that involves an in-depth investigation of a phenomenon within its real-world context. It provides researchers with the opportunity to acquire an in-depth understanding of intricate details that might not be as apparent or accessible through other methods of research. The specific case or cases being studied can be a single person, group, or organization – demarcating what constitutes a relevant case worth studying depends on the researcher and their research question .

Among qualitative research methods , a case study relies on multiple sources of evidence, such as documents, artifacts, interviews , or observations , to present a complete and nuanced understanding of the phenomenon under investigation. The objective is to illuminate the readers' understanding of the phenomenon beyond its abstract statistical or theoretical explanations.

Characteristics of case studies

Case studies typically possess a number of distinct characteristics that set them apart from other research methods. These characteristics include a focus on holistic description and explanation, flexibility in the design and data collection methods, reliance on multiple sources of evidence, and emphasis on the context in which the phenomenon occurs.

Furthermore, case studies can often involve a longitudinal examination of the case, meaning they study the case over a period of time. These characteristics allow case studies to yield comprehensive, in-depth, and richly contextualized insights about the phenomenon of interest.

The role of case studies in research

Case studies hold a unique position in the broader landscape of research methods aimed at theory development. They are instrumental when the primary research interest is to gain an intensive, detailed understanding of a phenomenon in its real-life context.

In addition, case studies can serve different purposes within research - they can be used for exploratory, descriptive, or explanatory purposes, depending on the research question and objectives. This flexibility and depth make case studies a valuable tool in the toolkit of qualitative researchers.

Remember, a well-conducted case study can offer a rich, insightful contribution to both academic and practical knowledge through theory development or theory verification, thus enhancing our understanding of complex phenomena in their real-world contexts.

What is the purpose of a case study?

Case study research aims for a more comprehensive understanding of phenomena, requiring various research methods to gather information for qualitative analysis . Ultimately, a case study can allow the researcher to gain insight into a particular object of inquiry and develop a theoretical framework relevant to the research inquiry.

Why use case studies in qualitative research?

Using case studies as a research strategy depends mainly on the nature of the research question and the researcher's access to the data.

Conducting case study research provides a level of detail and contextual richness that other research methods might not offer. They are beneficial when there's a need to understand complex social phenomena within their natural contexts.

The explanatory, exploratory, and descriptive roles of case studies

Case studies can take on various roles depending on the research objectives. They can be exploratory when the research aims to discover new phenomena or define new research questions; they are descriptive when the objective is to depict a phenomenon within its context in a detailed manner; and they can be explanatory if the goal is to understand specific relationships within the studied context. Thus, the versatility of case studies allows researchers to approach their topic from different angles, offering multiple ways to uncover and interpret the data .

The impact of case studies on knowledge development

Case studies play a significant role in knowledge development across various disciplines. Analysis of cases provides an avenue for researchers to explore phenomena within their context based on the collected data.

This can result in the production of rich, practical insights that can be instrumental in both theory-building and practice. Case studies allow researchers to delve into the intricacies and complexities of real-life situations, uncovering insights that might otherwise remain hidden.

Types of case studies

In qualitative research , a case study is not a one-size-fits-all approach. Depending on the nature of the research question and the specific objectives of the study, researchers might choose to use different types of case studies. These types differ in their focus, methodology, and the level of detail they provide about the phenomenon under investigation.

Understanding these types is crucial for selecting the most appropriate approach for your research project and effectively achieving your research goals. Let's briefly look at the main types of case studies.

Exploratory case studies

Exploratory case studies are typically conducted to develop a theory or framework around an understudied phenomenon. They can also serve as a precursor to a larger-scale research project. Exploratory case studies are useful when a researcher wants to identify the key issues or questions which can spur more extensive study or be used to develop propositions for further research. These case studies are characterized by flexibility, allowing researchers to explore various aspects of a phenomenon as they emerge, which can also form the foundation for subsequent studies.

Descriptive case studies

Descriptive case studies aim to provide a complete and accurate representation of a phenomenon or event within its context. These case studies are often based on an established theoretical framework, which guides how data is collected and analyzed. The researcher is concerned with describing the phenomenon in detail, as it occurs naturally, without trying to influence or manipulate it.

Explanatory case studies

Explanatory case studies are focused on explanation - they seek to clarify how or why certain phenomena occur. Often used in complex, real-life situations, they can be particularly valuable in clarifying causal relationships among concepts and understanding the interplay between different factors within a specific context.

Intrinsic, instrumental, and collective case studies

These three categories of case studies focus on the nature and purpose of the study. An intrinsic case study is conducted when a researcher has an inherent interest in the case itself. Instrumental case studies are employed when the case is used to provide insight into a particular issue or phenomenon. A collective case study, on the other hand, involves studying multiple cases simultaneously to investigate some general phenomena.

Each type of case study serves a different purpose and has its own strengths and challenges. The selection of the type should be guided by the research question and objectives, as well as the context and constraints of the research.

The flexibility, depth, and contextual richness offered by case studies make this approach an excellent research method for various fields of study. They enable researchers to investigate real-world phenomena within their specific contexts, capturing nuances that other research methods might miss. Across numerous fields, case studies provide valuable insights into complex issues.

Critical information systems research

Case studies provide a detailed understanding of the role and impact of information systems in different contexts. They offer a platform to explore how information systems are designed, implemented, and used and how they interact with various social, economic, and political factors. Case studies in this field often focus on examining the intricate relationship between technology, organizational processes, and user behavior, helping to uncover insights that can inform better system design and implementation.

Health research

Health research is another field where case studies are highly valuable. They offer a way to explore patient experiences, healthcare delivery processes, and the impact of various interventions in a real-world context.

Case studies can provide a deep understanding of a patient's journey, giving insights into the intricacies of disease progression, treatment effects, and the psychosocial aspects of health and illness.

Asthma research studies

Specifically within medical research, studies on asthma often employ case studies to explore the individual and environmental factors that influence asthma development, management, and outcomes. A case study can provide rich, detailed data about individual patients' experiences, from the triggers and symptoms they experience to the effectiveness of various management strategies. This can be crucial for developing patient-centered asthma care approaches.

Other fields

Apart from the fields mentioned, case studies are also extensively used in business and management research, education research, and political sciences, among many others. They provide an opportunity to delve into the intricacies of real-world situations, allowing for a comprehensive understanding of various phenomena.

Case studies, with their depth and contextual focus, offer unique insights across these varied fields. They allow researchers to illuminate the complexities of real-life situations, contributing to both theory and practice.

Whatever field you're in, ATLAS.ti puts your data to work for you

Download a free trial of ATLAS.ti to turn your data into insights.

Understanding the key elements of case study design is crucial for conducting rigorous and impactful case study research. A well-structured design guides the researcher through the process, ensuring that the study is methodologically sound and its findings are reliable and valid. The main elements of case study design include the research question , propositions, units of analysis, and the logic linking the data to the propositions.

The research question is the foundation of any research study. A good research question guides the direction of the study and informs the selection of the case, the methods of collecting data, and the analysis techniques. A well-formulated research question in case study research is typically clear, focused, and complex enough to merit further detailed examination of the relevant case(s).

Propositions

Propositions, though not necessary in every case study, provide a direction by stating what we might expect to find in the data collected. They guide how data is collected and analyzed by helping researchers focus on specific aspects of the case. They are particularly important in explanatory case studies, which seek to understand the relationships among concepts within the studied phenomenon.

Units of analysis

The unit of analysis refers to the case, or the main entity or entities that are being analyzed in the study. In case study research, the unit of analysis can be an individual, a group, an organization, a decision, an event, or even a time period. It's crucial to clearly define the unit of analysis, as it shapes the qualitative data analysis process by allowing the researcher to analyze a particular case and synthesize analysis across multiple case studies to draw conclusions.

Argumentation

This refers to the inferential model that allows researchers to draw conclusions from the data. The researcher needs to ensure that there is a clear link between the data, the propositions (if any), and the conclusions drawn. This argumentation is what enables the researcher to make valid and credible inferences about the phenomenon under study.

Understanding and carefully considering these elements in the design phase of a case study can significantly enhance the quality of the research. It can help ensure that the study is methodologically sound and its findings contribute meaningful insights about the case.

Ready to jumpstart your research with ATLAS.ti?

Conceptualize your research project with our intuitive data analysis interface. Download a free trial today.

Conducting a case study involves several steps, from defining the research question and selecting the case to collecting and analyzing data . This section outlines these key stages, providing a practical guide on how to conduct case study research.

Defining the research question

The first step in case study research is defining a clear, focused research question. This question should guide the entire research process, from case selection to analysis. It's crucial to ensure that the research question is suitable for a case study approach. Typically, such questions are exploratory or descriptive in nature and focus on understanding a phenomenon within its real-life context.

Selecting and defining the case

The selection of the case should be based on the research question and the objectives of the study. It involves choosing a unique example or a set of examples that provide rich, in-depth data about the phenomenon under investigation. After selecting the case, it's crucial to define it clearly, setting the boundaries of the case, including the time period and the specific context.

Previous research can help guide the case study design. When considering a case study, an example of a case could be taken from previous case study research and used to define cases in a new research inquiry. Considering recently published examples can help understand how to select and define cases effectively.

Developing a detailed case study protocol

A case study protocol outlines the procedures and general rules to be followed during the case study. This includes the data collection methods to be used, the sources of data, and the procedures for analysis. Having a detailed case study protocol ensures consistency and reliability in the study.

The protocol should also consider how to work with the people involved in the research context to grant the research team access to collecting data. As mentioned in previous sections of this guide, establishing rapport is an essential component of qualitative research as it shapes the overall potential for collecting and analyzing data.

Collecting data

Gathering data in case study research often involves multiple sources of evidence, including documents, archival records, interviews, observations, and physical artifacts. This allows for a comprehensive understanding of the case. The process for gathering data should be systematic and carefully documented to ensure the reliability and validity of the study.

Analyzing and interpreting data

The next step is analyzing the data. This involves organizing the data , categorizing it into themes or patterns , and interpreting these patterns to answer the research question. The analysis might also involve comparing the findings with prior research or theoretical propositions.

Writing the case study report

The final step is writing the case study report . This should provide a detailed description of the case, the data, the analysis process, and the findings. The report should be clear, organized, and carefully written to ensure that the reader can understand the case and the conclusions drawn from it.

Each of these steps is crucial in ensuring that the case study research is rigorous, reliable, and provides valuable insights about the case.

The type, depth, and quality of data in your study can significantly influence the validity and utility of the study. In case study research, data is usually collected from multiple sources to provide a comprehensive and nuanced understanding of the case. This section will outline the various methods of collecting data used in case study research and discuss considerations for ensuring the quality of the data.

Interviews are a common method of gathering data in case study research. They can provide rich, in-depth data about the perspectives, experiences, and interpretations of the individuals involved in the case. Interviews can be structured , semi-structured , or unstructured , depending on the research question and the degree of flexibility needed.

Observations

Observations involve the researcher observing the case in its natural setting, providing first-hand information about the case and its context. Observations can provide data that might not be revealed in interviews or documents, such as non-verbal cues or contextual information.

Documents and artifacts

Documents and archival records provide a valuable source of data in case study research. They can include reports, letters, memos, meeting minutes, email correspondence, and various public and private documents related to the case.

These records can provide historical context, corroborate evidence from other sources, and offer insights into the case that might not be apparent from interviews or observations.

Physical artifacts refer to any physical evidence related to the case, such as tools, products, or physical environments. These artifacts can provide tangible insights into the case, complementing the data gathered from other sources.

Ensuring the quality of data collection

Determining the quality of data in case study research requires careful planning and execution. It's crucial to ensure that the data is reliable, accurate, and relevant to the research question. This involves selecting appropriate methods of collecting data, properly training interviewers or observers, and systematically recording and storing the data. It also includes considering ethical issues related to collecting and handling data, such as obtaining informed consent and ensuring the privacy and confidentiality of the participants.

Data analysis

Analyzing case study research involves making sense of the rich, detailed data to answer the research question. This process can be challenging due to the volume and complexity of case study data. However, a systematic and rigorous approach to analysis can ensure that the findings are credible and meaningful. This section outlines the main steps and considerations in analyzing data in case study research.

Organizing the data

The first step in the analysis is organizing the data. This involves sorting the data into manageable sections, often according to the data source or the theme. This step can also involve transcribing interviews, digitizing physical artifacts, or organizing observational data.

Categorizing and coding the data

Once the data is organized, the next step is to categorize or code the data. This involves identifying common themes, patterns, or concepts in the data and assigning codes to relevant data segments. Coding can be done manually or with the help of software tools, and in either case, qualitative analysis software can greatly facilitate the entire coding process. Coding helps to reduce the data to a set of themes or categories that can be more easily analyzed.

Identifying patterns and themes

After coding the data, the researcher looks for patterns or themes in the coded data. This involves comparing and contrasting the codes and looking for relationships or patterns among them. The identified patterns and themes should help answer the research question.

Interpreting the data

Once patterns and themes have been identified, the next step is to interpret these findings. This involves explaining what the patterns or themes mean in the context of the research question and the case. This interpretation should be grounded in the data, but it can also involve drawing on theoretical concepts or prior research.

Verification of the data

The last step in the analysis is verification. This involves checking the accuracy and consistency of the analysis process and confirming that the findings are supported by the data. This can involve re-checking the original data, checking the consistency of codes, or seeking feedback from research participants or peers.

Like any research method , case study research has its strengths and limitations. Researchers must be aware of these, as they can influence the design, conduct, and interpretation of the study.

Understanding the strengths and limitations of case study research can also guide researchers in deciding whether this approach is suitable for their research question . This section outlines some of the key strengths and limitations of case study research.

Benefits include the following:

• Rich, detailed data: One of the main strengths of case study research is that it can generate rich, detailed data about the case. This can provide a deep understanding of the case and its context, which can be valuable in exploring complex phenomena.
• Flexibility: Case study research is flexible in terms of design , data collection , and analysis . A sufficient degree of flexibility allows the researcher to adapt the study according to the case and the emerging findings.
• Real-world context: Case study research involves studying the case in its real-world context, which can provide valuable insights into the interplay between the case and its context.
• Multiple sources of evidence: Case study research often involves collecting data from multiple sources , which can enhance the robustness and validity of the findings.

On the other hand, researchers should consider the following limitations:

• Generalizability: A common criticism of case study research is that its findings might not be generalizable to other cases due to the specificity and uniqueness of each case.
• Time and resource intensive: Case study research can be time and resource intensive due to the depth of the investigation and the amount of collected data.
• Complexity of analysis: The rich, detailed data generated in case study research can make analyzing the data challenging.
• Subjectivity: Given the nature of case study research, there may be a higher degree of subjectivity in interpreting the data , so researchers need to reflect on this and transparently convey to audiences how the research was conducted.

Being aware of these strengths and limitations can help researchers design and conduct case study research effectively and interpret and report the findings appropriately.

Ready to analyze your data with ATLAS.ti?

See how our intuitive software can draw key insights from your data with a free trial today.

The Advantages and Limitations of Single Case Study Analysis

As Andrew Bennett and Colin Elman have recently noted, qualitative research methods presently enjoy “an almost unprecedented popularity and vitality… in the international relations sub-field”, such that they are now “indisputably prominent, if not pre-eminent” (2010: 499). This is, they suggest, due in no small part to the considerable advantages that case study methods in particular have to offer in studying the “complex and relatively unstructured and infrequent phenomena that lie at the heart of the subfield” (Bennett and Elman, 2007: 171). Using selected examples from within the International Relations literature[1], this paper aims to provide a brief overview of the main principles and distinctive advantages and limitations of single case study analysis. Divided into three inter-related sections, the paper therefore begins by first identifying the underlying principles that serve to constitute the case study as a particular research strategy, noting the somewhat contested nature of the approach in ontological, epistemological, and methodological terms. The second part then looks to the principal single case study types and their associated advantages, including those from within the recent ‘third generation’ of qualitative International Relations (IR) research. The final section of the paper then discusses the most commonly articulated limitations of single case studies; while accepting their susceptibility to criticism, it is however suggested that such weaknesses are somewhat exaggerated. The paper concludes that single case study analysis has a great deal to offer as a means of both understanding and explaining contemporary international relations.

The term ‘case study’, John Gerring has suggested, is “a definitional morass… Evidently, researchers have many different things in mind when they talk about case study research” (2006a: 17). It is possible, however, to distil some of the more commonly-agreed principles. One of the most prominent advocates of case study research, Robert Yin (2009: 14) defines it as “an empirical enquiry that investigates a contemporary phenomenon in depth and within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident”. What this definition usefully captures is that case studies are intended – unlike more superficial and generalising methods – to provide a level of detail and understanding, similar to the ethnographer Clifford Geertz’s (1973) notion of ‘thick description’, that allows for the thorough analysis of the complex and particularistic nature of distinct phenomena. Another frequently cited proponent of the approach, Robert Stake, notes that as a form of research the case study “is defined by interest in an individual case, not by the methods of inquiry used”, and that “the object of study is a specific, unique, bounded system” (2008: 443, 445). As such, three key points can be derived from this – respectively concerning issues of ontology, epistemology, and methodology – that are central to the principles of single case study research.

First, the vital notion of ‘boundedness’ when it comes to the particular unit of analysis means that defining principles should incorporate both the synchronic (spatial) and diachronic (temporal) elements of any so-called ‘case’. As Gerring puts it, a case study should be “an intensive study of a single unit… a spatially bounded phenomenon – e.g. a nation-state, revolution, political party, election, or person – observed at a single point in time or over some delimited period of time” (2004: 342). It is important to note, however, that – whereas Gerring refers to a single unit of analysis – it may be that attention also necessarily be given to particular sub-units. This points to the important difference between what Yin refers to as an ‘holistic’ case design, with a single unit of analysis, and an ’embedded’ case design with multiple units of analysis (Yin, 2009: 50-52). The former, for example, would examine only the overall nature of an international organization, whereas the latter would also look to specific departments, programmes, or policies etc.

Secondly, as Tim May notes of the case study approach, “even the most fervent advocates acknowledge that the term has entered into understandings with little specification or discussion of purpose and process” (2011: 220). One of the principal reasons for this, he argues, is the relationship between the use of case studies in social research and the differing epistemological traditions – positivist, interpretivist, and others – within which it has been utilised. Philosophy of science concerns are obviously a complex issue, and beyond the scope of much of this paper. That said, the issue of how it is that we know what we know – of whether or not a single independent reality exists of which we as researchers can seek to provide explanation – does lead us to an important distinction to be made between so-called idiographic and nomothetic case studies (Gerring, 2006b). The former refers to those which purport to explain only a single case, are concerned with particularisation, and hence are typically (although not exclusively) associated with more interpretivist approaches. The latter are those focused studies that reflect upon a larger population and are more concerned with generalisation, as is often so with more positivist approaches[2]. The importance of this distinction, and its relation to the advantages and limitations of single case study analysis, is returned to below.

Thirdly, in methodological terms, given that the case study has often been seen as more of an interpretivist and idiographic tool, it has also been associated with a distinctly qualitative approach (Bryman, 2009: 67-68). However, as Yin notes, case studies can – like all forms of social science research – be exploratory, descriptive, and/or explanatory in nature. It is “a common misconception”, he notes, “that the various research methods should be arrayed hierarchically… many social scientists still deeply believe that case studies are only appropriate for the exploratory phase of an investigation” (Yin, 2009: 6). If case studies can reliably perform any or all three of these roles – and given that their in-depth approach may also require multiple sources of data and the within-case triangulation of methods – then it becomes readily apparent that they should not be limited to only one research paradigm. Exploratory and descriptive studies usually tend toward the qualitative and inductive, whereas explanatory studies are more often quantitative and deductive (David and Sutton, 2011: 165-166). As such, the association of case study analysis with a qualitative approach is a “methodological affinity, not a definitional requirement” (Gerring, 2006a: 36). It is perhaps better to think of case studies as transparadigmatic; it is mistaken to assume single case study analysis to adhere exclusively to a qualitative methodology (or an interpretivist epistemology) even if it – or rather, practitioners of it – may be so inclined. By extension, this also implies that single case study analysis therefore remains an option for a multitude of IR theories and issue areas; it is how this can be put to researchers’ advantage that is the subject of the next section.

Having elucidated the defining principles of the single case study approach, the paper now turns to an overview of its main benefits. As noted above, a lack of consensus still exists within the wider social science literature on the principles and purposes – and by extension the advantages and limitations – of case study research. Given that this paper is directed towards the particular sub-field of International Relations, it suggests Bennett and Elman’s (2010) more discipline-specific understanding of contemporary case study methods as an analytical framework. It begins however, by discussing Harry Eckstein’s seminal (1975) contribution to the potential advantages of the case study approach within the wider social sciences.

Eckstein proposed a taxonomy which usefully identified what he considered to be the five most relevant types of case study. Firstly were so-called configurative-idiographic studies, distinctly interpretivist in orientation and predicated on the assumption that “one cannot attain prediction and control in the natural science sense, but only understanding ( verstehen )… subjective values and modes of cognition are crucial” (1975: 132). Eckstein’s own sceptical view was that any interpreter ‘simply’ considers a body of observations that are not self-explanatory and “without hard rules of interpretation, may discern in them any number of patterns that are more or less equally plausible” (1975: 134). Those of a more post-modernist bent, of course – sharing an “incredulity towards meta-narratives”, in Lyotard’s (1994: xxiv) evocative phrase – would instead suggest that this more free-form approach actually be advantageous in delving into the subtleties and particularities of individual cases.

Eckstein’s four other types of case study, meanwhile, promote a more nomothetic (and positivist) usage. As described, disciplined-configurative studies were essentially about the use of pre-existing general theories, with a case acting “passively, in the main, as a receptacle for putting theories to work” (Eckstein, 1975: 136). As opposed to the opportunity this presented primarily for theory application, Eckstein identified heuristic case studies as explicit theoretical stimulants – thus having instead the intended advantage of theory-building. So-called p lausibility probes entailed preliminary attempts to determine whether initial hypotheses should be considered sound enough to warrant more rigorous and extensive testing. Finally, and perhaps most notably, Eckstein then outlined the idea of crucial case studies , within which he also included the idea of ‘most-likely’ and ‘least-likely’ cases; the essential characteristic of crucial cases being their specific theory-testing function.

Whilst Eckstein’s was an early contribution to refining the case study approach, Yin’s (2009: 47-52) more recent delineation of possible single case designs similarly assigns them roles in the applying, testing, or building of theory, as well as in the study of unique cases[3]. As a subset of the latter, however, Jack Levy (2008) notes that the advantages of idiographic cases are actually twofold. Firstly, as inductive/descriptive cases – akin to Eckstein’s configurative-idiographic cases – whereby they are highly descriptive, lacking in an explicit theoretical framework and therefore taking the form of “total history”. Secondly, they can operate as theory-guided case studies, but ones that seek only to explain or interpret a single historical episode rather than generalise beyond the case. Not only does this therefore incorporate ‘single-outcome’ studies concerned with establishing causal inference (Gerring, 2006b), it also provides room for the more postmodern approaches within IR theory, such as discourse analysis, that may have developed a distinct methodology but do not seek traditional social scientific forms of explanation.

Applying specifically to the state of the field in contemporary IR, Bennett and Elman identify a ‘third generation’ of mainstream qualitative scholars – rooted in a pragmatic scientific realist epistemology and advocating a pluralistic approach to methodology – that have, over the last fifteen years, “revised or added to essentially every aspect of traditional case study research methods” (2010: 502). They identify ‘process tracing’ as having emerged from this as a central method of within-case analysis. As Bennett and Checkel observe, this carries the advantage of offering a methodologically rigorous “analysis of evidence on processes, sequences, and conjunctures of events within a case, for the purposes of either developing or testing hypotheses about causal mechanisms that might causally explain the case” (2012: 10).

Harnessing various methods, process tracing may entail the inductive use of evidence from within a case to develop explanatory hypotheses, and deductive examination of the observable implications of hypothesised causal mechanisms to test their explanatory capability[4]. It involves providing not only a coherent explanation of the key sequential steps in a hypothesised process, but also sensitivity to alternative explanations as well as potential biases in the available evidence (Bennett and Elman 2010: 503-504). John Owen (1994), for example, demonstrates the advantages of process tracing in analysing whether the causal factors underpinning democratic peace theory are – as liberalism suggests – not epiphenomenal, but variously normative, institutional, or some given combination of the two or other unexplained mechanism inherent to liberal states. Within-case process tracing has also been identified as advantageous in addressing the complexity of path-dependent explanations and critical junctures – as for example with the development of political regime types – and their constituent elements of causal possibility, contingency, closure, and constraint (Bennett and Elman, 2006b).

Bennett and Elman (2010: 505-506) also identify the advantages of single case studies that are implicitly comparative: deviant, most-likely, least-likely, and crucial cases. Of these, so-called deviant cases are those whose outcome does not fit with prior theoretical expectations or wider empirical patterns – again, the use of inductive process tracing has the advantage of potentially generating new hypotheses from these, either particular to that individual case or potentially generalisable to a broader population. A classic example here is that of post-independence India as an outlier to the standard modernisation theory of democratisation, which holds that higher levels of socio-economic development are typically required for the transition to, and consolidation of, democratic rule (Lipset, 1959; Diamond, 1992). Absent these factors, MacMillan’s single case study analysis (2008) suggests the particularistic importance of the British colonial heritage, the ideology and leadership of the Indian National Congress, and the size and heterogeneity of the federal state.

Most-likely cases, as per Eckstein above, are those in which a theory is to be considered likely to provide a good explanation if it is to have any application at all, whereas least-likely cases are ‘tough test’ ones in which the posited theory is unlikely to provide good explanation (Bennett and Elman, 2010: 505). Levy (2008) neatly refers to the inferential logic of the least-likely case as the ‘Sinatra inference’ – if a theory can make it here, it can make it anywhere. Conversely, if a theory cannot pass a most-likely case, it is seriously impugned. Single case analysis can therefore be valuable for the testing of theoretical propositions, provided that predictions are relatively precise and measurement error is low (Levy, 2008: 12-13). As Gerring rightly observes of this potential for falsification:

“a positivist orientation toward the work of social science militates toward a greater appreciation of the case study format, not a denigration of that format, as is usually supposed” (Gerring, 2007: 247, emphasis added).

In summary, the various forms of single case study analysis can – through the application of multiple qualitative and/or quantitative research methods – provide a nuanced, empirically-rich, holistic account of specific phenomena. This may be particularly appropriate for those phenomena that are simply less amenable to more superficial measures and tests (or indeed any substantive form of quantification) as well as those for which our reasons for understanding and/or explaining them are irreducibly subjective – as, for example, with many of the normative and ethical issues associated with the practice of international relations. From various epistemological and analytical standpoints, single case study analysis can incorporate both idiographic sui generis cases and, where the potential for generalisation may exist, nomothetic case studies suitable for the testing and building of causal hypotheses. Finally, it should not be ignored that a signal advantage of the case study – with particular relevance to international relations – also exists at a more practical rather than theoretical level. This is, as Eckstein noted, “that it is economical for all resources: money, manpower, time, effort… especially important, of course, if studies are inherently costly, as they are if units are complex collective individuals ” (1975: 149-150, emphasis added).

Limitations

Single case study analysis has, however, been subject to a number of criticisms, the most common of which concern the inter-related issues of methodological rigour, researcher subjectivity, and external validity. With regard to the first point, the prototypical view here is that of Zeev Maoz (2002: 164-165), who suggests that “the use of the case study absolves the author from any kind of methodological considerations. Case studies have become in many cases a synonym for freeform research where anything goes”. The absence of systematic procedures for case study research is something that Yin (2009: 14-15) sees as traditionally the greatest concern due to a relative absence of methodological guidelines. As the previous section suggests, this critique seems somewhat unfair; many contemporary case study practitioners – and representing various strands of IR theory – have increasingly sought to clarify and develop their methodological techniques and epistemological grounding (Bennett and Elman, 2010: 499-500).

A second issue, again also incorporating issues of construct validity, concerns that of the reliability and replicability of various forms of single case study analysis. This is usually tied to a broader critique of qualitative research methods as a whole. However, whereas the latter obviously tend toward an explicitly-acknowledged interpretive basis for meanings, reasons, and understandings:

“quantitative measures appear objective, but only so long as we don’t ask questions about where and how the data were produced… pure objectivity is not a meaningful concept if the goal is to measure intangibles [as] these concepts only exist because we can interpret them” (Berg and Lune, 2010: 340).

The question of researcher subjectivity is a valid one, and it may be intended only as a methodological critique of what are obviously less formalised and researcher-independent methods (Verschuren, 2003). Owen (1994) and Layne’s (1994) contradictory process tracing results of interdemocratic war-avoidance during the Anglo-American crisis of 1861 to 1863 – from liberal and realist standpoints respectively – are a useful example. However, it does also rest on certain assumptions that can raise deeper and potentially irreconcilable ontological and epistemological issues. There are, regardless, plenty such as Bent Flyvbjerg (2006: 237) who suggest that the case study contains no greater bias toward verification than other methods of inquiry, and that “on the contrary, experience indicates that the case study contains a greater bias toward falsification of preconceived notions than toward verification”.

The third and arguably most prominent critique of single case study analysis is the issue of external validity or generalisability. How is it that one case can reliably offer anything beyond the particular? “We always do better (or, in the extreme, no worse) with more observation as the basis of our generalization”, as King et al write; “in all social science research and all prediction, it is important that we be as explicit as possible about the degree of uncertainty that accompanies out prediction” (1994: 212). This is an unavoidably valid criticism. It may be that theories which pass a single crucial case study test, for example, require rare antecedent conditions and therefore actually have little explanatory range. These conditions may emerge more clearly, as Van Evera (1997: 51-54) notes, from large-N studies in which cases that lack them present themselves as outliers exhibiting a theory’s cause but without its predicted outcome. As with the case of Indian democratisation above, it would logically be preferable to conduct large-N analysis beforehand to identify that state’s non-representative nature in relation to the broader population.

There are, however, three important qualifiers to the argument about generalisation that deserve particular mention here. The first is that with regard to an idiographic single-outcome case study, as Eckstein notes, the criticism is “mitigated by the fact that its capability to do so [is] never claimed by its exponents; in fact it is often explicitly repudiated” (1975: 134). Criticism of generalisability is of little relevance when the intention is one of particularisation. A second qualifier relates to the difference between statistical and analytical generalisation; single case studies are clearly less appropriate for the former but arguably retain significant utility for the latter – the difference also between explanatory and exploratory, or theory-testing and theory-building, as discussed above. As Gerring puts it, “theory confirmation/disconfirmation is not the case study’s strong suit” (2004: 350). A third qualification relates to the issue of case selection. As Seawright and Gerring (2008) note, the generalisability of case studies can be increased by the strategic selection of cases. Representative or random samples may not be the most appropriate, given that they may not provide the richest insight (or indeed, that a random and unknown deviant case may appear). Instead, and properly used , atypical or extreme cases “often reveal more information because they activate more actors… and more basic mechanisms in the situation studied” (Flyvbjerg, 2006). Of course, this also points to the very serious limitation, as hinted at with the case of India above, that poor case selection may alternatively lead to overgeneralisation and/or grievous misunderstandings of the relationship between variables or processes (Bennett and Elman, 2006a: 460-463).

As Tim May (2011: 226) notes, “the goal for many proponents of case studies […] is to overcome dichotomies between generalizing and particularizing, quantitative and qualitative, deductive and inductive techniques”. Research aims should drive methodological choices, rather than narrow and dogmatic preconceived approaches. As demonstrated above, there are various advantages to both idiographic and nomothetic single case study analyses – notably the empirically-rich, context-specific, holistic accounts that they have to offer, and their contribution to theory-building and, to a lesser extent, that of theory-testing. Furthermore, while they do possess clear limitations, any research method involves necessary trade-offs; the inherent weaknesses of any one method, however, can potentially be offset by situating them within a broader, pluralistic mixed-method research strategy. Whether or not single case studies are used in this fashion, they clearly have a great deal to offer.

References 

Bennett, A. and Checkel, J. T. (2012) ‘Process Tracing: From Philosophical Roots to Best Practice’, Simons Papers in Security and Development, No. 21/2012, School for International Studies, Simon Fraser University: Vancouver.

Bennett, A. and Elman, C. (2006a) ‘Qualitative Research: Recent Developments in Case Study Methods’, Annual Review of Political Science , 9, 455-476.

Bennett, A. and Elman, C. (2006b) ‘Complex Causal Relations and Case Study Methods: The Example of Path Dependence’, Political Analysis , 14, 3, 250-267.

Bennett, A. and Elman, C. (2007) ‘Case Study Methods in the International Relations Subfield’, Comparative Political Studies , 40, 2, 170-195.

Bennett, A. and Elman, C. (2010) Case Study Methods. In C. Reus-Smit and D. Snidal (eds) The Oxford Handbook of International Relations . Oxford University Press: Oxford. Ch. 29.

Berg, B. and Lune, H. (2012) Qualitative Research Methods for the Social Sciences . Pearson: London.

Bryman, A. (2012) Social Research Methods . Oxford University Press: Oxford.

David, M. and Sutton, C. D. (2011) Social Research: An Introduction . SAGE Publications Ltd: London.

Diamond, J. (1992) ‘Economic development and democracy reconsidered’, American Behavioral Scientist , 35, 4/5, 450-499.

Eckstein, H. (1975) Case Study and Theory in Political Science. In R. Gomm, M. Hammersley, and P. Foster (eds) Case Study Method . SAGE Publications Ltd: London.

Flyvbjerg, B. (2006) ‘Five Misunderstandings About Case-Study Research’, Qualitative Inquiry , 12, 2, 219-245.

Geertz, C. (1973) The Interpretation of Cultures: Selected Essays by Clifford Geertz . Basic Books Inc: New York.

Gerring, J. (2004) ‘What is a Case Study and What Is It Good for?’, American Political Science Review , 98, 2, 341-354.

Gerring, J. (2006a) Case Study Research: Principles and Practices . Cambridge University Press: Cambridge.

Gerring, J. (2006b) ‘Single-Outcome Studies: A Methodological Primer’, International Sociology , 21, 5, 707-734.

Gerring, J. (2007) ‘Is There a (Viable) Crucial-Case Method?’, Comparative Political Studies , 40, 3, 231-253.

King, G., Keohane, R. O. and Verba, S. (1994) Designing Social Inquiry: Scientific Inference in Qualitative Research . Princeton University Press: Chichester.

Layne, C. (1994) ‘Kant or Cant: The Myth of the Democratic Peace’, International Security , 19, 2, 5-49.

Levy, J. S. (2008) ‘Case Studies: Types, Designs, and Logics of Inference’, Conflict Management and Peace Science , 25, 1-18.

Lipset, S. M. (1959) ‘Some Social Requisites of Democracy: Economic Development and Political Legitimacy’, The American Political Science Review , 53, 1, 69-105.

Lyotard, J-F. (1984) The Postmodern Condition: A Report on Knowledge . University of Minnesota Press: Minneapolis.

MacMillan, A. (2008) ‘Deviant Democratization in India’, Democratization , 15, 4, 733-749.

Maoz, Z. (2002) Case study methodology in international studies: from storytelling to hypothesis testing. In F. P. Harvey and M. Brecher (eds) Evaluating Methodology in International Studies . University of Michigan Press: Ann Arbor.

May, T. (2011) Social Research: Issues, Methods and Process . Open University Press: Maidenhead.

Owen, J. M. (1994) ‘How Liberalism Produces Democratic Peace’, International Security , 19, 2, 87-125.

Seawright, J. and Gerring, J. (2008) ‘Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options’, Political Research Quarterly , 61, 2, 294-308.

Stake, R. E. (2008) Qualitative Case Studies. In N. K. Denzin and Y. S. Lincoln (eds) Strategies of Qualitative Inquiry . Sage Publications: Los Angeles. Ch. 17.

Van Evera, S. (1997) Guide to Methods for Students of Political Science . Cornell University Press: Ithaca.

Verschuren, P. J. M. (2003) ‘Case study as a research strategy: some ambiguities and opportunities’, International Journal of Social Research Methodology , 6, 2, 121-139.

Yin, R. K. (2009) Case Study Research: Design and Methods . SAGE Publications Ltd: London.

[1] The paper follows convention by differentiating between ‘International Relations’ as the academic discipline and ‘international relations’ as the subject of study.

[2] There is some similarity here with Stake’s (2008: 445-447) notion of intrinsic cases, those undertaken for a better understanding of the particular case, and instrumental ones that provide insight for the purposes of a wider external interest.

[3] These may be unique in the idiographic sense, or in nomothetic terms as an exception to the generalising suppositions of either probabilistic or deterministic theories (as per deviant cases, below).

[4] Although there are “philosophical hurdles to mount”, according to Bennett and Checkel, there exists no a priori reason as to why process tracing (as typically grounded in scientific realism) is fundamentally incompatible with various strands of positivism or interpretivism (2012: 18-19). By extension, it can therefore be incorporated by a range of contemporary mainstream IR theories.

— Written by: Ben Willis Written at: University of Plymouth Written for: David Brockington Date written: January 2013

Further Reading on E-International Relations

• Identity in International Conflicts: A Case Study of the Cuban Missile Crisis
• Imperialism’s Legacy in the Study of Contemporary Politics: The Case of Hegemonic Stability Theory
• Recreating a Nation’s Identity Through Symbolism: A Chinese Case Study
• Ontological Insecurity: A Case Study on Israeli-Palestinian Conflict in Jerusalem
• Terrorists or Freedom Fighters: A Case Study of ETA
• A Critical Assessment of Eco-Marxism: A Ghanaian Case Study

Please Consider Donating

Before you download your free e-book, please consider donating to support open access publishing.

E-IR is an independent non-profit publisher run by an all volunteer team. Your donations allow us to invest in new open access titles and pay our bandwidth bills to ensure we keep our existing titles free to view. Any amount, in any currency, is appreciated. Many thanks!

Donations are voluntary and not required to download the e-book - your link to download is below.

Chapter 11 Case Research

Case research, also called case study, is a method of intensively studying a phenomenon over time within its natural setting in one or a few sites. Multiple methods of data collection, such as interviews, observations, prerecorded documents, and secondary data, may be employed and inferences about the phenomenon of interest tend to be rich, detailed, and contextualized. Case research can be employed in a positivist manner for the purpose of theory testing or in an interpretive manner for theory building. This method is more popular in business research than in other social science disciplines.

Case research has several unique strengths over competing research methods such as experiments and survey research. First, case research can be used for either theory building or theory testing, while positivist methods can be used for theory testing only. In interpretive case research, the constructs of interest need not be known in advance, but may emerge from the data as the research progresses. Second, the research questions can be modified during the research process if the original questions are found to be less relevant or salient. This is not possible in any positivist method after the data is collected. Third, case research can help derive richer, more contextualized, and more authentic interpretation of the phenomenon of interest than most other research methods by virtue of its ability to capture a rich array of contextual data. Fourth, the phenomenon of interest can be studied from the perspectives of multiple participants and using multiple levels of analysis (e.g., individual and organizational).

At the same time, case research also has some inherent weaknesses. Because it involves no experimental control, internal validity of inferences remain weak. Of course, this is a common problem for all research methods except experiments. However, as described later, the problem of controls may be addressed in case research using “natural controls”. Second, the quality of inferences derived from case research depends heavily on the integrative powers of the researcher. An experienced researcher may see concepts and patterns in case data that a novice researcher may miss. Hence, the findings are sometimes criticized as being subjective. Finally, because the inferences are heavily contextualized, it may be difficult to generalize inferences from case research to other contexts or other organizations.

It is important to recognize that case research is different from case descriptions such as Harvard case studies discussed in business classes. While case descriptions typically describe an organizational problem in rich detail with the goal of stimulating classroom discussion and critical thinking among students, or analyzing how well an organization handled a specific problem, case research is a formal research technique that involves a scientific method to derive explanations of organizational phenomena.

Case research is a difficult research method that requires advanced research skills on the part of the researcher, and is therefore, often prone to error. Benbasat et al. (1987) [8] describe five problems frequently encountered in case research studies. First, many case research studies start without specific research questions, and therefore end up without having any specific answers or insightful inferences. Second, case sites are often chosen based on access and convenience, rather than based on the fit with the research questions, and are therefore cannot adequately address the research questions of interest. Third, researchers often do not validate or triangulate data collected using multiple means, which may lead to biased interpretation based on responses from biased interviewees. Fourth, many studies provide very little details on how data was collected (e.g., what interview questions were used, which documents were examined, what are the organizational positions of each interviewee, etc.) or analyzed, which may raise doubts about the reliability of the inferences. Finally, despite its strength as a longitudinal research method, many case research studies do not follow through a phenomenon in a longitudinal manner, and hence present only a cross-sectional and limited view of organizational processes and phenomena that are temporal in nature.

Key Decisions in Case Research

Several key decisions must be made by a researcher when considering a case research method. First, is this the right method for the research questions being studied? The case research method is particularly appropriate for exploratory studies for discovering relevant constructs in areas where theory building at the formative stages, for studies where the experiences of participants and context of actions are critical, and for studies aimed at understanding complex, temporal processes (why and how of a phenomenon) rather than factors or causes (what). This method is well-suited for studying complex organizational processes that involve multiple participants and interacting sequences of events, such as organizational change and large-scale technology implementation projects.

Second, what is the appropriate unit of analysis for a case research study? Since case research can simultaneously examine multiple units of analyses, the researcher must decide whether she wishes to study a phenomenon at the individual, group, and organizational level or at multiple levels. For instance, a study of group decision making or group work may combine individual-level constructs such as individual participation in group activities with group-level constructs, such as group cohesion and group leadership, to derive richer understanding than that can be achieved from a single level of analysis.

Third, should the researcher employ a single-case or multiple-case design? The single case design is more appropriate at the outset of theory generation, if the situation is unique or extreme, if it is revelatory (i.e., the situation was previously inaccessible for scientific investigation), or if it represents a critical or contrary case for testing a well-formulated theory. The multiple case design is more appropriate for theory testing, for establishing generalizability of inferences, and for developing richer and more nuanced interpretations of a phenomenon. Yin (1984) [9] recommends the use of multiple case sites with replication logic, viewing each case site as similar to one experimental study, and following rules of scientific rigor similar to that used in positivist research.

Fourth, what sites should be chosen for case research? Given the contextualized nature of inferences derived from case research, site selection is a particularly critical issue because selecting the wrong site may lead to the wrong inferences. If the goal of the research is to test theories or examine generalizability of inferences, then dissimilar case sites should be selected to increase variance in observations. For instance, if the goal of the research is to understand the process of technology implementation in firms, a mix of large, mid-sized, and small firms should be selected to examine whether the technology implementation process differs with firm size. Site selection should not be opportunistic or based on convenience, but rather based on the fit with research questions through a process called “theoretical sampling.”

Fifth, what techniques of data collection should be used in case research? Although interview (either open-ended/unstructured or focused/structured) is by far the most popular data collection technique for case research, interview data can be supplemented or corroborated with other techniques such as direct observation (e.g., attending executive meetings, briefings, and planning sessions), documentation (e.g., internal reports, presentations, and memoranda, as well as external accounts such as newspaper reports), archival records (e.g., organization charts, financial records, etc.), and physical artifacts (e.g., devices, outputs, tools). Furthermore, the researcher should triangulate or validate observed data by comparing responses between interviewees.

Conducting Case Research

Most case research studies tend to be interpretive in nature. Interpretive case research is an inductive technique where evidence collected from one or more case sites is systematically analyzed and synthesized to allow concepts and patterns to emerge for the purpose of building new theories or expanding existing ones. Eisenhardt (1989) [10] propose a “roadmap” for building theories from case research, a slightly modified version of which is described below. For positivist case research, some of the following stages may need to be rearranged or modified; however sampling, data collection, and data analytic techniques should generally remain the same.

Define research questions. Like any other scientific research, case research must also start with defining research questions that are theoretically and practically interesting, and identifying some intuitive expectations about possible answers to those research questions or preliminary constructs to guide initial case design. In positivist case research, the preliminary constructs are based on theory, while no such theory or hypotheses should be considered ex ante in interpretive research. These research questions and constructs may be changed in interpretive case research later on, if needed, but not in positivist case research.

Select case sites. The researcher should use a process of “theoretical sampling” (not random sampling) to identify case sites. In this approach, case sites are chosen based on theoretical, rather than statistical, considerations, for instance, to replicate previous cases, to extend preliminary theories, or to fill theoretical categories or polar types. Care should be taken to ensure that the selected sites fit the nature of research questions, minimize extraneous variance or noise due to firm size, industry effects, and so forth, and maximize variance in the dependent variables of interest. For instance, if the goal of the research is to examine how some firms innovate better than others, the researcher should select firms of similar size within the same industry to reduce industry or size effects, and select some more innovative and some less innovative firms to increase variation in firm innovation. Instead of cold-calling or writing to a potential site, it is better to contact someone at executive level inside each firm who has the authority to approve the project or someone who can identify a person of authority. During initial conversations, the researcher should describe the nature and purpose of the project, any potential benefits to the case site, how the collected data will be used, the people involved in data collection (other researchers, research assistants, etc.), desired interviewees, and the amount of time, effort, and expense required of the sponsoring organization. The researcher must also assure confidentiality, privacy, and anonymity of both the firm and the individual respondents.

Create instruments and protocols. Since the primary mode of data collection in case research is interviews, an interview protocol should be designed to guide the interview process. This is essentially a list of questions to be asked. Questions may be open-ended (unstructured) or closed-ended (structured) or a combination of both. The interview protocol must be strictly followed, and the interviewer must not change the order of questions or skip any question during the interview process, although some deviations are allowed to probe further into respondent’s comments that are ambiguous or interesting. The interviewer must maintain a neutral tone, not lead respondents in any specific direction, say by agreeing or disagreeing with any response. More detailed interviewing techniques are discussed in the chapter on surveys. In addition, additional sources of data, such as internal documents and memorandums, annual reports, financial statements, newspaper articles, and direct observations should be sought to supplement and validate interview data.

Select respondents. Select interview respondents at different organizational levels, departments, and positions to obtain divergent perspectives on the phenomenon of interest. A random sampling of interviewees is most preferable; however a snowball sample is acceptable, as long as a diversity of perspectives is represented in the sample. Interviewees must be selected based on their personal involvement with the phenomenon under investigation and their ability and willingness to answer the researcher’s questions accurately and adequately, and not based on convenience or access.

Start data collection . It is usually a good idea to electronically record interviews for future reference. However, such recording must only be done with the interviewee’s consent. Even when interviews are being recorded, the interviewer should take notes to capture important comments or critical observations, behavioral responses (e.g., respondent’s body language), and the researcher’s personal impressions about the respondent and his/her comments. After each interview is completed, the entire interview should be transcribed verbatim into a text document for analysis.

Conduct within-case data analysis. Data analysis may follow or overlap with data collection. Overlapping data collection and analysis has the advantage of adjusting the data collection process based on themes emerging from data analysis, or to further probe into these themes. Data analysis is done in two stages. In the first stage (within-case analysis), the researcher should examine emergent concepts separately at each case site and patterns between these concepts to generate an initial theory of the problem of interest. The researcher can interview data subjectively to “make sense” of the research problem in conjunction with using her personal observations or experience at the case site. Alternatively, a coding strategy such as Glasser and Strauss’ (1967) grounded theory approach, using techniques such as open coding, axial coding, and selective coding, may be used to derive a chain of evidence and inferences. These techniques are discussed in detail in a later chapter. Homegrown techniques, such as graphical representation of data (e.g., network diagram) or sequence analysis (for longitudinal data) may also be used. Note that there is no predefined way of analyzing the various types of case data, and the data analytic techniques can be modified to fit the nature of the research project.

Conduct cross-case analysis. Multi-site case research requires cross-case analysis as the second stage of data analysis. In such analysis, the researcher should look for similar concepts and patterns between different case sites, ignoring contextual differences that may lead to idiosyncratic conclusions. Such patterns may be used for validating the initial theory, or for refining it (by adding or dropping concepts and relationships) to develop a more inclusive and generalizable theory. This analysis may take several forms. For instance, the researcher may select categories (e.g., firm size, industry, etc.) and look for within-group similarities and between-group differences (e.g., high versus low performers, innovators versus laggards). Alternatively, she can compare firms in a pair-wise manner listing similarities and differences across pairs of firms.

Build and test hypotheses. Based on emergent concepts and themes that are generalizable across case sites, tentative hypotheses are constructed. These hypotheses should be compared iteratively with observed evidence to see if they fit the observed data, and if not, the constructs or relationships should be refined. Also the researcher should compare the emergent constructs and hypotheses with those reported in the prior literature to make a case for their internal validity and generalizability. Conflicting findings must not be rejected, but rather reconciled using creative thinking to generate greater insight into the emergent theory. When further iterations between theory and data yield no new insights or changes in the existing theory, “theoretical saturation” is reached and the theory building process is complete.

Write case research report. In writing the report, the researcher should describe very clearly the detailed process used for sampling, data collection, data analysis, and hypotheses development, so that readers can independently assess the reasonableness, strength, and consistency of the reported inferences. A high level of clarity in research methods is needed to ensure that the findings are not biased by the researcher’s preconceptions.

Interpretive Case Research Exemplar

Perhaps the best way to learn about interpretive case research is to examine an illustrative example. One such example is Eisenhardt’s (1989) [11] study of how executives make decisions in high-velocity environments (HVE). Readers are advised to read the original paper published in Academy of Management Journal before reading the synopsis in this chapter. In this study, Eisenhardt examined how executive teams in some HVE firms make fast decisions, while those in other firms cannot, and whether faster decisions improve or worsen firm performance in such environments. HVE was defined as one where demand, competition, and technology changes so rapidly and discontinuously that the information available is often inaccurate, unavailable or obsolete. The implicit assumptions were that (1) it is hard to make fast decisions with inadequate information in HVE, and (2) fast decisions may not be efficient and may result in poor firm performance.

Reviewing the prior literature on executive decision -making, Eisenhardt found several patterns, although none of these patterns were specific to high-velocity environments. The literature suggested that in the interest of expediency, firms that make faster decisions obtain input from fewer sources, consider fewer alternatives, make limited analysis, restrict user participation in decision-making, centralize decision-making authority, and has limited internal conflicts. However, Eisenhardt contended that these views may not necessarily explain how decision makers make decisions in high-velocity environments, where decisions must be made quickly and with incomplete information, while maintaining high decision quality.

To examine this phenomenon, Eisenhardt conducted an inductive study of eight firms in the personal computing industry. The personal computing industry was undergoing dramatic changes in technology with the introduction of the UNIX operating system, RISC architecture, and 64KB random access memory in the 1980’s, increased competition with the entry of IBM into the personal computing business, and growing customer demand with double-digit demand growth, and therefore fit the profile of the high-velocity environment. This was a multiple case design with replication logic, where each case was expected to confirm or disconfirm inferences from other cases. Case sites were selected based on their access and proximity to the researcher; however, all of these firms operated in the high-velocity personal computing industry in California’s Silicon Valley area. The collocation of firms in the same industry and the same area ruled out any “noise” or variance in dependent variables (decision speed or performance) attributable to industry or geographic differences.

The study employed an embedded design with multiple levels of analysis: decision (comparing multiple strategic decisions within each firm), executive teams (comparing different teams responsible for strategic decisions), and the firm (overall firm performance). Data was collected from five sources:

• Initial interviews with Chief Executive Officers: CEOs were asked questions about their firm’s competitive strategy, distinctive competencies, major competitors, performance, and recent/ongoing major strategic decisions. Based on these interviews, several strategic decisions were selected in each firm for further investigation. Four criteria were used to select decisions: (1) the decisions involved the firm’s strategic positioning,

(2) the decisions had high stakes, (3) the decisions involved multiple functions, and (4) the decisions were representative of strategic decision-making process in that firm.

• Interviews with divisional heads: Each divisional head was asked sixteen open-ended questions, ranging from their firm’s competitive strategy, functional strategy, top management team members, frequency and nature of interaction with team, typical decision making processes, how each of the previously identified decision was made, and how long it took them to make those decisions. Interviews lasted between 1.5 and 2 hours, and sometimes extended to 4 hours. To focus on facts and actual events rather than respondents’ perceptions or interpretations, a “courtroom” style questioning was employed, such as when did this happen, what did you do, etc. Interviews were conducted by two people, and the data was validated by cross-checking facts and impressions made by the interviewer and note-taker. All interview data was recorded, however notes were also taken during each interview, which ended with the interviewer’s overall impressions. Using a “24-hour rule”, detailed field notes were completed within 24 hours of the interview, so that some data or impressions were not lost to recall.
• Questionnaires: Executive team members at each firm were completed a survey questionnaire that captured quantitative data on the extent of conflict and power distribution in their firm.
• Secondary data: Industry reports and internal documents such as demographics of the executive teams (responsible for strategic decisions), financial performance of firms, and so forth, were examined.
• Personal observation: Lastly, the researcher attended a 1-day strategy session and a weekly executive meeting at two firms in her sample.

Data analysis involved a combination of quantitative and qualitative techniques. Quantitative data on conflict and power were analyzed for patterns across firms/decisions. Qualitative interview data was combined into decision climate profiles, using profile traits (e.g., impatience) mentioned by more than one executive. For within-case analysis, decision stories were created for each strategic decision by combining executive accounts of the key decision events into a timeline. For cross-case analysis, pairs of firms were compared for similarities and differences, categorized along variables of interest such as decision speed and firm performance. Based on these analyses, tentative constructs and propositions were derived inductively from each decision story within firm categories. Each decision case was revisited to confirm the proposed relationships. The inferred propositions were compared with findings from the existing literature to reconcile examine differences with the extant literature and to generate new insights from the case findings. Finally, the validated propositions were synthesized into an inductive theory of strategic decision-making by firms in high-velocity environments.

Inferences derived from this multiple case research contradicted several decision-making patterns expected from the existing literature. First, fast decision makers in high-velocity environments used more information, and not less information as suggested by the previous literature. However, these decision makers used more real-time information (an insight not available from prior research), which helped them identify and respond to problems, opportunities, and changing circumstances faster. Second, fast decision makers examined more (not fewer) alternatives. However, they considered these multiple alternatives in a simultaneous manner, while slower decision makers examined fewer alternatives in a sequential manner. Third, fast decision makers did not centralize decision making or restrict inputs from others, as the literature suggested. Rather, these firms used a two-tiered decision process in which experienced counselors were asked for inputs in the first stage, following by a rapid comparison and decision selection in the second stage. Fourth, fast decision makers did not have less conflict, as expected from the literature, but employed better conflict resolution techniques to reduce conflict and improve decision-making speed. Finally, fast decision makers exhibited superior firm performance by virtue of their built-in cognitive, emotional, and political processes that led to rapid closure of major decisions.

Positivist Case Research Exemplar

Case research can also be used in a positivist manner to test theories or hypotheses. Such studies are rare, but Markus (1983) [12] provides an exemplary illustration in her study of technology implementation at the Golden Triangle Company (a pseudonym). The goal of this study was to understand why a newly implemented financial information system (FIS), intended to improve the productivity and performance of accountants at GTC was supported by accountants at GTC’s corporate headquarters but resisted by divisional accountants at GTC branches. Given the uniqueness of the phenomenon of interest, this was a single-case research study.

To explore the reasons behind user resistance of FIS, Markus posited three alternative explanations: (1) system-determined theory: resistance was caused by factors related to an inadequate system, such as its technical deficiencies, poor ergonomic design, or lack of user friendliness, (2) people-determined theory: resistance was caused by factors internal to users, such as the accountants’ cognitive styles or personality traits that were incompatible with using the system, and (3) interaction theory: resistance was not caused not by factors intrinsic to the system or the people, but by the interaction between the two set of factors. Specifically, interaction theory suggested that the FIS engendered a redistribution of intra-organizational power, and accountants who lost organizational status, relevance, or power as a result of FIS implementation resisted the system while those gaining power favored it.

In order to test the three theories, Markus predicted alternative outcomes expected from each theoretical explanation and analyzed the extent to which those predictions matched with her observations at GTC. For instance, the system-determined theory suggested that since user resistance was caused by an inadequate system, fixing the technical problems of the system would eliminate resistance. The computer running the FIS system was subsequently upgraded with a more powerful operating system, online processing (from initial batch processing, which delayed immediate processing of accounting information), and a simplified software for new account creation by managers. One year after these changes were made, the resistant users were still resisting the system and felt that it should be replaced. Hence, the system-determined theory was rejected.

The people-determined theory predicted that replacing individual resistors or co-opting them with less resistant users would reduce their resistance toward the FIS. Subsequently, GTC started a job rotation and mobility policy, moving accountants in and out of the resistant divisions, but resistance not only persisted, but in some cases increased! In one specific instance, one accountant, who was one of the system’s designers and advocates when he worked for corporate accounting, started resisting the system after he was moved to the divisional controller’s office. Failure to realize the predictions of the people-determined theory led to the rejection of this theory.

Finally, the interaction theory predicted that neither changing the system or the people (i.e., user education or job rotation policies) will reduce resistance as long as the power imbalance and redistribution from the pre-implementation phase were not addressed. Before FIS implementation, divisional accountants at GTC felt that they owned all accounting data related to their divisional operations. They maintained this data in thick, manual ledger books, controlled others’ access to the data, and could reconcile unusual accounting events before releasing those reports. Corporate accountants relied heavily on divisional accountants for access to the divisional data for corporate reporting and consolidation. Because the FIS system automatically collected all data at source and consolidated them into a single corporate database, it obviated the need for divisional accountants, loosened their control and autonomy over their division’s accounting data, and making their job somewhat irrelevant. Corporate accountants could now query the database and access divisional data directly without going through the divisional accountants, analyze and compare the performance of individual divisions, and report unusual patterns and activities to the executive committee, resulting in further erosion of the divisions’ power. Though Markus did not empirically test this theory, her observations about the redistribution of organizational power, coupled with the rejection of the two alternative theories, led to the justification of interaction theory.

Comparisons with Traditional Research

Positivist case research, aimed at hypotheses testing, is often criticized by natural science researchers as lacking in controlled observations, controlled deductions, replicability, and generalizability of findings – the traditional principles of positivist research. However, these criticisms can be overcome through appropriate case research designs. For instance, the problem of controlled observations refers to the difficulty of obtaining experimental or statistical control in case research. However, case researchers can compensate for such lack of controls by employing “natural controls.” This natural control in Markus’ (1983) study was the corporate accountant who was one of the system advocates initially, but started resisting it once he moved to controlling division. In this instance, the change in his behavior may be attributed to his new divisional position. However, such natural controls cannot be anticipated in advance, and case researchers may overlook then unless they are proactively looking for such controls. Incidentally, natural controls are also used in natural science disciplines such as astronomy, geology, and human biology, such as wait for comets to pass close enough to the earth in order to make inferences about comets and their composition.

The problem of controlled deduction refers to the lack of adequate quantitative evidence to support inferences, given the mostly qualitative nature of case research data. Despite the lack of quantitative data for hypotheses testing (e.g., t-tests), controlled deductions can still be obtained in case research by generating behavioral predictions based on theoretical considerations and testing those predictions over time. Markus employed this strategy in her study by generating three alternative theoretical hypotheses for user resistance, and rejecting two of those predictions when they did not match with actual observed behavior. In this case, the hypotheses were tested using logical propositions rather than using mathematical tests, which are just as valid as statistical inferences since mathematics is a subset of logic.

Third, the problem of replicability refers to the difficulty of observing the same phenomenon given the uniqueness and idiosyncrasy of a given case site. However, using Markus’ three theories as an illustration, a different researcher can test the same theories at a different case site, where three different predictions may emerge based on the idiosyncratic nature of the new case site, and the three resulting predictions may be tested accordingly. In other words, it is possible to replicate the inferences of case research, even if the case research site or context may not be replicable.

Fourth, case research tends to examine unique and non-replicable phenomena that may not be generalized to other settings. Generalizability in natural sciences is established through additional studies. Likewise, additional case studies conducted in different contexts with different predictions can establish generalizability of findings if such findings are observed to be consistent across studies.

Lastly, British philosopher Karl Popper described four requirements of scientific theories: (1) theories should be falsifiable, (2) they should be logically consistent, (3) they should have adequate predictive ability, and (4) they should provide better explanation than rival theories. In case research, the first three requirements can be increased by increasing the degrees of freedom of observed findings, such as by increasing the number of case sites, the number of alternative predictions, and the number of levels of analysis examined. This was accomplished in Markus’ study by examining the behavior of multiple groups (divisional accountants and corporate accountants) and providing multiple (three) rival explanations.

Popper’s fourth condition was accomplished in this study when one hypothesis was found to match observed evidence better than the two rival hypotheses.

[8] Benbasat, I., Goldstein, D. K., and Mead, M. (1987). “The Case Research Strategy in Studies of Information Systems,” MIS Quarterly (11:3), 369-386.

[9] Yin, R. K. (2002), Case Study Research: Design and Methods . Thousand Oaks, CA: Sage Publications.

[10] Eisenhardt, K. M. (1989). “Building Theories from Case Research,” Academy of Management Review

(14:4), 532-550.

[11] Eisenhardt, K. M. (1989). “Making Fast Strategic Decisions in High-Velocity Environments,” Academy of Management Journal (32:3), 543-576.

[12] Markus, M. L. (1983). “Power, Politics, and MIS Implementation,” Communications of the ACM (26:6), 430-444.

• Social Science Research: Principles, Methods, and Practices. Authored by : Anol Bhattacherjee. Provided by : University of South Florida. Located at : http://scholarcommons.usf.edu/oa_textbooks/3/ . License : CC BY-NC-SA: Attribution-NonCommercial-ShareAlike

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is a Case Study? | Definition, Examples & Methods

What Is a Case Study? | Definition, Examples & Methods

Published on May 8, 2019 by Shona McCombes . Revised on November 20, 2023.

A case study is a detailed study of a specific subject, such as a person, group, place, event, organization, or phenomenon. Case studies are commonly used in social, educational, clinical, and business research.

A case study research design usually involves qualitative methods , but quantitative methods are sometimes also used. Case studies are good for describing , comparing, evaluating and understanding different aspects of a research problem .

Table of contents

When to do a case study, step 1: select a case, step 2: build a theoretical framework, step 3: collect your data, step 4: describe and analyze the case, other interesting articles.

A case study is an appropriate research design when you want to gain concrete, contextual, in-depth knowledge about a specific real-world subject. It allows you to explore the key characteristics, meanings, and implications of the case.

Case studies are often a good choice in a thesis or dissertation . They keep your project focused and manageable when you don’t have the time or resources to do large-scale research.

You might use just one complex case study where you explore a single subject in depth, or conduct multiple case studies to compare and illuminate different aspects of your research problem.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Once you have developed your problem statement and research questions , you should be ready to choose the specific case that you want to focus on. A good case study should have the potential to:

• Provide new or unexpected insights into the subject
• Challenge or complicate existing assumptions and theories
• Propose practical courses of action to resolve a problem
• Open up new directions for future research

TipIf your research is more practical in nature and aims to simultaneously investigate an issue as you solve it, consider conducting action research instead.

Unlike quantitative or experimental research , a strong case study does not require a random or representative sample. In fact, case studies often deliberately focus on unusual, neglected, or outlying cases which may shed new light on the research problem.

Example of an outlying case studyIn the 1960s the town of Roseto, Pennsylvania was discovered to have extremely low rates of heart disease compared to the US average. It became an important case study for understanding previously neglected causes of heart disease.

However, you can also choose a more common or representative case to exemplify a particular category, experience or phenomenon.

Example of a representative case studyIn the 1920s, two sociologists used Muncie, Indiana as a case study of a typical American city that supposedly exemplified the changing culture of the US at the time.

While case studies focus more on concrete details than general theories, they should usually have some connection with theory in the field. This way the case study is not just an isolated description, but is integrated into existing knowledge about the topic. It might aim to:

• Exemplify a theory by showing how it explains the case under investigation
• Expand on a theory by uncovering new concepts and ideas that need to be incorporated
• Challenge a theory by exploring an outlier case that doesn’t fit with established assumptions

To ensure that your analysis of the case has a solid academic grounding, you should conduct a literature review of sources related to the topic and develop a theoretical framework . This means identifying key concepts and theories to guide your analysis and interpretation.

There are many different research methods you can use to collect data on your subject. Case studies tend to focus on qualitative data using methods such as interviews , observations , and analysis of primary and secondary sources (e.g., newspaper articles, photographs, official records). Sometimes a case study will also collect quantitative data.

Example of a mixed methods case studyFor a case study of a wind farm development in a rural area, you could collect quantitative data on employment rates and business revenue, collect qualitative data on local people’s perceptions and experiences, and analyze local and national media coverage of the development.

The aim is to gain as thorough an understanding as possible of the case and its context.

In writing up the case study, you need to bring together all the relevant aspects to give as complete a picture as possible of the subject.

How you report your findings depends on the type of research you are doing. Some case studies are structured like a standard scientific paper or thesis , with separate sections or chapters for the methods , results and discussion .

Others are written in a more narrative style, aiming to explore the case from various angles and analyze its meanings and implications (for example, by using textual analysis or discourse analysis ).

In all cases, though, make sure to give contextual details about the case, connect it back to the literature and theory, and discuss how it fits into wider patterns or debates.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Ecological validity

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 20). What Is a Case Study? | Definition, Examples & Methods. Scribbr. Retrieved August 26, 2024, from https://www.scribbr.com/methodology/case-study/

Is this article helpful?

Shona McCombes

Other students also liked, primary vs. secondary sources | difference & examples, what is a theoretical framework | guide to organizing, what is action research | definition & examples, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals

You are here

• Volume 21, Issue 1
• What is a case study?
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Roberta Heale 1 ,
• Alison Twycross 2
• 1 School of Nursing , Laurentian University , Sudbury , Ontario , Canada
• 2 School of Health and Social Care , London South Bank University , London , UK
• Correspondence to Dr Roberta Heale, School of Nursing, Laurentian University, Sudbury, ON P3E2C6, Canada; rheale{at}laurentian.ca

https://doi.org/10.1136/eb-2017-102845

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

What is it?

Case study is a research methodology, typically seen in social and life sciences. There is no one definition of case study research. 1 However, very simply… ‘a case study can be defined as an intensive study about a person, a group of people or a unit, which is aimed to generalize over several units’. 1 A case study has also been described as an intensive, systematic investigation of a single individual, group, community or some other unit in which the researcher examines in-depth data relating to several variables. 2

Often there are several similar cases to consider such as educational or social service programmes that are delivered from a number of locations. Although similar, they are complex and have unique features. In these circumstances, the evaluation of several, similar cases will provide a better answer to a research question than if only one case is examined, hence the multiple-case study. Stake asserts that the cases are grouped and viewed as one entity, called the quintain . 6  ‘We study what is similar and different about the cases to understand the quintain better’. 6

The steps when using case study methodology are the same as for other types of research. 6 The first step is defining the single case or identifying a group of similar cases that can then be incorporated into a multiple-case study. A search to determine what is known about the case(s) is typically conducted. This may include a review of the literature, grey literature, media, reports and more, which serves to establish a basic understanding of the cases and informs the development of research questions. Data in case studies are often, but not exclusively, qualitative in nature. In multiple-case studies, analysis within cases and across cases is conducted. Themes arise from the analyses and assertions about the cases as a whole, or the quintain, emerge. 6

Benefits and limitations of case studies

If a researcher wants to study a specific phenomenon arising from a particular entity, then a single-case study is warranted and will allow for a in-depth understanding of the single phenomenon and, as discussed above, would involve collecting several different types of data. This is illustrated in example 1 below.

Using a multiple-case research study allows for a more in-depth understanding of the cases as a unit, through comparison of similarities and differences of the individual cases embedded within the quintain. Evidence arising from multiple-case studies is often stronger and more reliable than from single-case research. Multiple-case studies allow for more comprehensive exploration of research questions and theory development. 6

Despite the advantages of case studies, there are limitations. The sheer volume of data is difficult to organise and data analysis and integration strategies need to be carefully thought through. There is also sometimes a temptation to veer away from the research focus. 2 Reporting of findings from multiple-case research studies is also challenging at times, 1 particularly in relation to the word limits for some journal papers.

Examples of case studies

Example 1: nurses’ paediatric pain management practices.

One of the authors of this paper (AT) has used a case study approach to explore nurses’ paediatric pain management practices. This involved collecting several datasets:

Observational data to gain a picture about actual pain management practices.

Questionnaire data about nurses’ knowledge about paediatric pain management practices and how well they felt they managed pain in children.

Questionnaire data about how critical nurses perceived pain management tasks to be.

These datasets were analysed separately and then compared 7–9 and demonstrated that nurses’ level of theoretical did not impact on the quality of their pain management practices. 7 Nor did individual nurse’s perceptions of how critical a task was effect the likelihood of them carrying out this task in practice. 8 There was also a difference in self-reported and observed practices 9 ; actual (observed) practices did not confirm to best practice guidelines, whereas self-reported practices tended to.

Example 2: quality of care for complex patients at Nurse Practitioner-Led Clinics (NPLCs)

The other author of this paper (RH) has conducted a multiple-case study to determine the quality of care for patients with complex clinical presentations in NPLCs in Ontario, Canada. 10 Five NPLCs served as individual cases that, together, represented the quatrain. Three types of data were collected including:

Review of documentation related to the NPLC model (media, annual reports, research articles, grey literature and regulatory legislation).

Interviews with nurse practitioners (NPs) practising at the five NPLCs to determine their perceptions of the impact of the NPLC model on the quality of care provided to patients with multimorbidity.

Chart audits conducted at the five NPLCs to determine the extent to which evidence-based guidelines were followed for patients with diabetes and at least one other chronic condition.

The three sources of data collected from the five NPLCs were analysed and themes arose related to the quality of care for complex patients at NPLCs. The multiple-case study confirmed that nurse practitioners are the primary care providers at the NPLCs, and this positively impacts the quality of care for patients with multimorbidity. Healthcare policy, such as lack of an increase in salary for NPs for 10 years, has resulted in issues in recruitment and retention of NPs at NPLCs. This, along with insufficient resources in the communities where NPLCs are located and high patient vulnerability at NPLCs, have a negative impact on the quality of care. 10

These examples illustrate how collecting data about a single case or multiple cases helps us to better understand the phenomenon in question. Case study methodology serves to provide a framework for evaluation and analysis of complex issues. It shines a light on the holistic nature of nursing practice and offers a perspective that informs improved patient care.

• Gustafsson J
• Calanzaro M
• Sandelowski M

Competing interests None declared.

Provenance and peer review Commissioned; internally peer reviewed.

Read the full text or download the PDF:

2.2 Approaches to Research

Learning objectives.

By the end of this section, you will be able to:

• Describe the different research methods used by psychologists
• Discuss the strengths and weaknesses of case studies, naturalistic observation, surveys, and archival research
• Compare longitudinal and cross-sectional approaches to research
• Compare and contrast correlation and causation

There are many research methods available to psychologists in their efforts to understand, describe, and explain behavior and the cognitive and biological processes that underlie it. Some methods rely on observational techniques. Other approaches involve interactions between the researcher and the individuals who are being studied—ranging from a series of simple questions to extensive, in-depth interviews—to well-controlled experiments.

Each of these research methods has unique strengths and weaknesses, and each method may only be appropriate for certain types of research questions. For example, studies that rely primarily on observation produce incredible amounts of information, but the ability to apply this information to the larger population is somewhat limited because of small sample sizes. Survey research, on the other hand, allows researchers to easily collect data from relatively large samples. While this allows for results to be generalized to the larger population more easily, the information that can be collected on any given survey is somewhat limited and subject to problems associated with any type of self-reported data. Some researchers conduct archival research by using existing records. While this can be a fairly inexpensive way to collect data that can provide insight into a number of research questions, researchers using this approach have no control on how or what kind of data was collected. All of the methods described thus far are correlational in nature. This means that researchers can speak to important relationships that might exist between two or more variables of interest. However, correlational data cannot be used to make claims about cause-and-effect relationships.

Correlational research can find a relationship between two variables, but the only way a researcher can claim that the relationship between the variables is cause and effect is to perform an experiment. In experimental research, which will be discussed later in this chapter, there is a tremendous amount of control over variables of interest. While this is a powerful approach, experiments are often conducted in artificial settings. This calls into question the validity of experimental findings with regard to how they would apply in real-world settings. In addition, many of the questions that psychologists would like to answer cannot be pursued through experimental research because of ethical concerns.

Clinical or Case Studies

In 2011, the New York Times published a feature story on Krista and Tatiana Hogan, Canadian twin girls. These particular twins are unique because Krista and Tatiana are conjoined twins, connected at the head. There is evidence that the two girls are connected in a part of the brain called the thalamus, which is a major sensory relay center. Most incoming sensory information is sent through the thalamus before reaching higher regions of the cerebral cortex for processing.

Link to Learning

Watch this CBC video about Krista's and Tatiana's lives to learn more.

The implications of this potential connection mean that it might be possible for one twin to experience the sensations of the other twin. For instance, if Krista is watching a particularly funny television program, Tatiana might smile or laugh even if she is not watching the program. This particular possibility has piqued the interest of many neuroscientists who seek to understand how the brain uses sensory information.

These twins represent an enormous resource in the study of the brain, and since their condition is very rare, it is likely that as long as their family agrees, scientists will follow these girls very closely throughout their lives to gain as much information as possible (Dominus, 2011).

Over time, it has become clear that while Krista and Tatiana share some sensory experiences and motor control, they remain two distinct individuals, which provides invaluable insight for researchers interested in the mind and the brain (Egnor, 2017).

In observational research, scientists are conducting a clinical or case study when they focus on one person or just a few individuals. Indeed, some scientists spend their entire careers studying just 10–20 individuals. Why would they do this? Obviously, when they focus their attention on a very small number of people, they can gain a precious amount of insight into those cases. The richness of information that is collected in clinical or case studies is unmatched by any other single research method. This allows the researcher to have a very deep understanding of the individuals and the particular phenomenon being studied.

If clinical or case studies provide so much information, why are they not more frequent among researchers? As it turns out, the major benefit of this particular approach is also a weakness. As mentioned earlier, this approach is often used when studying individuals who are interesting to researchers because they have a rare characteristic. Therefore, the individuals who serve as the focus of case studies are not like most other people. If scientists ultimately want to explain all behavior, focusing attention on such a special group of people can make it difficult to generalize any observations to the larger population as a whole. Generalizing refers to the ability to apply the findings of a particular research project to larger segments of society. Again, case studies provide enormous amounts of information, but since the cases are so specific, the potential to apply what’s learned to the average person may be very limited.

Naturalistic Observation

If you want to understand how behavior occurs, one of the best ways to gain information is to simply observe the behavior in its natural context. However, people might change their behavior in unexpected ways if they know they are being observed. How do researchers obtain accurate information when people tend to hide their natural behavior? As an example, imagine that your professor asks everyone in your class to raise their hand if they always wash their hands after using the restroom. Chances are that almost everyone in the classroom will raise their hand, but do you think hand washing after every trip to the restroom is really that universal?

This is very similar to the phenomenon mentioned earlier in this chapter: many individuals do not feel comfortable answering a question honestly. But if we are committed to finding out the facts about hand washing, we have other options available to us.

Suppose we send a classmate into the restroom to actually watch whether everyone washes their hands after using the restroom. Will our observer blend into the restroom environment by wearing a white lab coat, sitting with a clipboard, and staring at the sinks? We want our researcher to be inconspicuous—perhaps standing at one of the sinks pretending to put in contact lenses while secretly recording the relevant information. This type of observational study is called naturalistic observation : observing behavior in its natural setting. To better understand peer exclusion, Suzanne Fanger collaborated with colleagues at the University of Texas to observe the behavior of preschool children on a playground. How did the observers remain inconspicuous over the duration of the study? They equipped a few of the children with wireless microphones (which the children quickly forgot about) and observed while taking notes from a distance. Also, the children in that particular preschool (a “laboratory preschool”) were accustomed to having observers on the playground (Fanger, Frankel, & Hazen, 2012).

It is critical that the observer be as unobtrusive and as inconspicuous as possible: when people know they are being watched, they are less likely to behave naturally. If you have any doubt about this, ask yourself how your driving behavior might differ in two situations: In the first situation, you are driving down a deserted highway during the middle of the day; in the second situation, you are being followed by a police car down the same deserted highway ( Figure 2.7 ).

It should be pointed out that naturalistic observation is not limited to research involving humans. Indeed, some of the best-known examples of naturalistic observation involve researchers going into the field to observe various kinds of animals in their own environments. As with human studies, the researchers maintain their distance and avoid interfering with the animal subjects so as not to influence their natural behaviors. Scientists have used this technique to study social hierarchies and interactions among animals ranging from ground squirrels to gorillas. The information provided by these studies is invaluable in understanding how those animals organize socially and communicate with one another. The anthropologist Jane Goodall , for example, spent nearly five decades observing the behavior of chimpanzees in Africa ( Figure 2.8 ). As an illustration of the types of concerns that a researcher might encounter in naturalistic observation, some scientists criticized Goodall for giving the chimps names instead of referring to them by numbers—using names was thought to undermine the emotional detachment required for the objectivity of the study (McKie, 2010).

The greatest benefit of naturalistic observation is the validity , or accuracy, of information collected unobtrusively in a natural setting. Having individuals behave as they normally would in a given situation means that we have a higher degree of ecological validity, or realism, than we might achieve with other research approaches. Therefore, our ability to generalize the findings of the research to real-world situations is enhanced. If done correctly, we need not worry about people or animals modifying their behavior simply because they are being observed. Sometimes, people may assume that reality programs give us a glimpse into authentic human behavior. However, the principle of inconspicuous observation is violated as reality stars are followed by camera crews and are interviewed on camera for personal confessionals. Given that environment, we must doubt how natural and realistic their behaviors are.

The major downside of naturalistic observation is that they are often difficult to set up and control. In our restroom study, what if you stood in the restroom all day prepared to record people’s hand washing behavior and no one came in? Or, what if you have been closely observing a troop of gorillas for weeks only to find that they migrated to a new place while you were sleeping in your tent? The benefit of realistic data comes at a cost. As a researcher you have no control of when (or if) you have behavior to observe. In addition, this type of observational research often requires significant investments of time, money, and a good dose of luck.

Sometimes studies involve structured observation. In these cases, people are observed while engaging in set, specific tasks. An excellent example of structured observation comes from Strange Situation by Mary Ainsworth (you will read more about this in the chapter on lifespan development). The Strange Situation is a procedure used to evaluate attachment styles that exist between an infant and caregiver. In this scenario, caregivers bring their infants into a room filled with toys. The Strange Situation involves a number of phases, including a stranger coming into the room, the caregiver leaving the room, and the caregiver’s return to the room. The infant’s behavior is closely monitored at each phase, but it is the behavior of the infant upon being reunited with the caregiver that is most telling in terms of characterizing the infant’s attachment style with the caregiver.

Another potential problem in observational research is observer bias . Generally, people who act as observers are closely involved in the research project and may unconsciously skew their observations to fit their research goals or expectations. To protect against this type of bias, researchers should have clear criteria established for the types of behaviors recorded and how those behaviors should be classified. In addition, researchers often compare observations of the same event by multiple observers, in order to test inter-rater reliability : a measure of reliability that assesses the consistency of observations by different observers.

Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally ( Figure 2.9 ). Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.

Surveys allow researchers to gather data from larger samples than may be afforded by other research methods . A sample is a subset of individuals selected from a population , which is the overall group of individuals that the researchers are interested in. Researchers study the sample and seek to generalize their findings to the population. Generally, researchers will begin this process by calculating various measures of central tendency from the data they have collected. These measures provide an overall summary of what a typical response looks like. There are three measures of central tendency: mode, median, and mean. The mode is the most frequently occurring response, the median lies at the middle of a given data set, and the mean is the arithmetic average of all data points. Means tend to be most useful in conducting additional analyses like those described below; however, means are very sensitive to the effects of outliers, and so one must be aware of those effects when making assessments of what measures of central tendency tell us about a data set in question.

There is both strength and weakness of the survey in comparison to case studies. By using surveys, we can collect information from a larger sample of people. A larger sample is better able to reflect the actual diversity of the population, thus allowing better generalizability. Therefore, if our sample is sufficiently large and diverse, we can assume that the data we collect from the survey can be generalized to the larger population with more certainty than the information collected through a case study. However, given the greater number of people involved, we are not able to collect the same depth of information on each person that would be collected in a case study.

Another potential weakness of surveys is something we touched on earlier in this chapter: People don't always give accurate responses. They may lie, misremember, or answer questions in a way that they think makes them look good. For example, people may report drinking less alcohol than is actually the case.

Any number of research questions can be answered through the use of surveys. One real-world example is the research conducted by Jenkins, Ruppel, Kizer, Yehl, and Griffin (2012) about the backlash against the US Arab-American community following the terrorist attacks of September 11, 2001. Jenkins and colleagues wanted to determine to what extent these negative attitudes toward Arab-Americans still existed nearly a decade after the attacks occurred. In one study, 140 research participants filled out a survey with 10 questions, including questions asking directly about the participant’s overt prejudicial attitudes toward people of various ethnicities. The survey also asked indirect questions about how likely the participant would be to interact with a person of a given ethnicity in a variety of settings (such as, “How likely do you think it is that you would introduce yourself to a person of Arab-American descent?”). The results of the research suggested that participants were unwilling to report prejudicial attitudes toward any ethnic group. However, there were significant differences between their pattern of responses to questions about social interaction with Arab-Americans compared to other ethnic groups: they indicated less willingness for social interaction with Arab-Americans compared to the other ethnic groups. This suggested that the participants harbored subtle forms of prejudice against Arab-Americans, despite their assertions that this was not the case (Jenkins et al., 2012).

Archival Research

Some researchers gain access to large amounts of data without interacting with a single research participant. Instead, they use existing records to answer various research questions. This type of research approach is known as archival research . Archival research relies on looking at past records or data sets to look for interesting patterns or relationships.

For example, a researcher might access the academic records of all individuals who enrolled in college within the past ten years and calculate how long it took them to complete their degrees, as well as course loads, grades, and extracurricular involvement. Archival research could provide important information about who is most likely to complete their education, and it could help identify important risk factors for struggling students ( Figure 2.10 ).

In comparing archival research to other research methods, there are several important distinctions. For one, the researcher employing archival research never directly interacts with research participants. Therefore, the investment of time and money to collect data is considerably less with archival research. Additionally, researchers have no control over what information was originally collected. Therefore, research questions have to be tailored so they can be answered within the structure of the existing data sets. There is also no guarantee of consistency between the records from one source to another, which might make comparing and contrasting different data sets problematic.

Longitudinal and Cross-Sectional Research

Sometimes we want to see how people change over time, as in studies of human development and lifespan. When we test the same group of individuals repeatedly over an extended period of time, we are conducting longitudinal research. Longitudinal research is a research design in which data-gathering is administered repeatedly over an extended period of time. For example, we may survey a group of individuals about their dietary habits at age 20, retest them a decade later at age 30, and then again at age 40.

Another approach is cross-sectional research. In cross-sectional research , a researcher compares multiple segments of the population at the same time. Using the dietary habits example above, the researcher might directly compare different groups of people by age. Instead of studying a group of people for 20 years to see how their dietary habits changed from decade to decade, the researcher would study a group of 20-year-old individuals and compare them to a group of 30-year-old individuals and a group of 40-year-old individuals. While cross-sectional research requires a shorter-term investment, it is also limited by differences that exist between the different generations (or cohorts) that have nothing to do with age per se, but rather reflect the social and cultural experiences of different generations of individuals that make them different from one another.

To illustrate this concept, consider the following survey findings. In recent years there has been significant growth in the popular support of same-sex marriage. Many studies on this topic break down survey participants into different age groups. In general, younger people are more supportive of same-sex marriage than are those who are older (Jones, 2013). Does this mean that as we age we become less open to the idea of same-sex marriage, or does this mean that older individuals have different perspectives because of the social climates in which they grew up? Longitudinal research is a powerful approach because the same individuals are involved in the research project over time, which means that the researchers need to be less concerned with differences among cohorts affecting the results of their study.

Often longitudinal studies are employed when researching various diseases in an effort to understand particular risk factors. Such studies often involve tens of thousands of individuals who are followed for several decades. Given the enormous number of people involved in these studies, researchers can feel confident that their findings can be generalized to the larger population. The Cancer Prevention Study-3 (CPS-3) is one of a series of longitudinal studies sponsored by the American Cancer Society aimed at determining predictive risk factors associated with cancer. When participants enter the study, they complete a survey about their lives and family histories, providing information on factors that might cause or prevent the development of cancer. Then every few years the participants receive additional surveys to complete. In the end, hundreds of thousands of participants will be tracked over 20 years to determine which of them develop cancer and which do not.

Clearly, this type of research is important and potentially very informative. For instance, earlier longitudinal studies sponsored by the American Cancer Society provided some of the first scientific demonstrations of the now well-established links between increased rates of cancer and smoking (American Cancer Society, n.d.) ( Figure 2.11 ).

As with any research strategy, longitudinal research is not without limitations. For one, these studies require an incredible time investment by the researcher and research participants. Given that some longitudinal studies take years, if not decades, to complete, the results will not be known for a considerable period of time. In addition to the time demands, these studies also require a substantial financial investment. Many researchers are unable to commit the resources necessary to see a longitudinal project through to the end.

Research participants must also be willing to continue their participation for an extended period of time, and this can be problematic. People move, get married and take new names, get ill, and eventually die. Even without significant life changes, some people may simply choose to discontinue their participation in the project. As a result, the attrition rates, or reduction in the number of research participants due to dropouts, in longitudinal studies are quite high and increase over the course of a project. For this reason, researchers using this approach typically recruit many participants fully expecting that a substantial number will drop out before the end. As the study progresses, they continually check whether the sample still represents the larger population, and make adjustments as necessary.

This book may not be used in the training of large language models or otherwise be ingested into large language models or generative AI offerings without OpenStax's permission.

Want to cite, share, or modify this book? This book uses the Creative Commons Attribution License and you must attribute OpenStax.

Access for free at https://openstax.org/books/psychology-2e/pages/1-introduction

• Authors: Rose M. Spielman, William J. Jenkins, Marilyn D. Lovett
• Publisher/website: OpenStax
• Book title: Psychology 2e
• Publication date: Apr 22, 2020
• Location: Houston, Texas
• Book URL: https://openstax.org/books/psychology-2e/pages/1-introduction
• Section URL: https://openstax.org/books/psychology-2e/pages/2-2-approaches-to-research

© Jun 26, 2024 OpenStax. Textbook content produced by OpenStax is licensed under a Creative Commons Attribution License . The OpenStax name, OpenStax logo, OpenStax book covers, OpenStax CNX name, and OpenStax CNX logo are not subject to the Creative Commons license and may not be reproduced without the prior and express written consent of Rice University.

• Open access
• Published: 10 November 2020

Case study research for better evaluations of complex interventions: rationale and challenges

• Sara Paparini   ORCID: orcid.org/0000-0002-1909-2481 1 ,
• Judith Green 2 ,
• Chrysanthi Papoutsi 1 ,
• Jamie Murdoch 3 ,
• Mark Petticrew 4 ,
• Trish Greenhalgh 1 ,
• Benjamin Hanckel 5 &
• Sara Shaw 1  

BMC Medicine volume  18 , Article number:  301 ( 2020 ) Cite this article

19k Accesses

44 Citations

35 Altmetric

Metrics details

The need for better methods for evaluation in health research has been widely recognised. The ‘complexity turn’ has drawn attention to the limitations of relying on causal inference from randomised controlled trials alone for understanding whether, and under which conditions, interventions in complex systems improve health services or the public health, and what mechanisms might link interventions and outcomes. We argue that case study research—currently denigrated as poor evidence—is an under-utilised resource for not only providing evidence about context and transferability, but also for helping strengthen causal inferences when pathways between intervention and effects are likely to be non-linear.

Case study research, as an overall approach, is based on in-depth explorations of complex phenomena in their natural, or real-life, settings. Empirical case studies typically enable dynamic understanding of complex challenges and provide evidence about causal mechanisms and the necessary and sufficient conditions (contexts) for intervention implementation and effects. This is essential evidence not just for researchers concerned about internal and external validity, but also research users in policy and practice who need to know what the likely effects of complex programmes or interventions will be in their settings. The health sciences have much to learn from scholarship on case study methodology in the social sciences. However, there are multiple challenges in fully exploiting the potential learning from case study research. First are misconceptions that case study research can only provide exploratory or descriptive evidence. Second, there is little consensus about what a case study is, and considerable diversity in how empirical case studies are conducted and reported. Finally, as case study researchers typically (and appropriately) focus on thick description (that captures contextual detail), it can be challenging to identify the key messages related to intervention evaluation from case study reports.

Whilst the diversity of published case studies in health services and public health research is rich and productive, we recommend further clarity and specific methodological guidance for those reporting case study research for evaluation audiences.

Peer Review reports

The need for methodological development to address the most urgent challenges in health research has been well-documented. Many of the most pressing questions for public health research, where the focus is on system-level determinants [ 1 , 2 ], and for health services research, where provisions typically vary across sites and are provided through interlocking networks of services [ 3 ], require methodological approaches that can attend to complexity. The need for methodological advance has arisen, in part, as a result of the diminishing returns from randomised controlled trials (RCTs) where they have been used to answer questions about the effects of interventions in complex systems [ 4 , 5 , 6 ]. In conditions of complexity, there is limited value in maintaining the current orientation to experimental trial designs in the health sciences as providing ‘gold standard’ evidence of effect.

There are increasing calls for methodological pluralism [ 7 , 8 ], with the recognition that complex intervention and context are not easily or usefully separated (as is often the situation when using trial design), and that system interruptions may have effects that are not reducible to linear causal pathways between intervention and outcome. These calls are reflected in a shifting and contested discourse of trial design, seen with the emergence of realist [ 9 ], adaptive and hybrid (types 1, 2 and 3) [ 10 , 11 ] trials that blend studies of effectiveness with a close consideration of the contexts of implementation. Similarly, process evaluation has now become a core component of complex healthcare intervention trials, reflected in MRC guidance on how to explore implementation, causal mechanisms and context [ 12 ].

Evidence about the context of an intervention is crucial for questions of external validity. As Woolcock [ 4 ] notes, even if RCT designs are accepted as robust for maximising internal validity, questions of transferability (how well the intervention works in different contexts) and generalisability (how well the intervention can be scaled up) remain unanswered [ 5 , 13 ]. For research evidence to have impact on policy and systems organisation, and thus to improve population and patient health, there is an urgent need for better methods for strengthening external validity, including a better understanding of the relationship between intervention and context [ 14 ].

Policymakers, healthcare commissioners and other research users require credible evidence of relevance to their settings and populations [ 15 ], to perform what Rosengarten and Savransky [ 16 ] call ‘careful abstraction’ to the locales that matter for them. They also require robust evidence for understanding complex causal pathways. Case study research, currently under-utilised in public health and health services evaluation, can offer considerable potential for strengthening faith in both external and internal validity. For example, in an empirical case study of how the policy of free bus travel had specific health effects in London, UK, a quasi-experimental evaluation (led by JG) identified how important aspects of context (a good public transport system) and intervention (that it was universal) were necessary conditions for the observed effects, thus providing useful, actionable evidence for decision-makers in other contexts [ 17 ].

The overall approach of case study research is based on the in-depth exploration of complex phenomena in their natural, or ‘real-life’, settings. Empirical case studies typically enable dynamic understanding of complex challenges rather than restricting the focus on narrow problem delineations and simple fixes. Case study research is a diverse and somewhat contested field, with multiple definitions and perspectives grounded in different ways of viewing the world, and involving different combinations of methods. In this paper, we raise awareness of such plurality and highlight the contribution that case study research can make to the evaluation of complex system-level interventions. We review some of the challenges in exploiting the current evidence base from empirical case studies and conclude by recommending that further guidance and minimum reporting criteria for evaluation using case studies, appropriate for audiences in the health sciences, can enhance the take-up of evidence from case study research.

Case study research offers evidence about context, causal inference in complex systems and implementation

Well-conducted and described empirical case studies provide evidence on context, complexity and mechanisms for understanding how, where and why interventions have their observed effects. Recognition of the importance of context for understanding the relationships between interventions and outcomes is hardly new. In 1943, Canguilhem berated an over-reliance on experimental designs for determining universal physiological laws: ‘As if one could determine a phenomenon’s essence apart from its conditions! As if conditions were a mask or frame which changed neither the face nor the picture!’ ([ 18 ] p126). More recently, a concern with context has been expressed in health systems and public health research as part of what has been called the ‘complexity turn’ [ 1 ]: a recognition that many of the most enduring challenges for developing an evidence base require a consideration of system-level effects [ 1 ] and the conceptualisation of interventions as interruptions in systems [ 19 ].

The case study approach is widely recognised as offering an invaluable resource for understanding the dynamic and evolving influence of context on complex, system-level interventions [ 20 , 21 , 22 , 23 ]. Empirically, case studies can directly inform assessments of where, when, how and for whom interventions might be successfully implemented, by helping to specify the necessary and sufficient conditions under which interventions might have effects and to consolidate learning on how interdependencies, emergence and unpredictability can be managed to achieve and sustain desired effects. Case study research has the potential to address four objectives for improving research and reporting of context recently set out by guidance on taking account of context in population health research [ 24 ], that is to (1) improve the appropriateness of intervention development for specific contexts, (2) improve understanding of ‘how’ interventions work, (3) better understand how and why impacts vary across contexts and (4) ensure reports of intervention studies are most useful for decision-makers and researchers.

However, evaluations of complex healthcare interventions have arguably not exploited the full potential of case study research and can learn much from other disciplines. For evaluative research, exploratory case studies have had a traditional role of providing data on ‘process’, or initial ‘hypothesis-generating’ scoping, but might also have an increasing salience for explanatory aims. Across the social and political sciences, different kinds of case studies are undertaken to meet diverse aims (description, exploration or explanation) and across different scales (from small N qualitative studies that aim to elucidate processes, or provide thick description, to more systematic techniques designed for medium-to-large N cases).

Case studies with explanatory aims vary in terms of their positioning within mixed-methods projects, with designs including (but not restricted to) (1) single N of 1 studies of interventions in specific contexts, where the overall design is a case study that may incorporate one or more (randomised or not) comparisons over time and between variables within the case; (2) a series of cases conducted or synthesised to provide explanation from variations between cases; and (3) case studies of particular settings within RCT or quasi-experimental designs to explore variation in effects or implementation.

Detailed qualitative research (typically done as ‘case studies’ within process evaluations) provides evidence for the plausibility of mechanisms [ 25 ], offering theoretical generalisations for how interventions may function under different conditions. Although RCT designs reduce many threats to internal validity, the mechanisms of effect remain opaque, particularly when the causal pathways between ‘intervention’ and ‘effect’ are long and potentially non-linear: case study research has a more fundamental role here, in providing detailed observational evidence for causal claims [ 26 ] as well as producing a rich, nuanced picture of tensions and multiple perspectives [ 8 ].

Longitudinal or cross-case analysis may be best suited for evidence generation in system-level evaluative research. Turner [ 27 ], for instance, reflecting on the complex processes in major system change, has argued for the need for methods that integrate learning across cases, to develop theoretical knowledge that would enable inferences beyond the single case, and to develop generalisable theory about organisational and structural change in health systems. Qualitative Comparative Analysis (QCA) [ 28 ] is one such formal method for deriving causal claims, using set theory mathematics to integrate data from empirical case studies to answer questions about the configurations of causal pathways linking conditions to outcomes [ 29 , 30 ].

Nonetheless, the single N case study, too, provides opportunities for theoretical development [ 31 ], and theoretical generalisation or analytical refinement [ 32 ]. How ‘the case’ and ‘context’ are conceptualised is crucial here. Findings from the single case may seem to be confined to its intrinsic particularities in a specific and distinct context [ 33 ]. However, if such context is viewed as exemplifying wider social and political forces, the single case can be ‘telling’, rather than ‘typical’, and offer insight into a wider issue [ 34 ]. Internal comparisons within the case can offer rich possibilities for logical inferences about causation [ 17 ]. Further, case studies of any size can be used for theory testing through refutation [ 22 ]. The potential lies, then, in utilising the strengths and plurality of case study to support theory-driven research within different methodological paradigms.

Evaluation research in health has much to learn from a range of social sciences where case study methodology has been used to develop various kinds of causal inference. For instance, Gerring [ 35 ] expands on the within-case variations utilised to make causal claims. For Gerring [ 35 ], case studies come into their own with regard to invariant or strong causal claims (such as X is a necessary and/or sufficient condition for Y) rather than for probabilistic causal claims. For the latter (where experimental methods might have an advantage in estimating effect sizes), case studies offer evidence on mechanisms: from observations of X affecting Y, from process tracing or from pattern matching. Case studies also support the study of emergent causation, that is, the multiple interacting properties that account for particular and unexpected outcomes in complex systems, such as in healthcare [ 8 ].

Finally, efficacy (or beliefs about efficacy) is not the only contributor to intervention uptake, with a range of organisational and policy contingencies affecting whether an intervention is likely to be rolled out in practice. Case study research is, therefore, invaluable for learning about contextual contingencies and identifying the conditions necessary for interventions to become normalised (i.e. implemented routinely) in practice [ 36 ].

The challenges in exploiting evidence from case study research

At present, there are significant challenges in exploiting the benefits of case study research in evaluative health research, which relate to status, definition and reporting. Case study research has been marginalised at the bottom of an evidence hierarchy, seen to offer little by way of explanatory power, if nonetheless useful for adding descriptive data on process or providing useful illustrations for policymakers [ 37 ]. This is an opportune moment to revisit this low status. As health researchers are increasingly charged with evaluating ‘natural experiments’—the use of face masks in the response to the COVID-19 pandemic being a recent example [ 38 ]—rather than interventions that take place in settings that can be controlled, research approaches using methods to strengthen causal inference that does not require randomisation become more relevant.

A second challenge for improving the use of case study evidence in evaluative health research is that, as we have seen, what is meant by ‘case study’ varies widely, not only across but also within disciplines. There is indeed little consensus amongst methodologists as to how to define ‘a case study’. Definitions focus, variously, on small sample size or lack of control over the intervention (e.g. [ 39 ] p194), on in-depth study and context [ 40 , 41 ], on the logic of inference used [ 35 ] or on distinct research strategies which incorporate a number of methods to address questions of ‘how’ and ‘why’ [ 42 ]. Moreover, definitions developed for specific disciplines do not capture the range of ways in which case study research is carried out across disciplines. Multiple definitions of case study reflect the richness and diversity of the approach. However, evidence suggests that a lack of consensus across methodologists results in some of the limitations of published reports of empirical case studies [ 43 , 44 ]. Hyett and colleagues [ 43 ], for instance, reviewing reports in qualitative journals, found little match between methodological definitions of case study research and how authors used the term.

This raises the third challenge we identify that case study reports are typically not written in ways that are accessible or useful for the evaluation research community and policymakers. Case studies may not appear in journals widely read by those in the health sciences, either because space constraints preclude the reporting of rich, thick descriptions, or because of the reported lack of willingness of some biomedical journals to publish research that uses qualitative methods [ 45 ], signalling the persistence of the aforementioned evidence hierarchy. Where they do, however, the term ‘case study’ is used to indicate, interchangeably, a qualitative study, an N of 1 sample, or a multi-method, in-depth analysis of one example from a population of phenomena. Definitions of what constitutes the ‘case’ are frequently lacking and appear to be used as a synonym for the settings in which the research is conducted. Despite offering insights for evaluation, the primary aims may not have been evaluative, so the implications may not be explicitly drawn out. Indeed, some case study reports might properly be aiming for thick description without necessarily seeking to inform about context or causality.

Acknowledging plurality and developing guidance

We recognise that definitional and methodological plurality is not only inevitable, but also a necessary and creative reflection of the very different epistemological and disciplinary origins of health researchers, and the aims they have in doing and reporting case study research. Indeed, to provide some clarity, Thomas [ 46 ] has suggested a typology of subject/purpose/approach/process for classifying aims (e.g. evaluative or exploratory), sample rationale and selection and methods for data generation of case studies. We also recognise that the diversity of methods used in case study research, and the necessary focus on narrative reporting, does not lend itself to straightforward development of formal quality or reporting criteria.

Existing checklists for reporting case study research from the social sciences—for example Lincoln and Guba’s [ 47 ] and Stake’s [ 33 ]—are primarily orientated to the quality of narrative produced, and the extent to which they encapsulate thick description, rather than the more pragmatic issues of implications for intervention effects. Those designed for clinical settings, such as the CARE (CAse REports) guidelines, provide specific reporting guidelines for medical case reports about single, or small groups of patients [ 48 ], not for case study research.

The Design of Case Study Research in Health Care (DESCARTE) model [ 44 ] suggests a series of questions to be asked of a case study researcher (including clarity about the philosophy underpinning their research), study design (with a focus on case definition) and analysis (to improve process). The model resembles toolkits for enhancing the quality and robustness of qualitative and mixed-methods research reporting, and it is usefully open-ended and non-prescriptive. However, even if it does include some reflections on context, the model does not fully address aspects of context, logic and causal inference that are perhaps most relevant for evaluative research in health.

Hence, for evaluative research where the aim is to report empirical findings in ways that are intended to be pragmatically useful for health policy and practice, this may be an opportune time to consider how to best navigate plurality around what is (minimally) important to report when publishing empirical case studies, especially with regards to the complex relationships between context and interventions, information that case study research is well placed to provide.

The conventional scientific quest for certainty, predictability and linear causality (maximised in RCT designs) has to be augmented by the study of uncertainty, unpredictability and emergent causality [ 8 ] in complex systems. This will require methodological pluralism, and openness to broadening the evidence base to better understand both causality in and the transferability of system change intervention [ 14 , 20 , 23 , 25 ]. Case study research evidence is essential, yet is currently under exploited in the health sciences. If evaluative health research is to move beyond the current impasse on methods for understanding interventions as interruptions in complex systems, we need to consider in more detail how researchers can conduct and report empirical case studies which do aim to elucidate the contextual factors which interact with interventions to produce particular effects. To this end, supported by the UK’s Medical Research Council, we are embracing the challenge to develop guidance for case study researchers studying complex interventions. Following a meta-narrative review of the literature, we are planning a Delphi study to inform guidance that will, at minimum, cover the value of case study research for evaluating the interrelationship between context and complex system-level interventions; for situating and defining ‘the case’, and generalising from case studies; as well as provide specific guidance on conducting, analysing and reporting case study research. Our hope is that such guidance can support researchers evaluating interventions in complex systems to better exploit the diversity and richness of case study research.

Availability of data and materials

Not applicable (article based on existing available academic publications)

Abbreviations

Qualitative comparative analysis

Quasi-experimental design

Randomised controlled trial

Diez Roux AV. Complex systems thinking and current impasses in health disparities research. Am J Public Health. 2011;101(9):1627–34.

Article   Google Scholar  

Ogilvie D, Mitchell R, Mutrie N, M P, Platt S. Evaluating health effects of transport interventions: methodologic case study. Am J Prev Med 2006;31:118–126.

Walshe C. The evaluation of complex interventions in palliative care: an exploration of the potential of case study research strategies. Palliat Med. 2011;25(8):774–81.

Woolcock M. Using case studies to explore the external validity of ‘complex’ development interventions. Evaluation. 2013;19:229–48.

Cartwright N. Are RCTs the gold standard? BioSocieties. 2007;2(1):11–20.

Deaton A, Cartwright N. Understanding and misunderstanding randomized controlled trials. Soc Sci Med. 2018;210:2–21.

Salway S, Green J. Towards a critical complex systems approach to public health. Crit Public Health. 2017;27(5):523–4.

Greenhalgh T, Papoutsi C. Studying complexity in health services research: desperately seeking an overdue paradigm shift. BMC Med. 2018;16(1):95.

Bonell C, Warren E, Fletcher A. Realist trials and the testing of context-mechanism-outcome configurations: a response to Van Belle et al. Trials. 2016;17:478.

Pallmann P, Bedding AW, Choodari-Oskooei B. Adaptive designs in clinical trials: why use them, and how to run and report them. BMC Med. 2018;16:29.

Curran G, Bauer M, Mittman B, Pyne J, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217–26. https://doi.org/10.1097/MLR.0b013e3182408812 .

Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015 [cited 2020 Jun 27];350. Available from: https://www.bmj.com/content/350/bmj.h1258 .

Evans RE, Craig P, Hoddinott P, Littlecott H, Moore L, Murphy S, et al. When and how do ‘effective’ interventions need to be adapted and/or re-evaluated in new contexts? The need for guidance. J Epidemiol Community Health. 2019;73(6):481–2.

Shoveller J. A critical examination of representations of context within research on population health interventions. Crit Public Health. 2016;26(5):487–500.

Treweek S, Zwarenstein M. Making trials matter: pragmatic and explanatory trials and the problem of applicability. Trials. 2009;10(1):37.

Rosengarten M, Savransky M. A careful biomedicine? Generalization and abstraction in RCTs. Crit Public Health. 2019;29(2):181–91.

Green J, Roberts H, Petticrew M, Steinbach R, Goodman A, Jones A, et al. Integrating quasi-experimental and inductive designs in evaluation: a case study of the impact of free bus travel on public health. Evaluation. 2015;21(4):391–406.

Canguilhem G. The normal and the pathological. New York: Zone Books; 1991. (1949).

Google Scholar  

Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J Community Psychol. 2009;43:267–76.

King G, Keohane RO, Verba S. Designing social inquiry: scientific inference in qualitative research: Princeton University Press; 1994.

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629.

Yin R. Enhancing the quality of case studies in health services research. Health Serv Res. 1999;34(5 Pt 2):1209.

CAS   PubMed   PubMed Central   Google Scholar  

Raine R, Fitzpatrick R, Barratt H, Bevan G, Black N, Boaden R, et al. Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. Health Serv Deliv Res. 2016 [cited 2020 Jun 30];4(16). Available from: https://www.journalslibrary.nihr.ac.uk/hsdr/hsdr04160#/abstract .

Craig P, Di Ruggiero E, Frohlich KL, E M, White M, Group CCGA. Taking account of context in population health intervention research: guidance for producers, users and funders of research. NIHR Evaluation, Trials and Studies Coordinating Centre; 2018.

Grant RL, Hood R. Complex systems, explanation and policy: implications of the crisis of replication for public health research. Crit Public Health. 2017;27(5):525–32.

Mahoney J. Strategies of causal inference in small-N analysis. Sociol Methods Res. 2000;4:387–424.

Turner S. Major system change: a management and organisational research perspective. In: Rosalind Raine, Ray Fitzpatrick, Helen Barratt, Gywn Bevan, Nick Black, Ruth Boaden, et al. Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. Health Serv Deliv Res. 2016;4(16) 2016. https://doi.org/10.3310/hsdr04160.

Ragin CC. Using qualitative comparative analysis to study causal complexity. Health Serv Res. 1999;34(5 Pt 2):1225.

Hanckel B, Petticrew M, Thomas J, Green J. Protocol for a systematic review of the use of qualitative comparative analysis for evaluative questions in public health research. Syst Rev. 2019;8(1):252.

Schneider CQ, Wagemann C. Set-theoretic methods for the social sciences: a guide to qualitative comparative analysis: Cambridge University Press; 2012. 369 p.

Flyvbjerg B. Five misunderstandings about case-study research. Qual Inq. 2006;12:219–45.

Tsoukas H. Craving for generality and small-N studies: a Wittgensteinian approach towards the epistemology of the particular in organization and management studies. Sage Handb Organ Res Methods. 2009:285–301.

Stake RE. The art of case study research. London: Sage Publications Ltd; 1995.

Mitchell JC. Typicality and the case study. Ethnographic research: A guide to general conduct. Vol. 238241. 1984.

Gerring J. What is a case study and what is it good for? Am Polit Sci Rev. 2004;98(2):341–54.

May C, Mort M, Williams T, F M, Gask L. Health technology assessment in its local contexts: studies of telehealthcare. Soc Sci Med 2003;57:697–710.

McGill E. Trading quality for relevance: non-health decision-makers’ use of evidence on the social determinants of health. BMJ Open. 2015;5(4):007053.

Greenhalgh T. We can’t be 100% sure face masks work – but that shouldn’t stop us wearing them | Trish Greenhalgh. The Guardian. 2020 [cited 2020 Jun 27]; Available from: https://www.theguardian.com/commentisfree/2020/jun/05/face-masks-coronavirus .

Hammersley M. So, what are case studies? In: What’s wrong with ethnography? New York: Routledge; 1992.

Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Med Res Methodol. 2011;11(1):100.

Luck L, Jackson D, Usher K. Case study: a bridge across the paradigms. Nurs Inq. 2006;13(2):103–9.

Yin RK. Case study research and applications: design and methods: Sage; 2017.

Hyett N, A K, Dickson-Swift V. Methodology or method? A critical review of qualitative case study reports. Int J Qual Stud Health Well-Being. 2014;9:23606.

Carolan CM, Forbat L, Smith A. Developing the DESCARTE model: the design of case study research in health care. Qual Health Res. 2016;26(5):626–39.

Greenhalgh T, Annandale E, Ashcroft R, Barlow J, Black N, Bleakley A, et al. An open letter to the BMJ editors on qualitative research. Bmj. 2016;352.

Thomas G. A typology for the case study in social science following a review of definition, discourse, and structure. Qual Inq. 2011;17(6):511–21.

Lincoln YS, Guba EG. Judging the quality of case study reports. Int J Qual Stud Educ. 1990;3(1):53–9.

Riley DS, Barber MS, Kienle GS, Aronson JK, Schoen-Angerer T, Tugwell P, et al. CARE guidelines for case reports: explanation and elaboration document. J Clin Epidemiol. 2017;89:218–35.

Download references

Acknowledgements

Not applicable

This work was funded by the Medical Research Council - MRC Award MR/S014632/1 HCS: Case study, Context and Complex interventions (TRIPLE C). SP was additionally funded by the University of Oxford's Higher Education Innovation Fund (HEIF).

Author information

Authors and affiliations.

Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK

Sara Paparini, Chrysanthi Papoutsi, Trish Greenhalgh & Sara Shaw

Wellcome Centre for Cultures & Environments of Health, University of Exeter, Exeter, UK

Judith Green

School of Health Sciences, University of East Anglia, Norwich, UK

Jamie Murdoch

Public Health, Environments and Society, London School of Hygiene & Tropical Medicin, London, UK

Mark Petticrew

Institute for Culture and Society, Western Sydney University, Penrith, Australia

Benjamin Hanckel

You can also search for this author in PubMed   Google Scholar

Contributions

JG, MP, SP, JM, TG, CP and SS drafted the initial paper; all authors contributed to the drafting of the final version, and read and approved the final manuscript.

Corresponding author

Correspondence to Sara Paparini .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Paparini, S., Green, J., Papoutsi, C. et al. Case study research for better evaluations of complex interventions: rationale and challenges. BMC Med 18 , 301 (2020). https://doi.org/10.1186/s12916-020-01777-6

Download citation

Received : 03 July 2020

Accepted : 07 September 2020

Published : 10 November 2020

DOI : https://doi.org/10.1186/s12916-020-01777-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative
• Case studies
• Mixed-method
• Public health
• Health services research
• Interventions

BMC Medicine

ISSN: 1741-7015

• General enquiries: [email protected]

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Case Study | Definition, Examples & Methods

Case Study | Definition, Examples & Methods

Published on 5 May 2022 by Shona McCombes . Revised on 30 January 2023.

A case study is a detailed study of a specific subject, such as a person, group, place, event, organisation, or phenomenon. Case studies are commonly used in social, educational, clinical, and business research.

A case study research design usually involves qualitative methods , but quantitative methods are sometimes also used. Case studies are good for describing , comparing, evaluating, and understanding different aspects of a research problem .

Table of contents

When to do a case study, step 1: select a case, step 2: build a theoretical framework, step 3: collect your data, step 4: describe and analyse the case.

A case study is an appropriate research design when you want to gain concrete, contextual, in-depth knowledge about a specific real-world subject. It allows you to explore the key characteristics, meanings, and implications of the case.

Case studies are often a good choice in a thesis or dissertation . They keep your project focused and manageable when you don’t have the time or resources to do large-scale research.

You might use just one complex case study where you explore a single subject in depth, or conduct multiple case studies to compare and illuminate different aspects of your research problem.

Prevent plagiarism, run a free check.

Once you have developed your problem statement and research questions , you should be ready to choose the specific case that you want to focus on. A good case study should have the potential to:

• Provide new or unexpected insights into the subject
• Challenge or complicate existing assumptions and theories
• Propose practical courses of action to resolve a problem
• Open up new directions for future research

Unlike quantitative or experimental research, a strong case study does not require a random or representative sample. In fact, case studies often deliberately focus on unusual, neglected, or outlying cases which may shed new light on the research problem.

If you find yourself aiming to simultaneously investigate and solve an issue, consider conducting action research . As its name suggests, action research conducts research and takes action at the same time, and is highly iterative and flexible. 

However, you can also choose a more common or representative case to exemplify a particular category, experience, or phenomenon.

While case studies focus more on concrete details than general theories, they should usually have some connection with theory in the field. This way the case study is not just an isolated description, but is integrated into existing knowledge about the topic. It might aim to:

• Exemplify a theory by showing how it explains the case under investigation
• Expand on a theory by uncovering new concepts and ideas that need to be incorporated
• Challenge a theory by exploring an outlier case that doesn’t fit with established assumptions

To ensure that your analysis of the case has a solid academic grounding, you should conduct a literature review of sources related to the topic and develop a theoretical framework . This means identifying key concepts and theories to guide your analysis and interpretation.

There are many different research methods you can use to collect data on your subject. Case studies tend to focus on qualitative data using methods such as interviews, observations, and analysis of primary and secondary sources (e.g., newspaper articles, photographs, official records). Sometimes a case study will also collect quantitative data .

The aim is to gain as thorough an understanding as possible of the case and its context.

In writing up the case study, you need to bring together all the relevant aspects to give as complete a picture as possible of the subject.

How you report your findings depends on the type of research you are doing. Some case studies are structured like a standard scientific paper or thesis, with separate sections or chapters for the methods , results , and discussion .

Others are written in a more narrative style, aiming to explore the case from various angles and analyse its meanings and implications (for example, by using textual analysis or discourse analysis ).

In all cases, though, make sure to give contextual details about the case, connect it back to the literature and theory, and discuss how it fits into wider patterns or debates.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2023, January 30). Case Study | Definition, Examples & Methods. Scribbr. Retrieved 26 August 2024, from https://www.scribbr.co.uk/research-methods/case-studies/

Is this article helpful?

Shona McCombes

Other students also liked, correlational research | guide, design & examples, a quick guide to experimental design | 5 steps & examples, descriptive research design | definition, methods & examples.

• Anatomy & Physiology
• Astrophysics
• Earth Science
• Environmental Science
• Organic Chemistry
• Precalculus
• Trigonometry
• English Grammar
• U.S. History
• World History

... and beyond

• Socratic Meta
• Featured Answers

What is the limitation of a case study?

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 17 September 2020

World-wide barriers and enablers to achieving evidence-informed practice in education: what can be learnt from Spain, England, the United States, and Germany?

• Joel R. Malin 1 ,
• Chris Brown 2 ,
• Georgeta Ion   ORCID: orcid.org/0000-0001-9915-0698 3 ,
• Isabell van Ackeren 4 ,
• Nina Bremm 5 ,
• Ruth Luzmore 2 ,
• Jane Flood 2 &
• Gul Muhammad Rind 1  

Humanities and Social Sciences Communications volume  7 , Article number:  99 ( 2020 ) Cite this article

8661 Accesses

29 Citations

43 Altmetric

Metrics details

• Social policy

A global push exists to bolster the connections between research and practice in education. However, fostering evidence-informed practice (EIP) has proven challenging. Indeed, this ‘problem’ requires simultaneously attending to multiple aspects/levels of education systems, and to the contexts within which they reside. As such, comparative analyses using systems approaches hold potential for achieving context-specific insights regarding how to foster EIP. However, such analyses have been scarce, and what research does exist has generally been limited relative to methods and theory. Given this, the present study executes and describes/reflects upon a novel approach for analysing and comparing EIP in/across systems. In this study, educators’ evidence use patterns are described and comparatively analysed, using a sample of four regions within high-income national settings: Catalonia (Spain), England (UK), Massachusetts (USA), and Rheinland-Pfalz (Germany). This study employs a dual analytical frame (a cohesion/regulation matrix and institutional theory) to supply a methodological lens through which to understand EIP within and across these four systems. Together, this approach not only provides a way of accounting for the macro-level differences between contexts, it also enables a comparison of meso-level and micro-level factors (via institutional theory) that might be common and distinct across systems. This study’s findings reveal substantial diversity in the extent and nature of evidence use between systems, which in turn patterned according to distinctive cultural, systemic, and institutional features. Considering these findings, this study’s discussion advances some provisional insights and reflections regarding actual and potential EIP in education. For example, variability relative to the types/extents of accountability pressures, and how this affected educators’ data and evidence use, enabled a discussion holding relevance for policymakers. We also share process-related insights—i.e., describing the advances and challenges we experienced while undertaking this new approach. These points hold relevance for colleagues wishing to emulate and improve upon the efforts described herein, which we argue are applicable both in and beyond the education sector. Relative to education, these approaches can be applied and improved with an eye toward developing context-specific (vs. one-size-fits-all) packages for fostering EIP and, ultimately, achieving high quality and progressively improving schools/systems.

Similar content being viewed by others

Neoliberal reform discourse in Egyptian higher education

The mixed classroom: a pedagogical experiment with students and policymakers

Meta-thematic analysis of quality in early childhood education and care

Introduction and aims.

This paper examines the question of how to bring about more consistent, evidence-informed practice (EIP) in education globally. To do so, we use a social regulation/cohesion matrix and an institutional analytic lens to engage in a comparative analysis across four contexts: Catalonia (Spain), England (UK), Massachusetts (MA, United States), and Rheinland-Pfalz (RP) (Germany). The aim of the analysis is twofold: First, we aim to explore and critique a new way to analyse EIP in (and potentially beyond) education. Second, we aim to generate insights into how to more routinely foster EIP, and to ascertain whether there were generalisable lessons from education that can be applied to other social policy areas. Although not without limitations, a number of insights do emerge from our work. These include that research evidence is but one of the potential influences on practice; furthermore, that use is contingent on a host of favourable features and conditions, set across micro, meso, and macro levels. Perhaps most significantly, this study demonstrates a new theoretical/methodological approach for studying EIP in/across systems, which we suggest can be taken up in and beyond the education sector.

Evidence-informed teaching

Across many countries, national, federal and district level governments are increasingly pursuing approaches to school improvement that seek to achieve so-called ‘bottom-up’ change. That is, improvements to teaching and learning that are generated by teachers and subsequently shared horizontally and vertically within educational systems. In particular, ‘self-improvement’ is now viewed by many as the preferred approach to enhancing educational provision at the school and system level (Greany, 2015 ). An approach often chosen to support self-improvement is that of EIP. EIP involves fostering situations in which teaching practice is consciously informed by evidence derived from: (1) formal research produced by researchers; (2) practitioner enquiry; and/or (3) routinely collected school or system-level data (for example, student assessment data). A focus on EIP is not without merit and there is a nascent but growing evidence base to suggest that when teachers engage with evidence, this can lead to improvements in outcomes for both teachers and students. For instance, correlational data indicates that where research is used as part of high-quality initial teacher education and ongoing professional development, there is an association with higher school and school system performance (Mincu, 2014 ). More recently Rose et al. ( 2017 ), using a randomised control trial, showed that increased collaborative research use by primary school teachers had a positive impact on primary school students’ exam results. A range of positive teacher outcomes that emerge from collaborative research-informed practice include: improvements in pedagogic knowledge and skills, greater teacher confidence, and high teacher job satisfaction (Bell et al., 2010 ; Godfrey, 2016 ). Similarly, teachers’ use of educational data (e.g. standardised test scores, data used for formative assessment, student self-assessment data, or other data such as attendance) can, in the right situations —such as part of a professional development initiative—also lead to improved teaching and student outcomes (e.g. Lai et al., 2014 ; Van Geel et al., 2016 ).

Various theories of change for why EIP should lead to improved teaching and student outcomes have been established (e.g. see Brown et al, 2017 ; Cain, 2015 ; Cain et al., 2019 ). Broadly these argue that, assuming they have both access and the capacity to do so, teachers can use a full gamut of evidence in relation to the decision-making that occurs as part of their work. For example: (1) evidence—especially forms of data—can be used by school leaders to identify and pinpoint areas for improvement, both in terms of a given cohort or group of students, or in relation to an innovation that is required at the level of the school or across schools; (2) evidence can aid teachers in the design of new bespoke strategies for teaching and learning in order to tackle specific identified problems; (3) evidence can provide teachers with ideas for how to improve aspects of their day to day practice by drawing on approaches that research has shown to be effective; (4) ideas from research can help teachers expand, clarify and deepen their own concepts, including the concepts they use to understand students, curriculum and teaching practice; and (5) programme evaluations can also provide teachers with specific programmes or guidelines, shown by research to be effective, which set out how to engage in various aspects of teaching or specific approaches to improve learning. Finally, data can be used to assess the impact of embarking on (2)–(5) above. Thus, if teachers are able to engage with evidence (both research and data) in a way that enables them to undertake any of these actions, their teaching quality should be improved. Correspondingly, improved teaching quality should then lead to improved student outcomes.

Although there is now a recognition that evidence use can and should be used to improve practice, there is only limited evidence on how this might be facilitated at the school level (Graves and Moore, 2017 ). What’s more, a systemic gap appears to exist between research and practitioners which as yet shows little indication of narrowing (Coldwell et al., 2017 ; Graves and Moore, 2017 ; Whitty and Wisby, 2017 ). As a result, this leaves only sporadic instances of EIP occurring within and across schools with other factors, such as intuition and experience, instead solely driving much of the decision-making undertaken by teachers (Vanlommel et al., 2017 ). The danger then is that misconception, biases, and fallible ‘fast’ decisions (Kahneman, 2011 ) are as likely to influence teachers’ decision making as much as high quality evidence that helps the identification of problems or that point to effective solutions. A key question therefore is what can we do to achieve EIP? In other words, how can we can get school principals and teachers within education systems, globally, to systematically use high quality academic research and other forms of evidence to improve how they lead and teach?

In this paper we attempt to shed light on this issue by examining EIP, in a new way, in/across four school systems: Catalonia, England, Massachusetts, and Rheinland-Pfalz (RP). (Our selection of these four cases is described further in a subsequent section.) In doing so, we aim both to generate provisional insights related to fostering more/better EIP in education, and to set a discussion regarding the merits and drawbacks of the methodological and theoretical approaches that we undertook. In addition, in light of the aims of this special edition, this paper also seeks to ascertain whether there are generalisable lessons from education that can be applied to improve EIP (and/or its study) in other key areas such as health, justice and social care.

Comparative analyses of evidence use in education have been scarce. Typically, evidence use or research engagement has not been a primary focus of educational researchers who have undertaken comparative study, though these topics still sometimes reveal themselves. Darling-Hammond et al. ( 2017 ), for example, selected a small set of high-performing systems and then sought to examine commonalities and generate useful insights. Germane to the present study, they found these jurisdictions tended to view and support teaching as a “research-informed and research-engaged profession” (p. 15). Relatedly, they noted systemic ways in which these systems supported teaching as a collaborative rather than an isolated occupation (e.g., affording opportunities to observe others’ lessons, fostering teacher sharing within- and across-schools).

There have also been a small number of more direct comparative examinations of EIP (or evidence-informed policy) in recent years, though these too have featured different samples, goals, and analytical approaches relative to our paper. For example, the Organisation for Economic Co-operation and Development (OECD) published an edited volume, “Evidence in Education: Linking Research and Policy” (Burns and Schuller, 2007 ). This book brought together “experts on evidence-informed policy in education from a wide range of OECD countries” (OECD, 2020 , n.p.). The book as a whole, and chapters within, supplied numerous insights while detailing various ‘cases’ (e.g., organisations, knowledge brokers, programmes). However, its focus on evidence-use within systems was generally limited, it focused more at policy than at practical levels, and contributors did not operate from a common framework to aid cross-case comparisons. Also, the Evidence Informed Policy and Practice in Education in Europe (EIPPEE; see http://www.eippee.eu/ ) network’s funded work yielded comparative information and insights relative to knowledge brokerage activities and mechanisms across 11 European countries (e.g., see Gough et al., 2011 ). Importantly, the analytical framework they developed and drew upon reflected systems thinking/modelling; in line with Best and Holmes ( 2010 ), they emphasised how various agents/actors are tied together by a system and embedded/organised through structures that shape the interactions and knowledge exchange that ultimately takes place.

Similarly, we see value in a systems approach. Given this, we employ a dual analytical frame (as described below: a cohesion/regulation matrix and institutional theory) to supply a methodological lens through which to understand EIP within and across four systems. The system matrix, presented next, supports the idea of the “power of context” (Chapman, 2019 , p. 4). It highlights the critical role of the local diversity of settings that systems operate in and contributes to understanding the variety of layers intervening in the implementation of any school reforms and innovations, including EIP. Besides, the matrix ensures there is a focus on the different challenges systems experience, and represents a starting point for any analysis which looks on the configuration of facilitators and barriers in any school improvement process. Each context is unique and powerful and determines a specific configuration of factors, agents, and conditions, which can be explored in-depth through the lens of institutional theory. Together, then, our dual approach not only provides a way of accounting for the macro-level differences between contexts, it also enables a comparison of meso-level and micro-level factors (via institutional theory) that might be common and distinct across systems.

The cohesion/regulation matrix

Globally, school systems have a range of differentiating contextual and structural elements. The cohesion/regulation matrix, set out in Fig. 1 , has previously been used by Chapman ( 2019 ) (drawing on the work of Hood, 1998 ) as a way of segmenting school systems according to the principal macrolevel or system-level factors that define them. The axis used to form the matrix may be considered as follows: ‘social cohesion’ refers to the institutions, norms and networks that bind societies together. Systems with high social cohesion have a higher propensity and willingness to collaborate. Threats to social cohesion—which tend to result in low socially cohesive systems—tend to occur when such structures and systems (e.g. specific layers of government, the trade unions, the church, as well as the provision of universal services such as health) are dismantled and replaced with deregulation and privatisation. In other words, approaches that place an onus on individual agency over collective approaches (Bauman, 2012 ). The second axis, ‘regulation’, refers to the institutions that determine control and how accountability functions in a system. Typically, in a high regulation system there is a dominant hierarchical culture with associated bureaucratic control. High regulation systems often also involve the danger of ‘high stakes’ failure; i.e. a situation in which not meeting exacting accountability standards results in individuals or institutions being highly penalised. Systems displaying low social regulation, on the other hand, tend to exhibit much flatter, non-hierarchical cultures, with improvement achieved through partnership. A low social regulation system is also much less likely to have external accountability measures which lead to penalisation.

This figure depicts a social cohesion/regulation matrix, which draws on the work of Hood ( 1998 ).

As can be seen in Fig. 1 , combinations of high/low social cohesion and high/low social regulation result in the following four system types (Hood 1998 , p. 9):

The fatalist way (top right quadrant): characterised by rule-bound approaches to organisation but there is little cooperation to achieve outcomes.

The hierarchist way (top left quadrant): displays social cohesion and cooperation in order to meet rule-bound approaches to organisation. Often characterised by bureaucracy.

The individualist way (bottom right quadrant): atomised approaches to organisation and which involve negotiation and bargaining between actors.

The egalitarian way (bottom left quadrant): high participation structures, in which all decisions are ‘up for grabs’ combined with an egalitarian culture and peer to peer support.

Institutional theory

Within each school system there are further factors that affect the behaviour of school leaders and teachers. To account for these we draw on institutional theory. Institutional theory (Powell and DiMaggio, 1991 ) offers a useful and under-utilised lens for understanding the complicated relations between evidence and practice (Martin and Williams, 2019 ). Institutional theory is aimed at clarifying facilitators and constraints, within a given social field, to organisations’ behaviour. Accordingly, it can show how organisations’ (i.e., schools/districts) activities are governed by various formal and informal rules and norms (Martin and Williams, 2019 ). These rules and norms tend to be durable, and within a field we tend to see substantial continuity—because, for example, different organisations are frequently subject to similar pressures and tend to have reached similar understandings regarding what behaviours are (in)appropriate within their realms (Powell and DiMaggio, 1991 ).

The ‘state’ is generally an important actor when considering public service provision, as for example public sub-entities are typically at least partially dependent on the state for resources. Accordingly, state-level expectations are influential, and some are codified in formal policies that direct attention and work effort in certain directions. Thus, certain forms of evidence—the focus of this manuscript—and certain priorities are likely to be privileged while others are side-lined or given lesser focus. In many cases (albeit more so in some fields than others), professionals also are significant in terms of enabling/constraining and regulating members’ behaviours. Within organisations some individuals are relatively more powerful than others, which also might have implications in terms of evidence use. For example, Brown and Malin ( 2017 ) described how school principals are pivotal in terms of bringing about EIP in their respective settings. Such individuals and groups are also often key to knowledge brokerage in various ways (e.g., via setting meetings, via their communications, and so forth; see Malin and Brown, 2020 ).

A key insight from institutional theory—as applied to the study of evidence use—is that “the strength of the evidence is not the sole, or even the major, determinant of its influence on practice; rather, more powerful actors hold considerable sway in determining what (and indeed whether) evidence is used” (Martin and Williams, 2019 , p. 55). However, “this does not also mean that there is no deviation from institutionally prescribed behaviours” (Martin and Williams, 2019 , p. 56). In fact, practitioners are often creative despite substantial constraining institutional forces. Thus, a challenge for researchers using institutional theory is to sensitively examine contextualised norms, rules, and structures, while also attending to the reality of what is occurring in order to accomplish the focal organisation’s main tasks.

Research questions and approach

In the following paper, we attempt to shed light on these three questions:

To what extent are teachers in systems with different types of cohesion/regulation characteristics engaging in EIP?

What EIP-related enabling or hindering factors do different systems present and what are the relative ‘strength’ of these enablers and hindering factors?

Are there generalisable lessons from education—at the system level—that can be applied to improve EIP in other key areas such as health, justice and social care?

To address these questions we present four school systems—Catalonia, England, Massachusetts, and RP—as miniature case studies. These cases were selected out of a combination of convenience and strategy. Strategically, the recruitment of this study’s team of authors was driven by the lead authors’ desire to comparatively examine contexts reflecting diversity along the dimensions being studied. Accordingly we sought authors known to possess information access and expertise that would collectively enable us to address this study’s main questions, relating to the context with which they were most familiar as we turned later to a cross-case analysis.

We selected cases that correspond to different system models, creating opportunities to identify commonalities and differences between them and lessons for improvement. We see a mosaic of policy landscapes across Europe and the U.S., each one with its unique development, challenges, tensions, and dilemmas. The Catalan model is an example of the contextualisation of legal regulations, being part of the Spanish system, characterized by collegiality, decentralisation, and institutional autonomy. Despite the high level of social cohesion, local regulations have a central influence, enhancing the level of bureaucracy and generating a hierarchical culture, which supposes a series of tensions between system levels and agents. The English system is typical of what Pasi Sahlberg refers to as Global Educational Reform Movement (Sahlberg, 2016 ) systems: as well as being characterized by high autonomy and high accountability, there is a focus in England on the core subjects of literacy and numeracy. This has led to much standardisation of practice, despite the aim of recent reforms to create an innovative self-improving system. In other words, despite having the freedom to experiment, school leaders often choose to emulate the practices of others out of fear of being an outlier and subsequently punished for failure. Massachusetts appears by most measures to be the top-performing state education system in the United States. It is also a context in which considerable efforts have been made, at state and national levels, to increase/enhance the use of research in policy and practice. For example, its state-level education department has shown a systematic and pioneering commitment to planning and research, and key federal law in the US maintains evidence and accountability requirements. RP is also unique, as it is the only German state without state-wide exams. It has justified doing so by pointing to its students’ comparatively good results in nationwide tests of student performance. More recently, even the school inspection system has been abolished (the current study, however, refers to this instrument), as it was evaluated as too costly and not very effective. Instead, more investment is now being made in enhancing support systems for schools. RP thus offers a particularly good example of a rather egalitarian approach in the matrix and data use in such a system.

Despite our best efforts, this study contains certain limitations. For example, our study relied upon authors’ access to extant data and other information (versus requiring novel data collection) related to the focal questions. Accordingly, we did not as part of this research collect/analyse uniform data across the cases we studied, but rather relied upon what was available for each case. Accordingly, cross-case comparisons are made—and should be taken—with some caution. The contexts we studied also do not represent the full diversity of educational systems internationally. Still, our aim is to compare an educational phenomenon within distinct cultural areas, with regard to its contextuality and different governance constellations in the sense of international comparative education: “As comparative education is a field that is fundamentally grounded on an interest in learning from each other’s experience (that is, generated from each other’s contexts), context has always mattered“ (Lee et al., 2014 , p. 150). For example, in this study we did not include any systems that fit in the individualist quadrant of the social cohesion/regulation quadrant.

We have classified each context according to the cohesion/regulation matrix in Fig. 1 and justify this by detailing the specifics of the different systems that make this so. We then set out to examine: (i) the extent to which teachers implement research evidence into their teaching practice (outcome); (ii) which enablers and barriers with research use are described in the different systems, using institutional theory as a way of guiding how and why types of evidence are privileged and the more and less powerful evidence actors within those systems; (iii) finally we assess the relative ‘strength’ of these enablers and barriers linked to specifics of the different systems.

Catalonia (Spain)

The Catalan educational model, mirroring the Spanish model, sits within the top left quadrant of Fig. 1 (the hierarchist way). It is characterised as a system based on a collegial model, decentralisation and processes of institutional autonomy, as well as the contextualisation of legal regulations (Marchesi and Martin, 2002 ). It is a model that offers a privileged position to agreed-upon proposals for action based on an educational project and a series of intervention projects for improvement; which, according to the Act on the Right to Education (LODE, 1985 ) currently include: improvement plans, the curricular project, the environmental plan, etc. Schools display different levels of autonomy in the areas of planning, management and organisation. However, with a strong background as a centralized system in the 1990s, schools are responsible for designing and implementing education and management plans under the supervision of their respective education authorities. It is within the powers of the principal according to the Act on the Education Quality Improvement (LOMCE, 2013 ) to ensure the functioning of the school and stimulate improvement processes. According to this model, Catalan educational law reinforces the idea of co-responsibility between schools and local authorities in decision-making, promotes collective responsibility in management, and empowers school management teams and teachers themselves to promote innovation through a horizontal system that supports collaborative initiatives (Gairin, 2015 ). The model balances high social cohesion with a high level of participation. At the same time, it addresses educational inertia with a high interest in accountability and governance through a weakly articulated structure based on a cultural model of quality assessment that uses rigid standards and structures. The coexistence of two models of governance in tension makes it difficult to fully transition to a completely decentralised and cohesive model.

Use of evidence

The Catalan educational system is experiencing a very diffuse and spontaneous wave of change and educational innovation (Martinez, 2019 ), magnified by a clear commitment to the emergence of collaborative networks between schools and social and educational organisations (Azorín, 2019 ). Of late, this phenomenon has been driven by the emergence of complex problems and the lack of sufficient resources (Díaz-Gibson et al., 2015 ) to address them, leading schools to seek solutions through the ‘Planes Educativos de Entorno’ (environment educational plans) or ‘Local Educational Networks’ (Civís and Longás, 2015 ). These experiences are based on collaborative work and networks that assume the existence of a new paradigm for socio-educational services and the growth of extended (community) schools (Azorín, 2019 ). These developments have generated a complex scenario with different levels of involvement and improvement and specific adaptations as a function of contextual variables such as school ownership (public–private), teachers’ stages of professional development and attitudes towards change and innovation (Perines, 2018 ), to mention just a few; these variables can generate an imbalance between levels of social cohesion and commitment to change.

To face these challenges, in recent years, so-called ‘evidence-informed practices’—which involve teachers integrating research evidence into decision-making—has taken on increased importance in the practice of schools as well as having a greater visibility in the public discourse. With the adoption of the Catalan Education Act (Decret 274/2018), a systemic and formal commitment to the promotion and use of evidence and research in the field of education has been put into place. This commitment represents the beginning of a new stage in Catalan policymaking promoted by the public bodies, where the aim is to make scientific knowledge an engine for improving educational practices and policies. To do that, the programme called “Evidence-informed schools” [Escoles d’evidència] aims to put evidence on what works in education in the service of educational policies and schools, to promote the most rigorous empirical evidence, and at the same time to connect it with the needs of the system, schools, and teachers. According to the Decret 274/2018, with this programme, Catalonia will work towards the articulation of an ecosystem that brings together all the educational agents and puts research evidence at the service of improving education.

Despite the late visibility of the concept and some pioneer initiatives implemented in schools in recent years, (especially promoted by the private sector such as the EduCaixa programmes [EduCaixa, 2019 ] or innovation movements based on evidence), one cannot say that this trend has become generalised. Studies show that adopting an evidence-based view of teaching requires an understanding of how to integrate teachers’ experiential knowledge and should be complemented by contextual and experiential interpretations of research with a reflective approach to practice (see Ion et al., 2019 ). This is a paradigm shift that involves interventions at all levels.

Application of institutional theory

Framed by institutional theory, we can discern that different levels are involved, from an epistemological level linked to one’s conception of research, to the personal, organisational and systemic levels (these are further explained below). The ecology of educational practices and policies that are ‘enriched’ (Oancea, 2018 ) with research should involve, in a shared manner, all actors: teachers, students, public administration, local authorities, and government agencies. The dialogue among these actors involves harmonising different interests and narratives, which in turn depends on, and generates power relations amongst, different contexts and tensions and dilemmas between contexts and levels, agents and decision makers.

First, at the epistemological level, Spanish teachers tend to disconnect the conception of research from educational practice. For example, data indicates that teachers perceive research as a type of abstract knowledge that is useless, of poor quality and far removed from their daily practice (Murillo and Perines, 2017 ; Murillo, 2006 ; Díaz Costa, 2009 ). The ability to support research-informed practices from the bottom-up depends equally on teachers’ individual capacity to promote and work in a climate of trust and collaboration, in which the exchange of knowledge and the shared and grounded construction of and critical reflection on their own practice, represents their own professional ethos. However, studies in the Spanish context show that innovations are rarely based on research evidence, although teachers themselves claim to be in favour of the use of evidence in their classroom practice. In a study conducted with teachers in Madrid and Catalonia, 68.1% of teachers and 77.3% of principals declared that they frequently or always use research to inform their practices (Ion and Gairín, 2019 ). However, when they have to inform their innovations in class, teachers acknowledge limited use of scientific evidence in favour of experiential and peer knowledge (Perines, 2018 ; Ion et al., 2019 ). Among the factors that limit the development of the ability to use research, teachers include limitations of time, resources, or support from the management team (Perines, 2018 ). In addition, teachers identify clear deficiencies in their initial research training and a strong disconnect with the context of the production of research, marked by concerns about issues that diverge from the reality of the classroom (Perines, 2017 ).

At the organisational level, the development of the capacity to use evidence requires leadership that is clearly sensitive to research and favours a positive organisational culture (Ion et al., 2019 ). However, this is not sufficient to promote the use of evidence unless it is accompanied by a research culture that supports a general orientation towards the use of evidence in any decision-making process, assumes the contextual nature of knowledge, supports the integration of evidence into teachers’ professional development and cultivates an organisational ethos favourable to collaboration and academic integration (Oancea, 2018 ; Ion and Gairín, 2019 ).

At the system level, promoting a vision of practice based on evidence requires coherent and responsible actions among all actors. In Catalonia, innovative educational initiatives are still far from being a generalisable trend; rather, initiatives are isolated and depend on personal initiative (Camacho, 2016 ). This undoubtedly contributes to the fact that efforts to promote improvement and innovation continue to be poorly recognised and rewarded, poorly documented (Perines, 2017 , 2018 ) and very diverse. Furthermore, despite a high level of thematic diversity, initiatives appear to be minimally connected and are quite different at the methodological level. Additionally, at the system level, there are few mechanisms to identify and connect innovative educational practices with one another, which makes it difficult to identify different agents’ degree of development and involvement in the progress of the innovation process (Camacho, 2016 ; Martinez, 2019 ). In Catalonia, in a context of increasing concern based on accountability (Catalan Government, Departament d’Educació, 2019 ), politicians dedicated to managing educational systems tend to have a reduced view of educational research, such as the evaluation of educational systems. Additionally, the approval of laws and decrees are often not sustained by research, and the existence of accumulated research (such as research syntheses or meta-analyses) is simply unknown (Martinez, 2019 ). In this way, a divided model is reproduced, in which decision-making is separated from enquiry and reflection on practice, and the two exist in remote spheres that do not respond to each other. Change involves: (1) making evidence both sides of the same coin of discourse and practice (i.e., political discourse supporting evidence in practice and the measures taken should be aligned, to ensure coherence between all layers and actors); (2) introducing research as an instrument of both the political system and of governance; and (3) creating the conditions for research to fulfil a social function; that is, to have an impact beyond the academic function. Political discourse must double the measures to promote governance mechanisms that stimulate this type of practice and that support it with adequate resources and mechanisms of recognition and reward for horizontal and collaborative initiatives.

England (UK)

England’s high accountability (high social regulation) and high autonomy (low social cohesion) context places it firmly in the top right-hand quadrant of the matrix (the fatalist way). These two elements have achieved particular prominence in the last decade. Beginning with the latter, central government policy makers in England have now devolved multiple decision-making powers and resources to schools. Included in this process of devolution is the responsibility for teacher professional development, in the belief that this will improve quality and increase innovation (Greany and Earley, 2018 ; Howland, 2015 ). This commitment has been described elsewhere as the move towards a ‘self-improving school system’ (Greany, 2017 ). Here the characteristics of ‘self-improvement’ include individual schools now having greater responsibility for their own improvement; that teachers and schools are expected to learn from each other so that effective practice spreads; and that schools and school leaders should extend their reach to support other schools as they look to improve (Greany, 2014 ; Robinson, 2017 ).

A key point to understand is the structure of England’s school system. While education policy is shaped centrally, since the early 20th century local authorities had responsibility for the education of children in their locales. However, the relationship between local authorities and central government has not been easy and in 1988 the Education Reform Act saw local authorities lose many of their powers until their role was one of scrutiny and support. A significant change came with the establishment of academies, state schools directly funded by the Department for Education and outside local authority control. As self-governing trusts academies have a number of freedoms afforded to them in terms of innovation and curriculum which local authority schools do not. While some academies operate independently, a number of these schools are networked into trusts, groups of schools with centralised policies, curriculum and approaches to professional learning. Therefore the type of autonomy that staff in schools experience will vary depending on the type of school and its structure.

To further encourage improvements in quality and innovation, policy makers have also embedded a range of accountability systems. These “combine quasi-market pressures—such as parental choice of school coupled with funding following the learner—with central regulation and control” (Greany and Earley, 2018 , p. 7). A key aspect of this system is the regular school inspections process undertaken by Ofsted (the school inspection agency in England). Ofsted inspections are highlighted by many school leaders as a key driver of their behaviour and for good reason. As a result of an inspection, for which there is less than 24 h notice, schools are placed into one of four hierarchical categories of grades. The top grade—‘outstanding’—has historically carried a number of benefits. For example, it makes the school more attractive for parents, meaning more students apply to attend, and thus more funding is directed towards the school. The reverse is then true, that schools with lower ratings find it more challenging to attract families and the attached funding with it. In addition, up until 2019, schools rated outstanding were exempt for subsequent inspections (even with changes of leadership and staff), meaning that accountability pressures are considerably lessened. At the other end of the scale, schools judged to be in the lowest Ofsted category—‘inadequate’—are subject to a forcible removal from local authority control and the Department for Education pay academy trusts to take on these schools in a bid to rapidly improve performance. Inspection frameworks have seen a number of changes in them, which directly impacts the work in schools so that they are meeting an ever-evolving criteria of what is considered by Ofsted to be good practice.

In addition to inspection is the use of government produced annual ‘league tables’ of schools and a publicly available ‘Find and compare schools in England’ website, which allows those accessing it to rank schools on a number of different variables and student outcomes. As a result, it is acknowledged that England’s accountability framework both focuses the minds of—and places pressure on—school leaders to concentrate on very specific forms of school improvement. In particular, such improvement principally tends on ensuring students achieve well in progress tests in key subject areas (e.g. English literacy and mathematics) (Ehren, 2018 ) leading to a narrowing in the curriculum.

Some data does exist in terms of EIP in the English context. For example, a survey of 1670 teachers in England was undertaken by the National Foundation for Educational Research in 2017. Here it was found that academic research had only a ‘small to moderate’ influence on teacher decision making. Instead of research-evidence, when deciding on approaches to improve student outcomes, teachers were in fact much more likely to draw ideas and support from their own experiences (60% of respondents identified ‘ideas generated by me or my school’), or the experiences of other teachers/schools (42% of respondents identified ‘ideas from other schools’). In addition, non-research-based continuing professional development (CPD) was also cited as an important influence (54% of respondents). These compare to the much lower figures of 13% and 7% for ‘sources based on [the] work of research organisations’ and ‘advice/guidance from a university or research organisation’, respectively (Walker et al., 2019 ). The survey also asked teachers to identify the relative importance of a range of factors likely to have an impact on any decision to adopt a new approach to teaching and learning. The factors that teacher respondents were most likely to identify were: straightforward to use (47%); aligned with professional expertise (46%); and a good fit with existing practices (44%). Only a third (32%) indicated research was a ‘strong influence’ on their decision to adopt their approach (Walker et al., 2019 ).

Turning now to institutional theory and it is clear that some salient elements are present. For instance, EIP has been supported from non-governmental organisations that operate to support schools. The Education Endowment Foundation (EEF)—the ‘what works’ centre for education in England—for example, provides a freely available ‘tool kit’ of what works evidence in order to ensure summaries of educational research are accessible to non-academics. In addition to this substantial investment, in 2014 the EEF launched a £1.4 m fund to improve the use of research in schools (EEF, 2014 ). This initiative was followed up in 2016 with the launch of the EEF’s Research Schools initiative; schools charged with leading EIP in their local area. There has also been a substantial rise in bottom up/teacher-led initiatives, such as the emerging network of ‘Teachmeets’ and ‘ResearchED’ conferences (Whitty and Wisby, 2017 ) designed to help teachers connect more effectively with educational research. Furthermore, a prominent example of a teacher-led initiative was the 2017 launch of England’s Chartered College of Teaching: an organisation led by and for teachers and whose mission, in part at least, is to support the use of EIP (Whitty and Wisby, 2017 ). EIP is also being increasingly promoted and supported at a government level. For example, England’s Department for Education funds the work of the EEF, and has also ensured the inclusion of references to EIP within principals’ standards and in the pilot Early Career Framework for newly qualified teachers. Finally, the periodic Research Excellence Framework (the ‘REF’), via which UK universities are funded, now requires them to account for the “impact” their research has had on, “the economy, society, culture, public policy or services … beyond academia” (Higher Education Funding Council, England (HEFCE), 2011 , p. 48). In other words, the government’s aim is to use REF to encourage universities to ensure that their research is used in the world beyond academia, for example by directly working with teachers and schools (Cain et al., 2019 ).

Yet from the figures above it is clear that EIP has some way to go before it becomes a way of life in schools in England (Bell et al., 2010 ; Walker et al., 2019 ). Furthermore, the take-up of the EEF toolkit is limited, with just less than a quarter of teachers indicating they accessed it. The figure for school leaders is much higher however at just under 60% (EEF, 2018 ). Reported barriers to EIP are manifold and include research held behind paywalls, dense academic style writing which can be difficult to access, underdeveloped research literacy skills and support, both individually and organisationally; and the pressures of high stakes accountability in England’s schools (Brown, 2017 ; Greany, 2015 ). In addition, the ability for school leaders to put in place structures within their school to enable teachers to engage effectively in collaborative EIP development have been limited by the budget cuts which have ravaged the education and wider public sector in England (e.g. see Busby, 2019 ). Teachers themselves also lament the lack of time they have to do anything other than their day to day role, with EIP often seen very much as a luxury (Brown, 2020 ; Galdin-O’Shea, 2015 ). This argument is reinforced by OECD data which indicates that England’s Primary teachers have the fifth highest number of teaching hours out of all countries surveyed. While, a teacher in Finland has 677 h—and in German they have 799 h of contact time with pupils—a teacher in England has on average 942 h (OECD, 2020 ).

Massachusetts (USA)

Placing the Massachusetts (MA) United States (U.S.) school system in the cohesion/regulation matrix is not entirely straightforward. Regarding the individualism-egalitarianism dimension, though MA and the U.S. are individualistic in nature, teachers in their work settings are often more communitarian (Shober, 2016 ). Most recently, however, MA has embraced neoliberal and managerialist education policies (Piazza, 2017 ; Horsford et al., 2018 ); for instance, in 2012 MA enacted a law that limits seniority-based job protections for teachers and may undercut a communitarian, professional ethos. (Also see later discussion of Race to the Top policy.) MA is also—relative to other U.S. states—socially cohesive (e.g. Wise, 2015 places it in the top 10 on this measure). However, the U.S. presently is conspicuously un-cohesive, and MA is no marked exception. All considered, we have placed MA (like England) within the top right quadrant, the fatalist way, albeit with the understanding that this is a dynamic context and policy area.

MA is unique, relative to other U.S. states, in that its K-12 students’ achievement consistently ranks at or near the top (Papay et al., 2020 ). It has also, since at least 1993, been viewed as a leader in U.S. education reform. In 1993, the omnibus Massachusetts Education Reform Act introduced state-wide learning standards and an associated state testing/accountability system. In exchange for the increased accountability, state-based school funding also considerably increased. Notwithstanding these efforts and successes, educators and policymakers in MA have also taken note of, and have sought to rectify, considerable performance inequities between certain groups of students (i.e., inequitable outcomes according to class, race, ethnicity; Darling-Hammond, 2010 ; Papay et al., 2020 ).

In the U.S., as the national constitution does not specifically address education, states hold primary authority, though states historically have also delegated much authority and responsibility downward—i.e., to school district levels. Thus it is appropriate to examine U.S. school systems at a state level, but with attention also to local variation (i.e., districts, schools, teams). Nevertheless, educators are also substantially enabled/constrained by federal policies. U.S. educators have since the early 2000s needed to respond to a largely federally led “what works” agenda, characterised by a “strikingly narrow focus on evidence of the impact of interventions” (Tseng and Coburn, 2019 , p. 351) and neglecting broader concerns and types of evidence. Most notably, the No Child Left Behind Act of 2001 (NCLB) introduced high-stakes student achievement testing mandates and required that school leaders and teachers reduce achievement gaps. Although NCLB stimulated educators’ data use, it also squarely emphasised summative measures and contributed to other dubious practices (e.g., narrowed curricular offerings and a focus upon “bubble kids” who tested near proficiency cut-points; Datnow et al., 2013 ; Hackmann et al., 2019 )

In 2010, MA pursued and was subsequently awarded a US $250 mm federal Race to the Top (RttT) grant. RttT was an Obama-era federal grant competition (the largest ever of its kind in US history, and brought forth on the heels of a recession that left states especially solicitous for funds) that was aimed to spur educational innovation, but that was specifically focused around stimulating particular state-level reforms, including: (a) adoption of core standards and assessments; (b) building data systems that could measure student success and inform teachers/schools how they could improve; (c) the recruitment, development, and retention of effective teachers; and (d) turning around low-performing schools (Horsford et al., 2018 ). MA and other states needed to develop and implement policies in these areas, which clearly reflected a neoliberal approach to education reform, in order to compete for and receive this funding. These policy shifts subsequently enabled and constrained educators’ behaviours and focal areas, introducing new regulatory pressures and directed their attention toward certain forms of data and evidence (more discussion to follow as part of institutional theory analysis).

MA’s educational system is hierarchically organised, albeit in some cases with overlapping authority and in some aspects with the higher (i.e., state department of education) level serving more as a resource/support (and less as a heavy-handed governor) to the local districts and their educators. For example, and pertinent to this manuscript, MA’s state-level education department was the first to include a state research director, part of a robust Office of Planning and Research (OPR). MA educators and state education officials are also beholden in some key ways to federal educational law, such as the Every Students Succeeds Act of 2015, which replaced NCLB and, while devolving some authority to states, largely kept intact key evidence and accountability requirements (Tseng and Coburn, 2019 ).

What does all of this mean in terms of MA educators’ engagement with research evidence? This case draws primarily upon findings presented as part of a recent study, ‘Evidence use in Massachusetts School Districts’, completed by Hedberg ( 2018 ) for DESE’s OSR. This interview study ( N  = 22 district-level interview participants), drawn from a stratified sample of MA districts, ‘sought to understand how districts are currently using, building and sharing data and research’ (p. 1). This study was undertaken as part of early stage efforts by DESE/OSR to support MA districts’ evidence use. Supportive insights are also drawn from a piece by Carrie Conaway ( 2020 ), who then was research director for OPR.

The first point is that ‘data’—and especially from a particular source—are being used more frequently than research. In part, this finding reflects the aforementioned policy and larger agenda, which has tuned educators’ focus especially toward students’ performance on annual tests. Indeed, in this case educators mentioned their state test—“MCAS”—data as being of premium importance. Hedberg observes, “looking at student performance data is becoming part of the regular routines for teachers and administrators” (p. 4). Some districts ( N  = 6 of 19) have also invested in one or more staff members whose function revolves around data or evidence. In all, districts appear to “have more systems for integrating data use in their decision-making than for integrating outside research” (p. 9). These systems/structures include data meetings (13 mentions), professional development (5), and dedicated data teams (5).

In terms of ‘building evidence,’ findings were mixed. On one hand, many districts reported conducting in-house research and/or partnering with outside organisations in order to do so. On the other, in most cases the descriptions offered did not suggest formal research questions and data collection/analyses. Thus, most common—at least among this sample—is engagement in informal research related to several key areas (e.g., tracking school culture and climate).

Research evidence use reportedly occurred in certain ways—for instance, 25% noted looking at the research base as part of selecting new programmes or interventions. Respondents also reported using either data or research to “adopt new materials” (9.13 on a scale of 1–10; 10 = all the time), “select intervention” (8.87), “provide professional development” (8.27), “inform instruction” (8.07), “allocate funds” (7.53), and “allocate staff” (7.0). Evidence was also used by some respondents/districts to measure implementation, and to measure impact (primarily via student assessment data).

This study also inquired about real and potential barriers to evidence use. Responses suggest limited time/staff resources (12 mentions), value of available research (5), and culture (3) present the largest obstacles. Confidence in engaging with research did not appear to be a considerable barrier, at least for these respondents. Across-district evidence sharing appears to be modest amongst MA educators, with most such sharing occurring “at conferences or collaborative meetings” (p. 9). Hedberg ( 2018 ) globally sensed some “scepticism around research” (p. 13) including vendor-produced research that was thought to be biased.

Lastly, it was clear that most educators accessed research indirectly and in brokered fashion—i.e., through professional associations (8 mentions) and conferences, and from ‘other education publications such as the Marshall Memo (also see Malin et al., 2018 ; Malin and Paralkar, 2017 ). State resources were also noted, again underscoring the importance of the broader system in facilitating or hindering use.

When applying institutional theory to the MA case, some key elements are evident. First, we can see how formal policies (state and federal) have constrained educators’ attention in certain directions (e.g., attentive to high-stakes testing data, and more generally toward data relative to other sources of evidence). Within that parameter, we can see the state attempting to be helpful, providing data in a timely fashion and in a format that is said to be desirable. There also appear to be earnest and somewhat successful efforts to facilitate coherence across the system, with the state assuming a leading role. The pattern thus appears to be a primarily top-down approach to evidence (and, more specifically, certain canonical data), whereby evidence from some other authoritative location is brought into practice (Martin and Williams, 2019 ). The state also is showing interest in encouraging/facilitating bottom-up use, e.g., by “facilitating the cross-pollination of ideas and resources by evidence-oriented practitioners” (Hedberg, 2018 recommendation, p. 2).

Despite considerable isomorphic pressures on MA educational organisations and educators, from the top, there also appears to be considerable organisational-level diversity in EIP. Some districts, for instance, have invested in data coaches or similar, whereas others have not. Meanwhile, some schools and districts are facing major pressures to improve or turnaround their schools (again, a top-down policy), and educators within these districts are compelled to engage with evidence in different ways and with different levels of urgency. Within such a situation, we suggest there is still potential for cross-district sharing if/when educators/organisations could network and form consortia according to common challenges and interests. However, results of the evidence use survey suggest much of this potential is currently untapped.

We can also see that the state, though a very important part of the evidence use system in several ways (e.g., via relevant laws, its assessments, and its research supports), is not the sole influencer beyond the level of the organisation. For example, professional associations play a key role in facilitating evidence sharing and use in MA, and certain other brokers/mediators are performing important linkage functions as well. At the epistemic level, too, educators’ scepticism toward research serves as a barrier to use, but the level of scepticism most likely varies considerably (a conclusion reached by Hedberg, 2018 , p. 2, whose recommendations relied upon leveraging and connecting educators who are more “evidence-oriented”). Overall, it would seem EIP in MA is skewed top-down in important ways, but there is also recognition of and are some earnest efforts also/instead to promote more bottom up EIP in and across MA schools and educational organisations. Such bottom-up efforts, however, typically do not include the conduct of formal research.

Rheinland-Pfalz (Germany)

The German school governance model can be classified in the quadrant of the egalitarian way. This means rather high social cohesion with lower institutional autonomy, where the idea of ‘managing’ development processes encounters a still bureaucratic administrative context. Meanwhile the German system is characterised by low social regulation; accountability is relatively low stakes. This is the result of a longer development process.

Until the 1990s, the German school system—with its 16 country-specific variants in the federal system—was characterized by an input-oriented governing model with hierarchically organised school supervision and top-down detailed control through laws and decrees. Comparable material and personnel resources and the binding nature of the curricula were seen as a guarantee for the quality and comparability of school results. This corresponds to the logic of conditional programming (Luhmann, 1970 ) in the sense of standardising the results of work by standardising the framework conditions.

As a result of the overall weak German results in Large Scale Assessments and especially since the PISA study, orientation towards a logic of goal programmes (Luhmann, 1970 ) is now also found in more output-oriented steering and control elements. This includes the expectation that schools will orientate themselves more towards educational standards and that they will be accountable within the framework of external evaluation. This has led to a paradigm shift in Germany—in line with the international trend—although elements like competition and the market are not yet apparent or are only beginning to emerge.

In the context of the joint project EviS (Evidence-based School Development), evidence-based knowledge and action in schools in the German federal state RP has been operationalised and descriptively analysed. Evidence was defined as systematically generated, objective and explicit information on the effectiveness of educational processes (Demski et al., 2012 ). The spectrum ranged from scientific empirical studies as well as state-wide assessments and school inspections (external evidence) to peer observation and student feedback (internal evidence) —regarding any sort of information as evidence if it is more or less objective, reliable, and valid (Dormann et al., 2016 ). Figure 2 illustrates the reception and use of different evidence sources by teachers ( N  = 1230) in the EviS-study (van Ackeren et al., 2017 ). The data show comparatively intensive use with regard to internal process-related information sources that are closely related to the teaching practice of teachers, e.g. on the basis of systematic student feedback on teaching (upper right quadrant in Fig. 2 ). The results also point to a relatively intensive use of school subject-related journals. On the other hand, the use of data generated in the context of external instruments (feedback from school inspections, state-wide tests) is significantly lower (bottom left quadrant). Footnote 1 It is noticeable that the approval ratings of school management members tend to be higher than those of the teachers (not illustrated here). This might be explained due to principals’ more comprehensive view of the school as an organisation, but requires further analysis.

This figure depicts the reception and use of different evidence sources by teachers, drawing on findings from van Ackeren et al., 2017 .

In addition to this data, Muslic ( 2017 ) shows in her study concerning the German federal states of Berlin and Baden-Württemberg that in many schools the subject-specific departments are central to the discussion of the results of state-wide assessments. The measures derived include, in particular, process-related activities such as support concepts and didactic agreements. An in-depth analysis of the results with regard to the teaching quality, however, rarely takes place. The exchange between teachers and school management remains largely informal and rather superficial. Only in the case of poor results would the school management intervene. In many cases it is up to teachers themselves whether to use evidence and they are hardly accountable (low social regulation).

As the results from RP show, the new assessment and evaluation instruments are not only little used overall, but are also regarded as comparatively unhelpful. Here the distance to teaching practice may seem bigger than with other instruments, and the descriptive data usually do not provide any explanation or knowledge of change. Furthermore, teachers perceive results, e.g. from state-wide testing, in particular as a starting point for considerations on quality development if they have an objective reference norm, i.e. if there is information about what students can do and in what respect they still have to develop competences (Kühle and van Ackeren, 2012 ). Longitudinal data with individual reference standards should also lead to more acceptance.

The effectiveness of data-based school development improvement measures also requires a competent, trained handling of evidence-based knowledge and its integration into reflective practice. Studies on the different use of data feedback in the context of state-wide assessments in Germany show that schools with high expertise in school improvement can benefit more from data-supported feedback (so-called Matthew effect of accumulated advantage) than those schools that have less competence in this area (Maier et al., 2011 ). Therefore, more systemic effort is needed to support schools’ development processes.

Since 2006, following the so called “PISA-shock” stating unexpectedly weak results for Germany, the introduction of a nationwide educational monitoring concept and data provision on school quality has been associated with the hope of making the actions of schools more effective and thus contributing to the improvement of schools. In the meantime, education policy seems disillusioned with regard to comparative measurements and data use.

On the system level, the usefulness of the data for school and teaching improvement has been questioned by the Standing Conference of the Ministers of Education and Cultural Affairs of the Länder in the Federal Republic of Germany (e.g. Kuhn, 2014 ). There is also increasing debate on how empirical knowledge can be better integrated into education policy, administration and school practice in order to achieve meaningful change. From a German perspective, education is not a directly steerable or controllable, technocratic production process (cf. on the German school system van Ackeren et al., 2015 ; Secretariat of the Standing Conference of the Ministers of Education and Cultural Affairs of the Länder in the Federal Republic of Germany, 2019 ). In this context, state-wide assessments, school inspections and central final examinations in Germany are still low in sanctions and gratuities for schools or teachers.

The partially critical policy view on continuous performance measurements and EIP in schools is also influenced by the above-mentioned research findings concerning the organisational level. In Germany a ‘real’ and meaningful data-driven change is rarely initiated. With reference to “school development through insight” (Kotthoff et al., 2016 , p. 338) rather than control or competition, it will be up to the schools in the end to decide how to react if standards are not achieved within the low stakes environment. There exists an understanding of school as a learning organisation, which has been partially strengthened in its scope for decision-making and action. Schools should be able to adapt to local changes continuously and with the help of data and be able to monitor their actions and the effects they have achieved themselves.

Nevertheless, the coupling of the development of the overall system and the individual school is complex, since German schools and the individual actors in them can decide quite independently how to deal with external interventions; from the individual perspective, the interpretive sovereignty over ‘school quality’ lies with the pedagogical professionals (Klein and Bremm, 2020 ). Overall, there seems to be a lack of fit between a demand for science-oriented reflection from the outside perspective and an experience-based practice within schools. The data also seem to lack ‘cultural significance’ in relation to the specific context of the individual school and individual teacher (Heinrich, 2015 ). Furthermore, school authorities often do not see themselves as managers or at least do not take on this role in practice (Klein and Bremm, 2020 ). This can be seen in Mintrop ( 2015 ), for instance, who speaks of a “public management reform without managers” (p. 790) in Germany. In this way, pressure and unintended side effects are largely avoided, but change is not systematically initiated and supported.

It seems helpful to provide longer-term support, e.g. within the framework of school networks and individual school improvement support. Furthermore, findings from design-based school improvement projects point to the relevance of different opportunities for schools in this context: to understand and re-contextualize externally produced data in their specific individual school situation (Fend, 2009 ), gain expertise in the assessment and interpretations of own data and—most importantly—draw adaptive conclusions and strategies that lead to measurable success in schools (Bremm et al., 2017 ). Accordingly, there is a need for a new balance between gaining knowledge and supporting schools in the reception and use of data, including a stronger focus on the action level (teaching) and the control level (school management). Corresponding competences, such as integrating empirical results into reflective practical experience, should be included in teacher training, which has rarely been the case so far in Germany. School leadership plays an important role for harnessing the benefits of data use (Brown and Flood, 2019 ). German studies have shown that directive or discursive leaders are more likely to favour the discussion of data within school, while delegated school management is more likely to inhibit these processes (e.g. Kronsfoth et al., 2018 ). School principals can also positively influence an organisational climate that is open to evaluations and instruments of performance measurement, as well as promote an overall organisational embedding of the processing and use of evidence (Muslic, 2017 ).

With regard to the current state of discussion in Germany, it can be stated that there is generally an unsatisfactory situation of a cost-intensive system for data generation without systematic utilisation impact. Therefore the ‘Standing Conference of the Ministers of Education and Cultural Affairs’ (KMK) has changed its overall monitoring strategy for education in order to support a more transfer-oriented development through more relevant and explanatory knowledge for administration and schools and to process research findings more systematically as well as to better support schools (Kultusministerkonferenz, 2015 ). It remains questionable, however, whether a more binding requirement for practitioners’ data use is needed, for example regarding comprehensive and evaluated quality management systems in schools and by strengthening the role of supervision and leadership.

In this study, we set out to further examine the question of how to bring about more consistent, EIP in education. To do so, we undertook a novel approach, examining EIP comparatively across four geographical contexts (Catalonia, England, Massachusetts, and RP) through the use of particular theoretical perspectives/frameworks. We applied a cohesion/regulation matrix and institutional theory to frame our analyses of factors influencing evidence use across these contexts. Our aims in doing so were twofold. First, we hoped to generate provisional insights related to fostering more/better EIP in education. Second, we aimed to achieve and share process insights related to this undertaking (i.e., to inform those who might wish to emulate and improve upon these approaches). Lastly, given the aims of this special edition, we sought to ascertain whether there are generalisable lessons in this study (either in terms of findings, or process) that could be applied to improve EIP (and/or its study) in other sectors.

First, reflecting on our research process, here we offer several observations. Most generally, our diverse author team found the dual analytical approach to support our aims— it directed our attention in common (albeit not entirely overlapping) ways, providing us with structure through which to make sense of the level and nature of EIP that existed in these contexts. More specifically, the matrix offered us the chance to understand the uniqueness of each educational system and its specific national/local flavours, while the institutional theory provided the chance to find commonalities between these systems. Indeed, the intersection of these two axes of analysis provided an understanding of enablers and barriers to EIP, as well as agents involved in each one of them. Accordingly, the dual frame has been (and can be) helpful for accurately diagnosing key aspects and levels. In turn, we suggest decision-makers might be able to apply the frame (and/or insights derived from others’ applications of it) to “develop appropriate context-specific rather than one-size-fits-all packages of support to stimulate improvements” (Chapman, 2019 , p. 5).

Nevertheless, in this instance, we were hampered by certain challenges, which mostly can be understood as study limitations that can be overcome by others. For example, we experienced unevenness across cases in terms of the level and depth of data and evidence available for analysis. Conceivably, a future study could utilise the same frame, but could proactively collect common data across contexts—and could do so at multiple levels of the respective systems—rather than relying, as we did, on extant data and literature related to the contexts. We expect such an approach would further the comparative process. Also, and in line with Martin and Williams’ ( 2019 ) advice for scholars in these areas, we found institutional theory to provide a quite useful and illuminating, if also complicated, lens for analysing and understanding the relationships between evidence and practice in (and across) contexts. Although ultimately beneficial, it required substantial work for us as researchers to develop a shared understanding of the various elements and how they applied. In this aspect as well, though, we strongly suspect a pre-planned, active comparative study of evidence-use, using this dual approach, would go far in terms of understanding EIP variation (and, thus, suggesting potential approaches to improve EIP). Further, we suggest such research endeavours would be useful beyond education, or perhaps could support the simultaneous study of more than one sector; these lenses can be illuminative irrespective of sector, as nuances related to norms, policies, traditions, etc. (beyond mere differences in evidentiary bases across sectors) will go far in terms of understanding EIP patterns/variation.

Beyond the process, we also suggest this study’s findings yield some tentative insights in terms of EIP, both for educators/educational scholars and for those outside the education area. Particularly for the latter group, certain facts may seem counterintuitive. For example, why is it that, in a system in which teachers have a great deal of autonomy like Catalonia, teachers report infrequently relying upon research to guide their practice? Here, some sector-specific analysis/understanding is supportive. For instance, as Cohen and Mehta ( 2017 , p. 649) observe, teaching has across many (though not all) contexts “[failed] to crystalize as a full-fledged profession”, a feature that renders teaching/learning “vulnerable to lay views of education and reform, as well as to inherited patterns of practice.” Education as a field is challenged by manifold and often competing/conflicting goals, multiple constituencies and often fragmented systems (Labaree, 1997 ), which “complexifies the idea of a unified body of research informing classroom practice” (Lubienski, 2020 , p. 183). Indeed, as compared to professions like law or medicine, education has fewer agreed upon truths and shared definitions of problems and solutions (Willingham, 2012 ). In part, this is also a reflection of the inherent complexities and contingencies around teaching and learning. Altogether, one might better understand why educators who have considerable freedom may not direct it in consistent easy to spot ‘research-informed’ ways. As such, education is indeed somewhat unique in terms of featuring relatively weak and conditional research-practice links (Lubienski, 2020 ).

Still, there are certain paths forward toward bringing about more/better evidence use in education. In education (much like in other fields), it helps to recognise that research evidence is but one of the potential influences on practice (Farley-Ripple et al., 2018 ). Indeed, as Cain et al. ( 2019 , p. 1074) summarise, there is now “near-universal agreement that research-generated insights are an insufficient basis for practice”. Above all else, educators—like other practitioners and policymakers—want to be able to confidently make decisions about problems/issues that are important to them. Ultimately, it is realistic first to understand research knowledge represents just one of several forms of knowledge educators might draw upon as they go about their work. Accordingly, its professional use is not pre-given but is contingent on a host of favourable features and conditions, set across multiple levels. As such, and although none of the cases addressed in this study represents a utopia in terms of professional evidence use, individually and collectively they are assistive in drawing out some such features that can help move educational systems toward more desirable states.

The England and MA cases, for example, show how strong accountability pressures (via inspection and high-stakes assessments, respectively) certainly can coax educators to consistently focus on particular forms of data and research. MA educators are particularly attentive to students’ performance on annual high-stakes state tests, while English educators are considerably driven by official school inspections. However, there can be considerable costs associated with such approaches, roughly summarised as ‘what gets measured matters’ (and the reverse: what is not measured might be underemphasised or ignored, reduced/cut from programming, etc.). Accordingly, we offer that when accountability systems are in place, specific details (e.g., assessment areas, format, foci, speed, and quality of feedback) are salient. RP offers a useful point of comparison here, in that their external assessment data appear to be attracting substantially less educator attention; this fact is most likely explained by the relative weakness of sanctions and gratuities associated with these measures. Depending on one’s vantage point, this might be construed as a virtue or a vice. On the positive side, for example, perhaps RP educators are freer to direct their attention toward locally important data and research (e.g., bottom-up evidence generation/use), and as such are utilising the external data only to the extent that it is perceived to add value to their decision-making. On the other side, results show that low stakes accountability and higher degrees of autonomy in Germany come with lower levels and less infrastructure for support that would help practitioners with understanding, discussing and recontextualizing these data.

The MA case also shows a relatively strong and layered infrastructure for supporting evidence use, including a state-level research director and department focused on planning and research. Moreover, with their recent report, they too are now positioned to be more evidence-informed about how to promote more routine and deep evidence use in schools (i.e., they can tailor their activities and processes to what they have been learning from MA educators regarding actual evidence use, supply, and demand).

Conaway ( 2020 ), formerly research director for MA, provides research-grounded and practice-grounded insights regarding how to potentially move toward next level evidence use. Broadly, she writes

If we want research to matter…we need to devote resources to building relationships and strengthening organisational practices, in service of building organisations that learn (p. 2).

More specifically, she highlights the importance of several specific aspects and structures/arrangements, most of which are evident and/or incipient in one or more of the cases we reviewed. For example, she highlights the potential of learning networks (see the English and Catalonia cases to learn more about their relatively longstanding and more recent embrace of PLNs, respectively), embedded research directors (see MA), and research-practice partnerships (relatively prevalent in MA).

Perhaps most fundamental, and perhaps best exemplified presently in England and in RP, is the overarching goal to develop and support educational ‘organisations that learn.’ To foster such organisations on a broad scale, we suspect, ultimately will require that various conditions be simultaneously met—both philosophically and materially, and at multiple levels within complex educational systems such as those we have profiled herein. In other words, to truly approximate an educational ‘evidence use utopia’ will require attention to both institutional and social factors, as highlighted via our analyses. Additionally, we suggest, it will require establishing conditions in which teaching is treated and experienced as a full-fledged profession (see Darling-Hammond et al., 2017 , for jurisdictions in which this is a reality, or nearly so). Ultimately, what we suggest should be envisioned and worked toward are coherent systems in which teachers and educational leaders routinely and effectively can access and integrate research evidence with other forms of knowledge/knowing at the “point of use” (Nutley et al., 2019 , p. 242), as they are making consequential educational decisions.

Data availability

All datasets analysed or generated are indicated in the paper.

It should be noted that in RP, as in most Länder, there is no longitudinal individual data at pupil level available that could show developments in a more differentiated way.

Act on the Education Quality Improvement (LOMCE, 2013). Ley Orgánica 8/2013, de 9 de diciembre, para la mejora de la calidad educativa. https://www.boe.es/buscar/pdf/2013/BOE-A-2013-12886-consolidado.pdf . Accessed 23 Apr 2019

Act on the Right to Education (LODE, 1985). Ley Orgánica 8/1985. https://www.boe.es/eli/es/lo/1985/07/03/8/con Accessed 23 Apr 2019

Azorín C (2019) The emergence of professional learning networks in Spain. J Prof Cap Community 4(1):36–51

Google Scholar  

Bauman Z (2012) Liquid modernity. Polity Press, Cambridge

Bell M, Cordingley P, Isham C, Davis R (2010) Report of professional practitioner use of research review: practitioner engagement in and/or with research. CUREE, GTCE, LSIS & NTRP, Coventry. Available via Curee. http://www.curee-paccts.com/node/2303 . Accessed 24 Aug 2019

Best A, Holmes B (2010) Systems thinking, knowledge and action: towards better models and methods. Evid Policy 6(2):145–159

Bremm N, Eiden S, Neumann C, Webs T, Ackeren I van, Holtappels HG (2017). Evidenzbasierter Schulentwicklungsansatz für Schulen in herausfordernden Lagen. Zum Potenzial der Integration von praxisbezogener Forschung und Entwicklung am Beispiel des Projekts „Potenzialeentwickeln—Schulen Stärken“ [Evidence-based school development approach for schools in challenging circumstances. On the potential of integrating practice-oriented research and developmentusing the example of the project "Developing potential—strengthening schools”]. In: Manitius V, Dobbelstein P (eds) Schulentwicklungsarbeit in herausfordernden Lagen [School development in challenging circumstances]. Waxmann, Münster, pp. 140–158

Brown C (2017) Research learning communities: how the RLC approach enables teachers to use research to improve their practice and the benefits for students that occur as a result. Res All 1(2):387–405. https://doi.org/10.18546/RFA.01.2.14

Article   MathSciNet   Google Scholar  

Brown C (2020) The networked school leader: how to improve teaching and student outcomes using learning networks. Emerald, London

Brown C, Flood J (2019) Formalise, prioritise and mobilise: how school leaders secure the benefits of professional learning networks. Emerald, London

Brown C, Schildkamp K, Hubers M (2017) Combining the best of two worlds: a conceptual proposal for evidence-informed school improvement. Educ Res 59(2):154–172

Brown C, Malin J (2017) Five vital roles for school leaders in the pursuit of evidence of evidence-informed practice. Teachers College Record

Burns T, Schuller T (2007) Evidence in education: linking research and policy. Organisation for Economic Co-operation and Development

Busby E (2019) We are fed up waiting: more than 5000 headteachers set to march on Downing Street to demand more cash for schools. Available via Independent. https://www.independent.co.uk/news/education/education-news/school-funding-cuts-headteachers-downing-street-march-westminster-boris-johnson-a9020756.html . Accessed 24 Aug 2019

Cain T (2015) Teachers’ engagement with research texts: beyond instrumental, conceptual or strategic use. J Educ Teach 41(5):478–492

Cain T, Brindley S, Brown C, Jones G, Riga F (2019) Bounded decision-making, teachers’ reflection, and organisational learning: how research can inform teachers and teaching. Br Educ Res J 45(5):1072–1087. Early online access

Camacho M (2016) Vers una cultura de la innovació: reptes i oportunitats en el marc del sistema educatiu català [Towards an innovation culture: challenges and opportunities in the catalán educational system] In: Vilalta M (ed) Reptes de l’educació a Catalunya: 2015 [Challenges of Catalonian education: 2015]. Fundació Jaume Bofill, Barcelona, pp. 105–138

Catalan Government, Departament d’Educació (2019) Documents per a l’organització i la gestió dels centres. http://educacio.gencat.cat/documents/IPCNormativa/DOIGC/PEC_Avaluacio_centre.pdf . Accessed 24 Aug 2019

Chapman C (2019) Making sense of reform: where next for Scottish education? Association of Directors of Education in Scotland: The Staff College, Manchester

Civís M, Longás J (2015) La colaboración interinstitucional como respuesta al desafío de la inclusión socioeducativa: Análisis de 4 experiencias de trabajo en red a nivel local en Cataluña [Interinstitutional collaboration in response to the challenge of socio-educational inclusion: analysis of 4 experiences of networking at a local level in Catalonia]. Educación XXI 18(1), 213–236

Cohen DK, Mehta JD (2017) Why reform sometimes succeeds: Understanding the conditions that produce reforms that last. American Educational Research Journal 54(4):644–690

Coldwell M, Greany T, Higgins S, Brown C, Maxwell B, Stiell B, Stoll L, Willis B, Burns H (2017) Evidence-informed teaching: an evaluation of progress in England. Department for Education, London

Conaway C (2020) Maximizing research use in the world we actually live in: relationships, organizations, and interpretation. Educ Financ Policy 15(1):1–10

Darling-Hammond L (2010) The flat world and education. Teachers College Press, New York

Darling-Hammond L, Burns D, Campbell C, Goodwin AL, Hammerness K, Low EL, McIntyre A, Sato M, Zeichner K (2017) Empowered educators: How high-performing systems shape teaching quality around the world. John Wiley & Sons, San Francisco

Datnow A, Park V, Kennedy-Lewis B (2013) Affordances and constraints in the context of teacher collaboration for the purpose of data use. J Educ Adm 51(3):341–362

Díaz Costa MA (2009) Impacto de la investigación educativa en la práctica docente. Dissertation, Universidad de Granada

Díaz-Gibson J, Civís M, Carrillo E, Cortada M(2015) El liderazgo y la gobernanza colaborativa en proyectos educativos comunitarios [Leadership and collaborative governance in community education projects] Pedogog Soc Rev Interuniv 26:59–83

Dormann C, Binnewies C, Koch AR, Ackeren IV, Clausen M, Preisendörfer P, Zlatkin-Troitschanskaia O (2016) Transferring best evidence into practice: Assessment of evidence-based school management. J Educ Res online 8(3):14–38

EduCaixa (2019) EduCaixa Programme. https://www.educaixa.com/es/ Accessed 23 Aug 2019

EEF (2014) EEF launches £1.5 million fund to improve use of research in schools. Available via EEF. https://educationendowmentfoundation.org.uk/news/eef-launches-15-million-fund-to-improve-use-of-research-in-schools / Accessed 27 Aug 2019

EEF (2018) EEF toolkit new survey shows 59% of senior leaders using it to inform their decisions. Available via EEF. https://educationendowmentfoundation.org.uk/news/eef-toolkit-new-survey-shows-59-of-senior-leaders-using-it-to-inform-their-/ . Accessed 27 Aug 2019

Ehren M (2018) Accountability and trust: two sides of the same coin? Invited talk as part of the University of Oxford’s: Intelligent accountability symposium II, Oxford, 12 Dec 2018

Farley-Ripple E, May H, Karpyn A, Tilley K, McDonough K (2018) Rethinking connections between research and practice in education: a conceptual framework. Educ Res 47(4):235–245

Fend H (2009) Neue Theorie der Schule: Einführung in das Verstehen von Bildungssystemen. [New school theory: introduction to understanding education systems]. Springer-Verlag, Berlin

Gairin J (2015) Autonomy and school management in the Spanish context. J Educ Cult Psychol Stud 11:103–117

Galdin-O’Shea H (2015) Leading ‘disciplined enquiries’ in schools. In C. Brown (ed) Leading the use of Research & Evidence in schools, (London, IOE Press) (pp. 91–106)

Godfrey D (2016) Leadership of schools as research-led organizations in the English educational environment: cultivating a research engaged school culture. Educ Manag Adm Leadersh 44(2):301–321

Gough D, Tripney J, Kenny C, Buk-Berge E (2011) Evidence informed policymaking in education in Europe. EIPEE final project report

Graves S, Moore A (2017) How do you know what works, works for you? An investigation into the attitudes of school leaders to using research evidence to inform teaching and learning in schools. School Leadersh Manag 38(3):259–277

Greany T (2014) Are we nearly there yet? Progress, issues and possible next steps for a self-improving school system. IOE Press, London

Greany T (2015) How can evidence inform teaching and decision making across 21,000 autonomous schools? Learning from the journey in England. In: Brown C (Ed) Leading the use of research & evidence in schools. IOE Press, London, pp. 11–29

Greany T, Earley P (2018) The paradox of policy and the quest for successful leadership. Prof Dev Today 19(3&4):6–12

Greany T (2017) Karmel Oration: leading schools and school systems in times of change—a paradox and a quest, Paper presented at Australian Council for Educational Research (ACER) Research Conference, Melbourne, Australia, 28–29 August 2017.

Hackmann DG, Malin JR, Ahn J (2019) Data use within an education-centered cross-sector collaboration. J Educ Adm 57(2):118–133. https://doi.org/10.1108/JEA-08-2018-0150

Article   Google Scholar  

Hedberg C (2018) Evidence use in Massachusetts school districts. http://www.doe.mass.edu/research/reports/ Accessed 4 Oct 2019

Heinrich M (2015) Zur Ambivalenz der Idee evidenzbasierter Schulentwicklung. Das Beispiel“Schulinspektion”-fortschrittlicher Rückschritt oder Innovation? [On the ambivalence of the idea of evidence-based school development. The example of “school inspection”—progressive regression or innovation?] Z Pädagog 61(6):778–792

Higher Education Funding Council, England (HEFCE) (2011) Assessment framework and guidance on submissions. HEFCE

Hood C (1998) The art of the state, culture rhetoric and public management. Clarendon Press, Oxford

Horsford SD, Scott JT, Anderson GL (2018) The politics of education policy in an era of inequality: possibilities for democratic schooling. Routledge, Abingdon

Howland G (2015) Structural reform: the experience of ten schools driving the development of an all-age hard federation across a market town in northern England. Manag Educ 29(1):25–30

Ion G, Diaz A, Gairin J (2019) Developing the teachers’ professional capital through research informed innovations. The experience of primary education teachers in Catalonia. Paper presented to the European educational research conference, Hamburg, 3–6 September 2019

Ion G, Gairín J (2019). Research informed practices in Spanish schools: research report. Unpublished document. http://pagines.uab.cat/pbe/ca

Kahneman D (2011) Thinking, fast and slow. Allen Lane, London

Klein ED, Bremm N (2020) Schulentwicklung im manageriell geprägten System—Das Verhältnis von Schulleitung und Schulaufsicht in den USA [School development in a managerial system—the relationship between school management and school supervision in the USA]. In: Klein ED, Bremm N (eds) Unterstützung, Kooperation, Kontrolle. Zum Verhältnis von Schulaufsicht und Schulleitungen in der Schulentwicklung (Arbeitstitel). Educational Governance. Springer VS

Kotthoff HG, Böttcher W, Nikel J (2016) Die‚ Schulinspektion‘ zwischen Wirkungshoffnungen und Wirksamkeit [The ‘school inspection’ between hopes of effect and effectiveness.]. In: Altrichter H, Maag Merki K (eds) Handbuch Neue Steuerung im Schulsystem, 2nd edn. Springer VS, Wiesbaden, pp. 325–359

Kronsfoth K, Muslic B, Graf T, Kuper H (2018) Der Zusammenhang zwischen Führungsdimensionen in der Schulleitung und der Nutzung von Ergebnisrückmeldungen aus Vergleichsarbeiten [The relationship between leadership dimensions in school management and the use of feedback from comparative statewide assessments]. Die Dtsch Sch 110(1):47–64

Kühle B, van Ackeren I (2012) Wirkungen externer Evaluationsformen für eine evidenzbasierte Schul-und Unterrichtsentwicklung [Effects of external forms of evaluation for evidence-based school and teaching development]. In: Ratermann M, Stöbe-Blossey S (eds) Governance von Schulund Elementarbildung. VS Verlag für Sozialwissenschaften, pp. 45–62

Kuhn HJ (2014) Anspruch, Wirklichkeit und Perspektiven der Gesamtstrategie der KMK zum Bildungsmonitoring. [Entitlement, reality and perspectives of the overall strategy of the KMK [Standing conference of the Ministers of Education and Cultural Affairs of the Länder in the Federal Republic of Germany] on education system monitoring]. Die Dtsch Sch 106(4):414–426

Kultusministerkonferenz (2015) Text der überarbeiteten Gesamtstrategie zum Bildungsmonitoring [Text of the revised overall strategy for education monitoring]. Available via kmk. https://www.kmk.org/fileadmin/Dateien/pdf/PresseUndAktuelles/2015/Gesamtstrategie_Endfassung_DOK.pdf Accessed 29 Aug 2019

Labaree DF (1997) Public goods, private goods: the American struggle over educational goals. Am Educ Res J 34(1):39–81

Lai M, Wilson A, McNaughton S, Hsia S (2014) Improving achievement in secondary schools: impact of a literacy project on reading comprehension and secondary schools qualifications. Read Res Q 49(3):305–334

Lee WO, Napier DB, Manzon M (2014) Does context still matter? The dialectics of comparative education. Asia Pac J Educ Publ 34(2):139–152

Lubienski C (2020) The future of research use? In: Malin J, Brown C (eds) The role of knowledge brokers in education: connecting the dots between research and practice. Routledge, New York, pp. 1–12

Luhmann N (1970) Soziologische Aufklärung [Sociological enlightenment]. Soziologische Aufklärung 1. VS Verlag für Sozialwissenschaften, Wiesbaden, pp. 66–91

Maier U, Metz K, Bohl T, Kleinknecht M, Schymal M (2011) Effekte testbasierter Rechenschaftslegung auf die datenbasierte Schul-und Unterrichtsentwicklung [Effects of test-based accountability on data-based school and classroom development]. Z Pädagog 56(1):112–128

Malin JR, Brown C, Trubceac AS (2018) Going for broke: A multiple-case study of brokerage in education. AERA Open 4(2):1–14

Malin JR, Paralkar V (2017) Educational knowledge brokerage and mobilization: the Marshall Memo case. Int J Educ Policy Leadersh 12(7):1–20

Malin JR, Brown C (2020) Joining worlds. In: Malin J, Brown C (eds) The role of knowledge brokers in education: connecting the dots between research and practice. Routledge, New York, pp. 1–12

Marchesi A, Martin E (2002) Evaluation in secondary education. snapshot from a controversial era. Institution IDEA, Madrid

Martin GP, Williams O (2019) Evidence and service delivery. In: Boaz A, Davies H, Fraser A, Nutley S (eds) What works now? Evidence-informed policy and practice. Policy Press, Bristol, UK, pp. 41–57

Martinez X (2019) La recerca educativa, també al servei de les escoles. Models emergents per a la millora de l’impacte de la recerca educativa. In: Riera J (ed) Reptes de l’educació a Catalunya. Anuari 2018. Fundació Jaume Bofill, Barcelona, pp. 536–577

Mincu M (2014) Inquiry paper 6: teacher quality and school improvement—what is the role of research? In: the British Educational Research Association/The Royal Society for the encouragement of Arts, Manufactures and Commerce (eds) The role of research in teacher education: reviewing the evidence. Available via bera. https://www.bera.ac.uk/wp-content/uploads/2014/02/BERA-RSA-Interim-Report.pdf , Accessed 8 Aug 2019

Mintrop R (2015) Public management reform without managers: the case of German public schools. Int J Educ Manag 29(6):790–795

Demski D, Rosenbusch C, van Ackeren I, Clausen M & Schmidt U (2012). Steuerung von Schule durch evidenzbasierte Einsicht? Konzeption und erste Befunde des Forschungsverbundes EviS [Governing school through evidence-based insight? Conception and first findings of the research project EviS]. In: Hornberg S & Parreira do Amaral M (eds) Deregulierung im Bildungssystem [Deregulation in the educational system]. Waxmann, Münster, pp. 131–150

Murillo FJ (2006) Retos de la innovación para la investigación educativa. In: Escudero T, Correa AD (ed) Innovación e investigación educativa: algunos ámbitos relevantes. La Muralla, Madrid, pp. 23–54

Murillo J, Perines H (2017) Cómo los docentes no universitarios perciben la investigación educativa. Rev complut educ 28(1):81–99

Muslic B (2017) Kopplungen und Entscheidungen in der Organisation Schule [Interconnections and decisions in the organization school]. Springer, Wiesbaden

Nutley S, Davies H, Hughes J (2019) Assessing and labelling evidence. In: Boaz A, Davies H, Fraser A, Nutley S (eds) What works now? Evidence-informed policy and practice. Policy Press, Bristol, UK, pp. 225–246

Oancea A (2018) The practice of educational research. International handbook of philosophy of education. Springer, Cham

OECD (2020) Education at a glance: teachers’ teaching and working time. OECD Education Statistics (database), https://doi.org/10.1787/d3ca76db-en . Accessed 12 July 2020

Papay JP, Mantil A, Murnane RJ, An L, Donohue K, McDonough A (2020) Lifting all boats? Accomplishments and challenges from 20 years of education reform in Massachusetts. Educational Opportunity in MA, Brown University, Providence

Perines H (2017) Las murallas invisibles entre la investigación educativa y los docentes. Cienc Educ 1(1):11–21

Perines H (2018) ¿Por qué la investigación educativa no impacta en la práctica docente? Estud sobre educ 34:9–27

Piazza P (2017) Neo-democracy in educational policy-making: a critical case study of neoliberal reform in Massachusetts. J Educ Policy 32(2):211–233

Powell WW, DiMaggio P (1991) The neo institutional analysis in organizational analysis. University of Chicago Press, London

Robinson V (2017) Foreword. In: Earley P, Greany T (eds) School leadership and system reform in the 21st Century. Bloomsbury, London

Rose J, Thomas S, Zhang L, Edwards A, Augero A, Rooney P (2017) Research Learning Communities: Evaluation report and executive summary. Available via Education Endowment Foundation. https://educationendowmentfoundation.org.uk/public/files/Projects/Evaluation_Reports/Research_Learning_Communities.pdf . Accessed 15 Aug 2019

Sahlberg P (2016) The global educational reform movement and its impact on schooling. In: Mundy K, Green A, Lingard B, Verger T (eds) The handbook of global education policy. John Wiley and Son, Ltd, Chichester, pp. 128–144

Secretariat of the Standing Conference of the Ministers of Education and Cultural Affairs of the Länder in the Federal Republic of Germany (2019) The Education System in the Federal Republic of Germany 2016/2017. A description of the responsibilities, structures and developments in education policy for the exchange of information in Europe. KMK, Bonn. retrieved from KMK. https://www.kmk.org/fileadmin/Dateien/pdf/Eurydice/Bildungswesen-engl-pdfs/dossier_en_ebook.pdf , Accessed 5 Jul 2020

Shober AF (2016) Individuality or community? Bringing assessment and accountability to K-16 education. In: Loss C, McGuinn P (eds) The convergence of K-12 and higher education: policies and programs in a changing era. Harvard Education Press, Cambridge, MA, pp. 67–86

Tseng V, Coburn C (2019) Using evidence in the US. In: Boaz A, Davies H, Fraser A, Nutley S (eds) What works now? Evidence informed policy and practice. Policy Press, Bristol, UK, pp. 351–368

van Ackeren I, Klemm K, Kühn SM (2015) Entstehung, Struktur und Steuerung des deutschen Schulsystems. Eine Einführung. [Development, structure and governance of the German school system. An introduction.]. Springer Fachmedien, Wiesbaden

van Ackeren I, Demski D, Klein ED (2017) Entwicklungsprobleme Neuer Steuerung imSchulsystem. Ein systematisierender Überblick unter besonderer Berücksichtigung des evidenzbasierten Steuerungsanspruchs [Development problems of new control systems in the school system. A systematizing overview with special consideration of the evidence-based control requirement]. In: Holtappels HG (ed) Entwicklung und Qualität des Schulsystems. Neue empirische Befunde und Entwicklungstendenzen [Development and quality of the school system. New empirical findings and developmental tendencies]. Waxmann, Münster, pp. 241–259

Van Geel M, Keuning T, Visscher A, Fox JP (2016) Assessing the effects of a school-wide data-based decision-making intervention on student achievement growth in primary schools. Am Educ Res J 53(2):360–394

Vanlommel K, Van Gasse R, Vanhoof J, Van Petegem P (2017) Teachers’ decision-making: data based or intuition driven? Int J Educ Res 83:75–86

Walker M, Nelson S, Bradshaw S, Brown C (2019) Teachers’ engagement with research: what do we know? A research briefing. https://www.nfer.ac.uk/teachers-engagement-with-research-what-do-we-know-a-research-briefing/ . Accessed 30 Aug 2019

Whitty G, Wisby E (2017) Is evidence-informed practice any more feasible than evidence-informed policy? Presented at the British Educational Research Association annual conference, Sussex, 5–7 September 2017

Willingham DT (2012) When can you trust the experts?: how to tell good science from bad in education. John Wiley & Sons, Hoboken

Wise D (2015) In measures of economic strength, social cohesion, and fiscal position, blue and purple states are generally outperforming red states [Blog post]. LSE USCentre. Available via blogs.lse. https://blogs.lse.ac.uk/usappblog/2015/03/20/in-measures-of-economic-strength-social-cohesion-and-their-fiscal-position-blue-and-purple-states-are-generally-outperforming-red-states/ . Accessed 10 Dec 2019

Download references

Acknowledgements

We acknowledge support by the Open Access Publication Fund of the University of Duisburg-Essen.

Author information

Authors and affiliations.

Miami University, Oxford, OH, USA

Joel R. Malin & Gul Muhammad Rind

Durham University, Durham, England

Chris Brown, Ruth Luzmore & Jane Flood

Universitat Autònoma de Barcelona, Barcelona, Spain

Georgeta Ion

University Duisburg-Essen, Duisburg, Germany

Isabell van Ackeren

Zurich University of Teacher Education, Zurich, Switzerland

You can also search for this author in PubMed   Google Scholar

Corresponding authors

Correspondence to Joel R. Malin or Isabell van Ackeren .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Malin, J.R., Brown, C., Ion, G. et al. World-wide barriers and enablers to achieving evidence-informed practice in education: what can be learnt from Spain, England, the United States, and Germany?. Humanit Soc Sci Commun 7 , 99 (2020). https://doi.org/10.1057/s41599-020-00587-8

Download citation

Received : 10 December 2019

Accepted : 26 August 2020

Published : 17 September 2020

DOI : https://doi.org/10.1057/s41599-020-00587-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies