literature review on research methodologies

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Don't submit your assignments before you do this

The academic proofreading tool has been trained on 1000s of academic texts. Making it the most accurate and reliable proofreading tool for students. Free citation check included.

Try for free

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved August 21, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Libraries | Research Guides

Literature reviews, what is a literature review, learning more about how to do a literature review.

• Planning the Review
• The Research Question
• Choosing Where to Search
• Organizing the Review
• Writing the Review

A literature review is a review and synthesis of existing research on a topic or research question. A literature review is meant to analyze the scholarly literature, make connections across writings and identify strengths, weaknesses, trends, and missing conversations. A literature review should address different aspects of a topic as it relates to your research question. A literature review goes beyond a description or summary of the literature you have read. 

• Sage Research Methods Core This link opens in a new window SAGE Research Methods supports research at all levels by providing material to guide users through every step of the research process. SAGE Research Methods is the ultimate methods library with more than 1000 books, reference works, journal articles, and instructional videos by world-leading academics from across the social sciences, including the largest collection of qualitative methods books available online from any scholarly publisher. – Publisher

• Next: Planning the Review >>
• Last Updated: Jul 8, 2024 11:22 AM
• URL: https://libguides.northwestern.edu/literaturereviews

Research Methods

• Getting Started
• Literature Review Research
• Research Design
• Research Design By Discipline
• SAGE Research Methods
• Teaching with SAGE Research Methods

Literature Review

• What is a Literature Review?
• What is NOT a Literature Review?
• Purposes of a Literature Review
• Types of Literature Reviews
• Literature Reviews vs. Systematic Reviews
• Systematic vs. Meta-Analysis

Literature Review  is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

Also, we can define a literature review as the collected body of scholarly works related to a topic:

• Summarizes and analyzes previous research relevant to a topic
• Includes scholarly books and articles published in academic journals
• Can be an specific scholarly paper or a section in a research paper

The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic

• Help gather ideas or information
• Keep up to date in current trends and findings
• Help develop new questions

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Helps focus your own research questions or problems
• Discovers relationships between research studies/ideas.
• Suggests unexplored ideas or populations
• Identifies major themes, concepts, and researchers on a topic.
• Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
• Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
• Indicates potential directions for future research.

All content in this section is from Literature Review Research from Old Dominion University 

Keep in mind the following, a literature review is NOT:

Not an essay 

Not an annotated bibliography  in which you summarize each article that you have reviewed.  A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.

Not a research paper   where you select resources to support one side of an issue versus another.  A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.

A literature review serves several purposes. For example, it

• provides thorough knowledge of previous studies; introduces seminal works.
• helps focus one’s own research topic.
• identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
• suggests previously unused or underused methodologies, designs, quantitative and qualitative strategies.
• identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
• helps the researcher avoid repetition of earlier research.
• suggests unexplored populations.
• determines whether past studies agree or disagree; identifies controversy in the literature.
• tests assumptions; may help counter preconceived ideas and remove unconscious bias.

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

Robinson, P. and Lowe, J. (2015),  Literature reviews vs systematic reviews.  Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393

What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California

Systematic review or meta-analysis?

A  systematic review  answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.

A  meta-analysis  is the use of statistical methods to summarize the results of these studies.

Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:

• clearly stated objectives with pre-defined eligibility criteria for studies
• explicit, reproducible methodology
• a systematic search that attempts to identify all studies
• assessment of the validity of the findings of the included studies (e.g. risk of bias)
• systematic presentation, and synthesis, of the characteristics and findings of the included studies

Not all systematic reviews contain meta-analysis. 

Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.  More information on meta-analyses can be found in  Cochrane Handbook, Chapter 9 .

A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies.  It is a systematic review that uses quantitative methods to synthesize and summarize the results.

An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings.  Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted.  In that case, an integrative review is an appropriate strategy. 

Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.

• << Previous: Getting Started
• Next: Research Design >>
• Last Updated: Jul 15, 2024 10:34 AM
• URL: https://guides.lib.udel.edu/researchmethods

Warning: The NCBI web site requires JavaScript to function. more...

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

• DOI: 10.1016/J.JBUSRES.2019.07.039
• Corpus ID: 201321905

Literature review as a research methodology: An overview and guidelines

• Published in Journal of business research 1 November 2019

4,283 Citations

The art of writing literature review: what do we know and what do we need to know, review research as scientific inquiry.

• Highly Influenced

Methodology of Writing Literature Review on Management and Business: Digest of Overseas Highly-Cited Articles

• 10 Excerpts

How to conduct systematic literature reviews in management research: a guide in 6 steps and 14 decisions

• 23 Excerpts

On Quality 4.0 in project-based industries

International business research on “emerging economies”: a critical review and recommendations, how to undertake an impactful literature review: understanding review approaches and guidelines for high-impact systematic literature reviews, management accounting and new product development: a systematic literature review and future research directions, introduction: the role and relevance of literature reviews and research in the asia pacific, advancing knowledge through literature reviews: ‘what’, ‘why’, and ‘how to contribute’, 32 references, towards a methodology for developing evidence-informed management knowledge by means of systematic review, review articles: purpose, process, and structure, analyzing the past to prepare for the future: writing a literature review, a typology of reviews: an analysis of 14 review types and associated methodologies., developing a framework for transferring knowledge into action: a thematic analysis of the literature, defining service innovation: a review and synthesis, the effectiveness of customer participation in new product development: a meta-analysis, the integrative review: updated methodology., identifying categories of service innovation: a review and synthesis of the literature, diffusion of innovations in service organizations: systematic review and recommendations., related papers.

Showing 1 through 3 of 0 Related Papers

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jun 20, 2024 9:08 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

• Privacy Policy

Home » Literature Review – Types Writing Guide and Examples

Literature Review – Types Writing Guide and Examples

Table of Contents

Literature Review

Definition:

A literature review is a comprehensive and critical analysis of the existing literature on a particular topic or research question. It involves identifying, evaluating, and synthesizing relevant literature, including scholarly articles, books, and other sources, to provide a summary and critical assessment of what is known about the topic.

Types of Literature Review

Types of Literature Review are as follows:

• Narrative literature review : This type of review involves a comprehensive summary and critical analysis of the available literature on a particular topic or research question. It is often used as an introductory section of a research paper.
• Systematic literature review: This is a rigorous and structured review that follows a pre-defined protocol to identify, evaluate, and synthesize all relevant studies on a specific research question. It is often used in evidence-based practice and systematic reviews.
• Meta-analysis: This is a quantitative review that uses statistical methods to combine data from multiple studies to derive a summary effect size. It provides a more precise estimate of the overall effect than any individual study.
• Scoping review: This is a preliminary review that aims to map the existing literature on a broad topic area to identify research gaps and areas for further investigation.
• Critical literature review : This type of review evaluates the strengths and weaknesses of the existing literature on a particular topic or research question. It aims to provide a critical analysis of the literature and identify areas where further research is needed.
• Conceptual literature review: This review synthesizes and integrates theories and concepts from multiple sources to provide a new perspective on a particular topic. It aims to provide a theoretical framework for understanding a particular research question.
• Rapid literature review: This is a quick review that provides a snapshot of the current state of knowledge on a specific research question or topic. It is often used when time and resources are limited.
• Thematic literature review : This review identifies and analyzes common themes and patterns across a body of literature on a particular topic. It aims to provide a comprehensive overview of the literature and identify key themes and concepts.
• Realist literature review: This review is often used in social science research and aims to identify how and why certain interventions work in certain contexts. It takes into account the context and complexities of real-world situations.
• State-of-the-art literature review : This type of review provides an overview of the current state of knowledge in a particular field, highlighting the most recent and relevant research. It is often used in fields where knowledge is rapidly evolving, such as technology or medicine.
• Integrative literature review: This type of review synthesizes and integrates findings from multiple studies on a particular topic to identify patterns, themes, and gaps in the literature. It aims to provide a comprehensive understanding of the current state of knowledge on a particular topic.
• Umbrella literature review : This review is used to provide a broad overview of a large and diverse body of literature on a particular topic. It aims to identify common themes and patterns across different areas of research.
• Historical literature review: This type of review examines the historical development of research on a particular topic or research question. It aims to provide a historical context for understanding the current state of knowledge on a particular topic.
• Problem-oriented literature review : This review focuses on a specific problem or issue and examines the literature to identify potential solutions or interventions. It aims to provide practical recommendations for addressing a particular problem or issue.
• Mixed-methods literature review : This type of review combines quantitative and qualitative methods to synthesize and analyze the available literature on a particular topic. It aims to provide a more comprehensive understanding of the research question by combining different types of evidence.

Parts of Literature Review

Parts of a literature review are as follows:

Introduction

The introduction of a literature review typically provides background information on the research topic and why it is important. It outlines the objectives of the review, the research question or hypothesis, and the scope of the review.

Literature Search

This section outlines the search strategy and databases used to identify relevant literature. The search terms used, inclusion and exclusion criteria, and any limitations of the search are described.

Literature Analysis

The literature analysis is the main body of the literature review. This section summarizes and synthesizes the literature that is relevant to the research question or hypothesis. The review should be organized thematically, chronologically, or by methodology, depending on the research objectives.

Critical Evaluation

Critical evaluation involves assessing the quality and validity of the literature. This includes evaluating the reliability and validity of the studies reviewed, the methodology used, and the strength of the evidence.

The conclusion of the literature review should summarize the main findings, identify any gaps in the literature, and suggest areas for future research. It should also reiterate the importance of the research question or hypothesis and the contribution of the literature review to the overall research project.

The references list includes all the sources cited in the literature review, and follows a specific referencing style (e.g., APA, MLA, Harvard).

How to write Literature Review

Here are some steps to follow when writing a literature review:

• Define your research question or topic : Before starting your literature review, it is essential to define your research question or topic. This will help you identify relevant literature and determine the scope of your review.
• Conduct a comprehensive search: Use databases and search engines to find relevant literature. Look for peer-reviewed articles, books, and other academic sources that are relevant to your research question or topic.
• Evaluate the sources: Once you have found potential sources, evaluate them critically to determine their relevance, credibility, and quality. Look for recent publications, reputable authors, and reliable sources of data and evidence.
• Organize your sources: Group the sources by theme, method, or research question. This will help you identify similarities and differences among the literature, and provide a structure for your literature review.
• Analyze and synthesize the literature : Analyze each source in depth, identifying the key findings, methodologies, and conclusions. Then, synthesize the information from the sources, identifying patterns and themes in the literature.
• Write the literature review : Start with an introduction that provides an overview of the topic and the purpose of the literature review. Then, organize the literature according to your chosen structure, and analyze and synthesize the sources. Finally, provide a conclusion that summarizes the key findings of the literature review, identifies gaps in knowledge, and suggests areas for future research.
• Edit and proofread: Once you have written your literature review, edit and proofread it carefully to ensure that it is well-organized, clear, and concise.

Examples of Literature Review

Here’s an example of how a literature review can be conducted for a thesis on the topic of “ The Impact of Social Media on Teenagers’ Mental Health”:

• Start by identifying the key terms related to your research topic. In this case, the key terms are “social media,” “teenagers,” and “mental health.”
• Use academic databases like Google Scholar, JSTOR, or PubMed to search for relevant articles, books, and other publications. Use these keywords in your search to narrow down your results.
• Evaluate the sources you find to determine if they are relevant to your research question. You may want to consider the publication date, author’s credentials, and the journal or book publisher.
• Begin reading and taking notes on each source, paying attention to key findings, methodologies used, and any gaps in the research.
• Organize your findings into themes or categories. For example, you might categorize your sources into those that examine the impact of social media on self-esteem, those that explore the effects of cyberbullying, and those that investigate the relationship between social media use and depression.
• Synthesize your findings by summarizing the key themes and highlighting any gaps or inconsistencies in the research. Identify areas where further research is needed.
• Use your literature review to inform your research questions and hypotheses for your thesis.

For example, after conducting a literature review on the impact of social media on teenagers’ mental health, a thesis might look like this:

“Using a mixed-methods approach, this study aims to investigate the relationship between social media use and mental health outcomes in teenagers. Specifically, the study will examine the effects of cyberbullying, social comparison, and excessive social media use on self-esteem, anxiety, and depression. Through an analysis of survey data and qualitative interviews with teenagers, the study will provide insight into the complex relationship between social media use and mental health outcomes, and identify strategies for promoting positive mental health outcomes in young people.”

Reference: Smith, J., Jones, M., & Lee, S. (2019). The effects of social media use on adolescent mental health: A systematic review. Journal of Adolescent Health, 65(2), 154-165. doi:10.1016/j.jadohealth.2019.03.024

Reference Example: Author, A. A., Author, B. B., & Author, C. C. (Year). Title of article. Title of Journal, volume number(issue number), page range. doi:0000000/000000000000 or URL

Applications of Literature Review

some applications of literature review in different fields:

• Social Sciences: In social sciences, literature reviews are used to identify gaps in existing research, to develop research questions, and to provide a theoretical framework for research. Literature reviews are commonly used in fields such as sociology, psychology, anthropology, and political science.
• Natural Sciences: In natural sciences, literature reviews are used to summarize and evaluate the current state of knowledge in a particular field or subfield. Literature reviews can help researchers identify areas where more research is needed and provide insights into the latest developments in a particular field. Fields such as biology, chemistry, and physics commonly use literature reviews.
• Health Sciences: In health sciences, literature reviews are used to evaluate the effectiveness of treatments, identify best practices, and determine areas where more research is needed. Literature reviews are commonly used in fields such as medicine, nursing, and public health.
• Humanities: In humanities, literature reviews are used to identify gaps in existing knowledge, develop new interpretations of texts or cultural artifacts, and provide a theoretical framework for research. Literature reviews are commonly used in fields such as history, literary studies, and philosophy.

Role of Literature Review in Research

Here are some applications of literature review in research:

• Identifying Research Gaps : Literature review helps researchers identify gaps in existing research and literature related to their research question. This allows them to develop new research questions and hypotheses to fill those gaps.
• Developing Theoretical Framework: Literature review helps researchers develop a theoretical framework for their research. By analyzing and synthesizing existing literature, researchers can identify the key concepts, theories, and models that are relevant to their research.
• Selecting Research Methods : Literature review helps researchers select appropriate research methods and techniques based on previous research. It also helps researchers to identify potential biases or limitations of certain methods and techniques.
• Data Collection and Analysis: Literature review helps researchers in data collection and analysis by providing a foundation for the development of data collection instruments and methods. It also helps researchers to identify relevant data sources and identify potential data analysis techniques.
• Communicating Results: Literature review helps researchers to communicate their results effectively by providing a context for their research. It also helps to justify the significance of their findings in relation to existing research and literature.

Purpose of Literature Review

Some of the specific purposes of a literature review are as follows:

• To provide context: A literature review helps to provide context for your research by situating it within the broader body of literature on the topic.
• To identify gaps and inconsistencies: A literature review helps to identify areas where further research is needed or where there are inconsistencies in the existing literature.
• To synthesize information: A literature review helps to synthesize the information from multiple sources and present a coherent and comprehensive picture of the current state of knowledge on the topic.
• To identify key concepts and theories : A literature review helps to identify key concepts and theories that are relevant to your research question and provide a theoretical framework for your study.
• To inform research design: A literature review can inform the design of your research study by identifying appropriate research methods, data sources, and research questions.

Characteristics of Literature Review

Some Characteristics of Literature Review are as follows:

• Identifying gaps in knowledge: A literature review helps to identify gaps in the existing knowledge and research on a specific topic or research question. By analyzing and synthesizing the literature, you can identify areas where further research is needed and where new insights can be gained.
• Establishing the significance of your research: A literature review helps to establish the significance of your own research by placing it in the context of existing research. By demonstrating the relevance of your research to the existing literature, you can establish its importance and value.
• Informing research design and methodology : A literature review helps to inform research design and methodology by identifying the most appropriate research methods, techniques, and instruments. By reviewing the literature, you can identify the strengths and limitations of different research methods and techniques, and select the most appropriate ones for your own research.
• Supporting arguments and claims: A literature review provides evidence to support arguments and claims made in academic writing. By citing and analyzing the literature, you can provide a solid foundation for your own arguments and claims.
• I dentifying potential collaborators and mentors: A literature review can help identify potential collaborators and mentors by identifying researchers and practitioners who are working on related topics or using similar methods. By building relationships with these individuals, you can gain valuable insights and support for your own research and practice.
• Keeping up-to-date with the latest research : A literature review helps to keep you up-to-date with the latest research on a specific topic or research question. By regularly reviewing the literature, you can stay informed about the latest findings and developments in your field.

Advantages of Literature Review

There are several advantages to conducting a literature review as part of a research project, including:

• Establishing the significance of the research : A literature review helps to establish the significance of the research by demonstrating the gap or problem in the existing literature that the study aims to address.
• Identifying key concepts and theories: A literature review can help to identify key concepts and theories that are relevant to the research question, and provide a theoretical framework for the study.
• Supporting the research methodology : A literature review can inform the research methodology by identifying appropriate research methods, data sources, and research questions.
• Providing a comprehensive overview of the literature : A literature review provides a comprehensive overview of the current state of knowledge on a topic, allowing the researcher to identify key themes, debates, and areas of agreement or disagreement.
• Identifying potential research questions: A literature review can help to identify potential research questions and areas for further investigation.
• Avoiding duplication of research: A literature review can help to avoid duplication of research by identifying what has already been done on a topic, and what remains to be done.
• Enhancing the credibility of the research : A literature review helps to enhance the credibility of the research by demonstrating the researcher’s knowledge of the existing literature and their ability to situate their research within a broader context.

Limitations of Literature Review

Limitations of Literature Review are as follows:

• Limited scope : Literature reviews can only cover the existing literature on a particular topic, which may be limited in scope or depth.
• Publication bias : Literature reviews may be influenced by publication bias, which occurs when researchers are more likely to publish positive results than negative ones. This can lead to an incomplete or biased picture of the literature.
• Quality of sources : The quality of the literature reviewed can vary widely, and not all sources may be reliable or valid.
• Time-limited: Literature reviews can become quickly outdated as new research is published, making it difficult to keep up with the latest developments in a field.
• Subjective interpretation : Literature reviews can be subjective, and the interpretation of the findings can vary depending on the researcher’s perspective or bias.
• Lack of original data : Literature reviews do not generate new data, but rather rely on the analysis of existing studies.
• Risk of plagiarism: It is important to ensure that literature reviews do not inadvertently contain plagiarism, which can occur when researchers use the work of others without proper attribution.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Problem Statement – Writing Guide, Examples and...

Research Gap – Types, Examples and How to...

Institutional Review Board – Application Sample...

Research Problem – Examples, Types and Guide

Data Interpretation – Process, Methods and...

Evaluating Research – Process, Examples and...

Research Methods: Literature Reviews

• Annotated Bibliographies
• Literature Reviews
• Scoping Reviews
• Systematic Reviews
• Scholarship of Teaching and Learning
• Persuasive Arguments
• Subject Specific Methodology

A literature review involves researching, reading, analyzing, evaluating, and summarizing scholarly literature (typically journals and articles) about a specific topic. The results of a literature review may be an entire report or article OR may be part of a article, thesis, dissertation, or grant proposal. A literature review helps the author learn about the history and nature of their topic, and identify research gaps and problems.

Steps & Elements

Problem formulation

• Determine your topic and its components by asking a question
• Research: locate literature related to your topic to identify the gap(s) that can be addressed
• Read: read the articles or other sources of information
• Analyze: assess the findings for relevancy
• Evaluating: determine how the article are relevant to your research and what are the key findings
• Synthesis: write about the key findings and how it is relevant to your research

Elements of a Literature Review

• Summarize subject, issue or theory under consideration, along with objectives of the review
• Divide works under review into categories (e.g. those in support of a particular position, those against, those offering alternative theories entirely)
• Explain how each work is similar to and how it varies from the others
• Conclude which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of an area of research

Writing a Literature Review Resources

• How to Write a Literature Review From the Wesleyan University Library
• Write a Literature Review From the University of California Santa Cruz Library. A Brief overview of a literature review, includes a list of stages for writing a lit review.
• Literature Reviews From the University of North Carolina Writing Center. Detailed information about writing a literature review.
• Undertaking a literature review: a step-by-step approach Cronin, P., Ryan, F., & Coughan, M. (2008). Undertaking a literature review: A step-by-step approach. British Journal of Nursing, 17(1), p.38-43

Literature Review Tutorial

• << Previous: Annotated Bibliographies
• Next: Scoping Reviews >>
• Last Updated: Jul 8, 2024 3:13 PM
• URL: https://guides.auraria.edu/researchmethods

1100 Lawrence Street Denver, CO 80204 303-315-7700 Ask Us Directions

Teaching and Research guides

Literature reviews.

• Introduction
• Plan your search
• Where to search
• Refine and update your search
• Finding grey literature
• Writing the review
• Referencing

Research methods overview

Finding literature on research methodologies, sage research methods online.

• Get material not at RMIT
• Further help

What are research methods?

Research methodology is the specific strategies, processes, or techniques utilised in the collection of information that is created and analysed.

The methodology section of a research paper, or thesis, enables the reader to critically evaluate the study’s validity and reliability by addressing how the data was collected or generated, and how it was analysed.

Types of research methods

There are three main types of research methods which use different designs for data collection.  

(1) Qualitative research

Qualitative research gathers data about lived experiences, emotions or behaviours, and the meanings individuals attach to them. It assists in enabling researchers to gain a better understanding of complex concepts, social interactions or cultural phenomena. This type of research is useful in the exploration of how or why things have occurred, interpreting events and describing actions.

Examples of qualitative research designs include:

• focus groups
• observations
• document analysis
• oral history or life stories  

(2) Quantitative research

Quantitative research gathers numerical data which can be ranked, measured or categorised through statistical analysis. It assists with uncovering patterns or relationships, and for making generalisations. This type of research is useful for finding out how many, how much, how often, or to what extent.

Examples of quantitative research designs include:

• surveys or questionnaires
• observation
• document screening
• experiments  

(3) Mixed method research

Mixed Methods research integrates both Qualitative research and Quantitative research. It provides a holistic approach combining and analysing the statistical data with deeper contextualised insights. Using Mixed Methods also enables triangulation, or verification, of the data from two or more sources.

Sometimes in your literature review, you might need to discuss and evaluate relevant research methodologies in order to justify your own choice of research methodology.

When searching for literature on research methodologies it is important to search across a range of sources. No single information source will supply all that you need. Selecting appropriate sources will depend upon your research topic.

Developing a robust search strategy will help reduce irrelevant results. It is good practice to plan a strategy before you start to search.

Search tips

(1) free text keywords.

Free text searching is the use of natural language words to conduct your search. Use selective free text keywords such as: phenomenological, "lived experience", "grounded theory", "life experiences", "focus groups", interview, quantitative, survey, validity, variance, correlation and statistical.

To locate books on your desired methodology, try LibrarySearch . Remember to use  refine  options such as books, ebooks, subject, and publication date.  

(2) Subject headings in Databases

Databases categorise their records using subject terms, or a controlled vocabulary (thesaurus). These subject headings may be useful to use, in addition to utilising free text keywords in a database search.

Subject headings will differ across databases, for example, the PubMed database uses 'Qualitative Research' whilst the CINHAL database uses 'Qualitative Studies.'  

(3) Limiting search results

Databases enable sets of results to be limited or filtered by specific fields, look for options such as Publication Type, Article Type, etc. and apply them to your search.  

(4) Browse the Library shelves

To find books on  research methods  browse the Library shelves at call number  001.42

• SAGE Research Methods Online SAGE Research Methods Online (SRMO) is a research tool supported by a newly devised taxonomy that links content and methods terms. It provides the most comprehensive picture available today of research methods (quantitative, qualitative and mixed methods) across the social and behavioural sciences.

SAGE Research Methods Overview  (2:07 min) by SAGE Publishing  ( YouTube ) 

• << Previous: Referencing
• Next: Get material not at RMIT >>

• Last Updated: Aug 12, 2024 8:44 AM
• URL: https://rmit.libguides.com/literature-review

Literature review as a research methodology: An overview and guidelines

Hannah snyder.

This paper is in the following e-collection/theme issue:

Published on 19.8.2024 in Vol 26 (2024)

This is a member publication of Open University

Prevalence of Health Misinformation on Social Media—Challenges and Mitigation Before, During, and Beyond the COVID-19 Pandemic: Scoping Literature Review

Authors of this article:

• Dhouha Kbaier 1 , PhD   ; 
• Annemarie Kane 2 , PhD   ; 
• Mark McJury 3   ; 
• Ian Kenny 1 , PhD  

1 School of Computing and Communications, The Open University, Milton Keynes, United Kingdom

2 Faculty of Arts and Social Sciences, The Open University, Milton Keynes, United Kingdom

3 School of Physical Sciences, The Open University, Milton Keynes, United Kingdom

Corresponding Author:

Dhouha Kbaier, PhD

School of Computing and Communications

The Open University

Walton Hall

Milton Keynes, MK7 6AA

United Kingdom

Email: [email protected]

Background: This scoping review accompanies our research study “The Experience of Health Professionals With Misinformation and Its Impact on Their Job Practice: Qualitative Interview Study.” It surveys online health misinformation and is intended to provide an understanding of the communication context in which health professionals must operate.

Objective: Our objective was to illustrate the impact of social media in introducing additional sources of misinformation that impact health practitioners’ ability to communicate effectively with their patients. In addition, we considered how the level of knowledge of practitioners mitigated the effect of misinformation and additional stress factors associated with dealing with outbreaks, such as the COVID-19 pandemic, that affect communication with patients.

Methods: This study used a 5-step scoping review methodology following Arksey and O’Malley’s methodology to map relevant literature published in English between January 2012 and March 2024, focusing on health misinformation on social media platforms. We defined health misinformation as a false or misleading health-related claim that is not based on valid evidence or scientific knowledge. Electronic searches were performed on PubMed, Scopus, Web of Science, and Google Scholar. We included studies on the extent and impact of health misinformation in social media, mitigation strategies, and health practitioners’ experiences of confronting health misinformation. Our independent reviewers identified relevant articles for data extraction.

Results: Our review synthesized findings from 70 sources on online health misinformation. It revealed a consensus regarding the significant problem of health misinformation disseminated on social network platforms. While users seek trustworthy sources of health information, they often lack adequate health and digital literacies, which is exacerbated by social and economic inequalities. Cultural contexts influence the reception of such misinformation, and health practitioners may be vulnerable, too. The effectiveness of online mitigation strategies like user correction and automatic detection are complicated by malicious actors and politicization. The role of health practitioners in this context is a challenging one. Although they are still best placed to combat health misinformation, this review identified stressors that create barriers to their abilities to do this well. Investment in health information management at local and global levels could enhance their capacity for effective communication with patients.

Conclusions: This scoping review underscores the significance of addressing online health misinformation, particularly in the postpandemic era. It highlights the necessity for a collaborative global interdisciplinary effort to ensure equitable access to accurate health information, thereby empowering health practitioners to effectively combat the impact of online health misinformation. Academic research will need to be disseminated into the public domain in a way that is accessible to the public. Without equipping populations with health and digital literacies, the prevalence of online health misinformation will continue to pose a threat to global public health efforts.

Introduction

The global adoption of the internet has made health information more accessible, and the development of digital technology has enabled its rapid dissemination. However, the internet has also made possible the dissemination of false and misleading health misinformation and disinformation, with negative consequences, including the potential to exacerbate health inequalities. Health practitioners have found themselves at the forefront of communicating with patients who have taken on board health misinformation in the context of its proliferation on the web. This paper (associated with the study by Ismail et al [ 1 ]) surveyed the current literature concerning online health misinformation to establish the extent and scope of the problem, with special reference to the needs of health practitioners in their efforts to mitigate its impact. Several studies have established useful definitions of the terms misinformation and disinformation and distinctions between them. Misinformation has been defined as information that is not supported by evidence and contradicts the best-supported evidence available [ 2 , 3 ]. Wang et al [ 4 ] made a further distinction between online misinformation and disinformation, in particular on social media platforms. For Wang et al [ 4 ], misinformation is information that is not known to be false and is shared without malice. By contrast, disinformation involves the knowing and sharing of false information with the purpose of causing harm. This paper follows the distinctions of Wang et al [ 4 ] to use the terms misinformation and disinformation as appropriate.

It is important to acknowledge at the outset that digital technology in health and social contexts presents both risks and opportunities for equity among different information audiences [ 5 ]. However, there has recently been a change in the reception and assessment of the role of the internet, social media in particular, among researchers, even predating the COVID-19 pandemic. In the early days of social media, researchers largely identified prosocial and altruistic uses of social media platforms such as Facebook and Twitter by the public. However, considerable disquiet about the impact of social media and its potential for the spread of “fake news” and the amplification of conspiracy theories has displaced the more positive evaluation that was apparent when social media was in its infancy [ 6 ]. In the majority of the current research, there is a view that digital technology, particularly social media, has amplified the problem of health misinformation. The risk most frequently identified, either explicitly or implicitly, is the susceptibility of ordinary users, who may be lacking sophisticated levels of health and digital literacies, to health misinformation. Further risks noted in the literature include disinformation disseminated by organized trolling networks and bots that can be hard to distinguish from human users. The recognition of these risks underpins an emerging policy discourse about the threat of health misinformation, particularly the role of social media in its spread, in which health information and misinformation has become a politicized issue. From one policy perspective, there is an assumption that social media users are vulnerable, even passive, recipients of health misinformation rather than reflective interpreters of the available information. The corollary of this is that correcting misinformation with authoritative knowledge will solve the problem. However, a survey of the literature suggested that neither assumption fully expresses the complexity of how health misinformation is disseminated, received, and used via the internet. This may be because although there is a growing body of evidence on the extent of online health misinformation, there is much less research about what users do with health misinformation, why users consume health misinformation, and why (and which) people believe health misinformation [ 7 - 9 ].

In this scoping review, we reviewed the current state of knowledge regarding the prevalence of online misinformation before and during the COVID-19 pandemic and the impact that has on users’ understanding of health information. We considered this context with special reference to patients’ understanding, health practitioners’ practice in response to that, and policy makers’ concerns. The pressures and distractions that health professionals face in attempting to mitigate the impacts of online health misinformation are discussed in relation to patients’ health and digital literacies and the politicization of health information and misinformation.

Information Sources

We conducted a comprehensive literature search to identify relevant studies that explored health misinformation on social media platforms. The search was conducted across multiple electronic databases, including PubMed, Scopus, Web of Science, and Google Scholar.

The search terms included a combination of relevant keywords and phrases, including “health misinformation,” “social media,” “online health communities,” and “COVID-19 pandemic.” The search was not limited by publication date. Detailed search strategies are provided in Multimedia Appendix 1 .

Study Selection

Our study selection process followed a scoping approach, where we aimed to identify and include studies that provided insights into the prevalence and challenges of health misinformation on social media platforms. Initially, 2 researchers independently screened titles and abstracts of the identified articles to determine their relevance. Articles that did not meet the inclusion criteria were excluded at this stage.

Inclusion Criteria

Articles were included if they discussed health misinformation on social media, addressed the challenges posed by health misinformation, or were relevant to the period before, during, and after the COVID-19 pandemic.

Any disagreements between the 2 researchers were resolved through discussion and consensus. Full-text articles were then retrieved for the remaining studies, and a further assessment of eligibility was conducted based on the same inclusion criteria.

Data Extraction

We gathered information on (1) study objectives, (2) research methods, (3) findings, and (4) key themes related to health misinformation. This process was performed independently by 2 researchers, and any discrepancies were resolved through discussion.

Data Synthesis and Analysis

We adopted a scoping review content analysis approach to analyze the data extracted from the selected articles. The analysis process involved identifying key themes and patterns related to health misinformation on social media. The content analysis allowed us to gain a deeper understanding of the challenges posed by health misinformation and the strategies for its mitigation, both before and during the COVID-19 pandemic.

Results of Search

In our article selection process ( Figure 1 ), we initiated our search by identifying a total of 4563 articles from various databases. Following the removal of duplicates, 1295 articles were excluded, leaving us with 3268 unique articles. Subsequently, these articles underwent an initial screening, which involved evaluating their abstracts and titles, resulting in the exclusion of 2635 articles that did not align with our inclusion criteria. Further scrutiny was applied during full-text screening, which was conducted on 633 articles. Among these, 563 articles were found ineligible due to reasons such as not meeting the inclusion criteria (n=378 articles), being classified as literature reviews, editorials, or letters (n=174 articles), or the full texts being inaccessible (n=11 articles). A total of 70 articles were ultimately included in this scoping review.

Characteristics of Included Documents (n=70)

The majority (65/70, 93%) of documents were published in the last 10 years and originated predominantly in North America (42/70, 60%), Europe (19/70, 27%), and Asia (11/70, 16%). The funding sources were mainly public (61/70, 87%). The documents were classified as original research papers (38/70, 54%), subjective “opinion” articles (editorials, viewpoints, commentaries, and letters to the journal; 11/70, 16%), and knowledge syntheses or reviews (9/70, 13%) which included systematic reviews (n=6), descriptive reviews (n=2), and 1 integrated theoretic review.

Extent and Impact of Health Misinformation Disseminated Across a Range of Outlets

This section will review the literature concerning the extent and impact of the problem of health misinformation, including the spread of antivaccination discourse. In a study by Wood et al [ 10 ] among health practitioners in North Carolina, 94.2% of the respondents reported encounters with patient health misinformation within the previous year. While the sources of this misinformation were not broken down and identified by Wood et al [ 10 ], several other studies linked patient health misinformation to the prevalence of health misinformation on social media sites, identifying the latter as a significant problem [ 4 , 11 - 15 ]. There is a growing consensus among researchers, health professionals, and policy makers about the need to confront, challenge, and even prevent the online dissemination of health misinformation [ 16 ]. Since the emergence of online social networks, users have increasingly sought and shared health information on social media sites. It is estimated that around 70% of adult internet users search health matters on the web. With the emergence of social media platforms, there has been a rise in “peer-to-peer health care,” through which individuals seek and share health information, forming online health communities with others who have similar health concerns [ 3 ]. In addition, health organizations and health professionals are increasingly using social media to disseminate and promote health information and advice. The opportunities for sharing and promoting good health information via the internet are evident, and it is important to acknowledge that in online health communities, users share experiences and receive and give different kinds of support, including emotional support, to cope with specific health conditions. However, the medium has also enabled the dissemination of health misinformation, and the prosocial aspects of sharing are also likely to involve the sharing of misinformation, putting the health of users at risk [ 3 ].

Misinformation Spreads on Social Media

There is a high degree of consensus among researchers that mainly because of the increasing popularity of social media, the internet has become a space for the dissemination and amplification of “fake news,” misleading information, and rumor, including health misinformation and antivaccine conspiracy theories [ 17 ]. The COVID-19 pandemic has heightened these concerns, resulting in a proliferation of recent studies and rapid reviews focusing on the online spread of misinformation. Lee et al [ 18 ] proposed that the proliferation of health misinformation during the COVID-19 pandemic became a major public health issue. At the earliest signs of the emerging COVID-19 pandemic, the director-general of the World Health Organization, Tedros Adhanom Ghebreyesus, speaking at the February 2020 Munich Security Conference, expressed concern about the risk of an infodemic of health misinformation disseminated via social media, identifying “vaccine hesitancy” as 1 of the top 10 global health threats [ 19 ]. Bapaye and Bapaye [ 20 ] agreed that the risks of misinformation on social networking sites constitute a global issue, referring specifically to the COVID-19 infodemic.

However, this is not in itself a new problem; longstanding concerns about “fake news” and misinformation in traditional media have been evident since the early decades of the 20th century [ 21 ], and the prevalence of misinformation on internet platforms certainly predates the COVID-19 pandemic. Therefore, because the COVID-19 pandemic has only intensified the concern regarding health misinformation, it might be more appropriate to see the pandemic as symptomatic of, and crystallizing, the challenges of countering health misinformation in the digital age, as the development of digital technology and the internet have brought about profound changes in the capacity of both misinformation and disinformation to spread globally and amplify rapidly [ 4 ].

Suarez-Lledo and Alvarez-Galvez [ 16 ] undertook a review of 69 studies of health misinformation on social media to identify the main health misinformation topics and their frequency on different social media platforms. The studies surveyed used a variety of research methods, including social network analysis (28%), evaluation of content (26%), evaluation of quality (24%), content/text analysis (16%), and sentiment analysis (6%). Suarez-Lledo and Alvarez-Galvez [ 16 ] concluded that the incidence of health misinformation was highest on Twitter, in particular, regarding the use of tobacco and other drugs, with some studies citing 87% of such posts containing misinformation. However, health misinformation about vaccines was also prevalent, with around 43% of posts containing misinformation, with the human papillomavirus vaccine being the most affected. This review by Suarez-Lledo and Alvarez-Galvez [ 16 ] confirmed many of the findings from earlier surveys. For example, in their survey of 57 articles, Wang et al [ 4 ] found that the most frequently discussed topics were regarding vaccination and infectious diseases, including Ebola and the Zika virus. Other topics such as nutrition, cancer, water fluoridation, and smoking were also prevalent. The studies they surveyed had tended to find that a high degree of misinformation on these topics was being shared and liked on social media.

Lee et al [ 18 ] conducted a cross-sectional online survey in South Korea to examine the prevalence of COVID-19 misinformation and the impact of exposure to COVID-19 misinformation on beliefs and behaviors. They found that exposure to COVID-19 misinformation was associated with misinformation belief, which then resulted in fewer preventive behaviors. Therefore, they highlighted the potential of misinformation to undermine global efforts in disease control and argued that public health strategies are needed to combat the proliferation of misinformation. Bapaye and Bapaye [ 20 ] conducted a cross-sectional online questionnaire survey of 1137 WhatsApp users in India. They noted that most research on the prevalence of misinformation in social media has focused on Twitter and Facebook and on the Global North. Measured by age, researchers found that users aged >65 years were the most vulnerable to accepting the veracity of messages containing health misinformation (K=0.38, 95% CI 0.341-0.419) Respondents aged 19 to 25 years displayed much lower vulnerability (K=0.31, 95% CI 0.301-0.319) than those aged >25 years ( P <.05). Measured by occupational category, users employed in nonprofessional occupations had the highest vulnerability (K=0.38, 95% CI 0.356-0.404); this was significantly higher than those of professionals and students ( P <.05). Notably, the vulnerability of health professionals was not significantly different from those of other occupation groups ( P >.05).

The authors concluded that in a developing country, WhatsApp users aged >65 years and those involved in nonprofessional occupations are the most vulnerable to false information disseminated via WhatsApp. Crucially, they noted that health care workers, who might be expected by laypersons to have expert knowledge, were as likely to be vulnerable to health misinformation as other occupation groups.

Antivaxxer Spread Before, During, and Beyond the COVID-19 Pandemic

Much of the current unease from researchers, understandably, centers on health misinformation about vaccines in the wake of the COVID-19 pandemic. In particular, there is concern about the growth and spread of so-called antivaxxer misinformation and beliefs. In 2019, the United States had its biggest measles outbreak in 30 years, with most cases involving people who had not been vaccinated. Hotez [ 22 ] claimed that much of the reason for the growth of antivaccine beliefs is because of a campaign of misinformation. He argued that social media sites are meeting places for the sharing of antivaccine views. To evade social media platforms’ automated moderation tools, which tend to focus on words, several antivaxxer groups, including one with around 250,000 members, began using visual codes, such as the carrot emoji, to hide antivaxxer content.

However, some of the misinformation has gained credibility because it has come from sources that laypersons would expect to be trustworthy. For example, in 1998, the British medical journal The Lancet published a paper by Dr Andrew Wakefield claiming a link between the measles, mumps, rubella vaccine and the onset of autism spectrum disorder. Wakefield’s paper was later rebutted, and an overwhelming body of evidence now refutes its conclusions [ 23 ]. However, despite long being discredited, Wakefield’s claims have remained a part of the antivaccine discourse. The persistence of the antivaccination narrative demonstrates the power of such discourses even in the face of evidence to challenge them.

Although strong antivaccine beliefs, and the more ambivalent attitude of vaccine hesitancy, have been around as long as there have been vaccines, until recent decades, they were on the margins. However, evidence supports the claim that they have been gaining momentum in the United States and Europe.

A survey by Skafle et al [ 24 ] aimed to synthesize the results from 19 studies in which the effect of social media misinformation on vaccine hesitancy was measured or discussed. The authors noted that the “vast majority” of studies were from industrialized Western countries. Only 1 study contained misinformation about autism as a side effect of COVID-19 vaccines. Nevertheless, the studies implied that information spread on social media had a negative effect on vaccine hesitancy and uptake. The conclusions from Skafle et al [ 24 ] were supported by data from online polling agencies. For example, a US YouGov poll from May 2020 found that only 55% of respondents would definitely take a COVID-19 vaccine if one were to become available, whereas 19% of respondents said that they would refuse and 26% were still undecided [ 25 ].

While much of the research about online vaccine discourse comes from the United States, there is also evidence that vaccine hesitancy has risen elsewhere. For example, in an Ipsos-MORI survey taken in December 2020, only 40% of respondents in France said they would take a COVID-19 vaccine, a figure symptomatic of a steep and swift decline in vaccine confidence in France [ 26 ]. However, interestingly, the same Ipsos-MORI poll indicated a rise in vaccine confidence among respondents in the United States since the earlier YouGov poll, cited earlier, by approximately 10% to 65%, and respondents in the United Kingdom expressed a still higher willingness to take a COVID-19 vaccine at approximately 77%. It is notable that in the United States and United Kingdom, the Ipsos-MORI results came after a period of intermittent lockdowns. The contrast with the results from France is, nevertheless, striking.

Understanding the Challenges Surrounding Health Misinformation

Here, we consider the challenges created by health misinformation on the web: (1) the role played by malicious actors on social media in spreading vaccine disinformation and misinformation and (2) how contextual and cultural issues have different effects on patients’ understanding of what is considered genuine, valid, and authentic health information.

Spread of Health Misinformation on Social Media by Malicious Actors

One strand of research presents the issue of health misinformation as a contest between trolls and bots on the one hand and the voices of trustworthy public health agencies on the other [ 6 ]. This view was supported by Hotez [ 22 ] and Broniatowski et al [ 11 ]. The latter investigated the role of bots and trolls as malicious actors mobilizing vaccination discourse on the web. Their study focused specifically on vaccine-related health messaging on Twitter. Comparing the rates of vaccine-related messages, they found that sophisticated bots and Russian trolls tweeted at higher rates than “average users.” However, the respective content from bots and trolls differed. Whereas bots communicated antivaccine messages, Russian troll accounts provided a seemingly balanced discussion of both provaccination and antivaccination arguments, implying an equivalence between them. The authors argued that amplifying and normalizing a debate is done with the purpose of sowing discord and may lead to undermining public confidence in scientific consensus about the effectiveness of vaccines. Wang et al [ 4 ] acknowledged that it is a challenge to readily distinguish between misinformation and disinformation on the web. They noted that disinformation, such as antivaccine propaganda, can unknowingly be spread by users with genuine concerns [ 4 ], as individuals increasingly seek health and healthy lifestyle information via the internet.

Contextual Factors Influencing the Reception of and Responses to Misinformation: Politicization of the Problem of Health Misinformation

The identification of online trolls, bots, and orchestrated networks as major contributors to the spread of health disinformation and misinformation is now part of mainstream political discourse in the United States. On July 16, 2021, a quarrel broke out between the president of the United States, Joe Biden, and Facebook over the spread of health misinformation on the company’s social media platforms. Speaking to journalists, Biden blamed social media companies for a rise in the number of deaths from COVID-19 among the unvaccinated in the United States. Referring explicitly to Facebook, the president claimed that by allowing the proliferation of health misinformation on its platforms, the company was “killing people” [ 27 ]. Discursive interventions from politicians are never neutral; nevertheless, Biden’s claim about the impact of health misinformation on social media is backed up by many of the studies surveyed for this paper. Facebook immediately rebutted Biden’s accusation by citing their rules, introduced in February 2021, which banned posts that make identifiably false claims about vaccines. Furthermore, Facebook challenged Biden’s claim by asserting that not only has Facebook provided more authoritative information about COVID-19 and vaccines than any other internet site, reaching 2 billion people with such posts, but also that the platform’s vaccine finder tool had been used by more than 3 million Americans.

These figures suggest that although antivaxxer groups find ways to evade detection, their reach may be countered by that of information grounded in current science. A spokesperson for the company said that, far from killing people, “The facts show that Facebook is helping save lives. Period” [ 27 ]. The argument between Biden and Facebook may indeed signal more lay awareness of the problem and echo the concerns of the recent academic research about the dissemination of health misinformation by organized bot and troll networks. Framed as it is, in terms of apportioning the blame for the spread of health misinformation, Biden’s intervention mirrors much of the academic discourse in the United States on the subject. However, it is also symptomatic of the politicization of health misinformation, arguably accelerated by the COVID-19 pandemic, which may thwart evidence-based decision-making. This point was emphasized strongly by Kyabaggu et al [ 5 ]. They framed the problem of pervasive misinformation and disinformation in terms of prime movers and beneficiaries who use it to advance sociopolitical agendas and entrench asymmetrical power, especially in times of uncertainty and threat, such as the COVID-19 pandemic.

Kyabaggu et al [ 5 ] identified government failures to adopt evidence-informed decision-making. They noted that such failures have costs that not only are economic but, crucially, result in poorer health outcomes. They cited as an example the United Kingdom government’s initial prevaccine herd immunity strategy. The intention of this strategy was to allow SARS-CoV-2 to indiscriminately spread to a critical mass to build up population immunity. The authors noted that this was “a particularly concerning example of evidence framing by a government.” Kyabaggu et al [ 5 ] argued that public acceptance of health risk messages and adoption of health-protecting behaviors is highly contingent on the degree to which governments engage in evidence-informed decision-making and communicate this basis effectively. The authors cited several instances of government actors failing to recognize misinformation, disseminating inconsistent or inaccurate information, and not using evidence- and information-based decision-making processes. In recent years, the public policy discourse in the United Kingdom has been veering away from evidence- and information-based decision-making, as politicians have denounced “experts” and their “influence” on policy [ 28 , 29 ].

Finally, Gruzd et al [ 30 ] reported on the impact of coordinated link-sharing behavior to spread and amplify conspiracy-related misinformation. They found a coalition of Facebook accounts that engaged in coordinated link sharing behavior to promote COVID-19 related misinformation. This coalition included US-based pro-Trump, QAnon, and antivaccination accounts.

Contextual Factors Influencing the Reception of and Responses to Misinformation: Health Literacies and Inequality

While the approach of Broniatowski et al [ 11 ], for example, provided a persuasive account of ways in which online health misinformation can be disseminated, there are limitations to this approach, as it did not provide an account of how users respond to the misinformation they encounter. The responses of ordinary users were assumed rather than investigated. Research by Vosoughi et al [ 31 ] provided a caveat to the claim that it is bots that accelerate the spread of misinformation. Their work supported that of Broniatowski et al [ 11 ] in suggesting that bots spread accurate and false information at the same rate. However, Vosoughi et al [ 31 ] also explained that misinformation spreads more rapidly than accurate information because humans, rather than bots, are more likely to spread misinformation [ 31 ]. This claim was further supported by Wang [ 32 ], who suggested that in democracies, where ideas compete for attention in a marketplace, accurate scientific information, which, for the layperson, may be boring or difficult to understand, is easily crowded out by information that is more easily grasped or sensational. Mokhtari and Mirzaei [ 12 ] located this problem specifically in the context of the COVID-19 pandemic. They considered that high mortality from COVID-19, its complexity, and its unknown features resulted in fear, anxiety, and mental pressure among people worldwide. To allay anxiety, people needed health information literacy, defined by the American Library Association as a set of abilities individuals require to recognize when information is needed and to locate, evaluate, and use it effectively [ 33 ]. In addition, Wang [ 32 ] noted that individuals are differentially vulnerable to health misinformation depending on their level of health literacy and that models need to account for this. Mokhtari and Mirzaei [ 12 ] argued that not only information and health literacies but also media literacy are needed. However, studies in the field of health literacy suggest that significant inequalities in health and digital literacies exist.

Researchers have argued that “vastly undervalued and unrecognized” health literacy ought to be considered the best “social vaccine” for preventing COVID-19 in populations [ 5 ]. However, inequalities in health literacy persist. Kyabaggu et al [ 5 ] defined health literacy as encompassing cognitive and social skills that determine individuals’ motivation and ability to access, understand, and use information, including quantitative health risk information, in ways that promote and maintain good health across the life course. They asserted that health literacy is an essential self-management skill and community resource for health, noting that health literacy is positively associated with patients’ involvement in clinical decision-making, willingness to express health concerns, and compliance with clinical guidance. However, despite research demonstrating the importance of health literacy, evidence, even from high-income countries, suggested relatively low levels of health literacy.

Kyabaggu et al [ 5 ] drew a link between health literacy and digital literacy. They suggested that the latter can be understood as health literacy in digital information and technology spaces. They argued that inequalities in health outcomes are exacerbated by a widening digital divide. While digital technology in health and social contexts presents both new risks and opportunities for equity in different information audiences, the ways in which power and privilege operated in the COVID-19 misinformation discourse have not been sufficiently examined. Although socially and economically disadvantaged groups were at a greater risk of exposure to COVID-19, their voices and experiences were often marginalized. In addition, inequalities in access to accurate information are not only related to issues of digital access and literacy but are also situational. For example, disadvantaged individuals may have fewer social connections, and low pay may necessitate longer working hours, militating against individuals having the resources of time and energy to seek out accurate health information and enhance their level of health literacy.

The experiences of specific groups may also go unreported. Quraishi [ 34 ] addressed the impact of misinformation on South Asian students—a fast-growing group in the United States, but one that often receives little media attention. Quraishi [ 34 ] concluded that there is a relationship between the COVID-19 pandemic and students’ academic performance and mental health, as well as an increase in the spread of misinformation regarding COVID-19 public safety guidelines.

Older adults can be a vulnerable group in relation to their comparatively poor digital literacy. Zhou et al [ 35 ] reported on the accuracy of older adults in judging health information credibility. They found that on average, participants only successfully judged 41.38% of health articles. Attractive headlines increased participant credibility judgments on the content, and of the articles shared with others, 62.5% contained falsehoods.

Contextual Factors Influencing the Reception of and Responses to Misinformation: Cultures and Values

Larson and Broniatowski [ 19 ] argued that developing the kinds of literacy advocated by Mokhtari and Mirzaei [ 12 ] and Tully et al [ 2 ] will not address the deep-seated problems they identified. The work by Kyabaggu et al [ 5 ] supported this, and noted that the infodemic crisis is not merely a health and digital literacy issue. Some demographics may be more vulnerable to persuasive communication from broader sociocultural forces. Kyabaggu et al [ 5 ] argued that in considering the social determinants of health, attention must be paid not only to digital and health literacies but also to the ways in which these literacies coexist and interact with other influences. Larson and Broniatowski [ 19 ] suggested that one of the strongest determinants of vaccine confidence or vaccine hesitancy is the level of trust or distrust in the institutions that produce vaccines. A higher level of trust encourages the willingness to accept a high level of risk for a greater benefit. A lower level of trust militates against the acceptance of even a low level of perceived risk. For Larson and Broniatowski [ 19 ], it is not simply the presence of misinformation on social media networks but the social and cultural context of users’ reception of that information that influences responses. Health information operates in a complex and contentious social world. Individuals and communities respond to new information in terms of already developed political, cultural, and social values that influence whether they trust or distrust authority. Populations may be characterized by trust or mistrust of scientific institutions and government. Trust has been eroded through the exposure of fraud, research scandals, and misconduct by major multinational pharmaceutical companies, for example. Communities may be predisposed to distrust the government and its agents depending on their own status or identity. According to Goldenberg [ 36 ], these contexts can make misinformation and health conspiracy theories compelling.

Strategies to Correct Online Misinformation

We address the additional pressures on health professionals in communicating accurate information to mitigate the effects of misinformation, particularly with regard to the additional requirements imposed as a result of the precautions being taken during the pandemic. One area of disagreement in the literature concerns the usefulness of user correction response.

Research Into User Correction Strategies

There is some disagreement as to whether engagement with misinformation by users spreads and reinforces it or even whether extended debates over health misinformation cause users to doubt the possibility of knowable facts. For example, Broniatowski et al [ 11 ] argued that when ordinary users directly confront vaccine-skeptic messages from bots, it only serves to legitimize the “debate.” By contrast, Tully et al [ 2 ] argued that social media users have a role to play in either spreading or stopping the spread of misinformation across platforms. Their research aimed to uncover what factors influenced users’ responses. Tully et al [ 2 ] acknowledged that a range of factors can influence the spread or prevention of misinformation, including the behavior of malicious actors such as bots and trolls; the platform’s terms of service; and content moderation policies. As already noted, while most users are not creators of misinformation, they may spread and amplify it by liking, sharing, or replying. In opposition to the work of Broniatowski et al [ 11 ], Tully et al [ 2 ] argued that the content of engagement is particularly important, as their research suggested that multiple corrections by social media users may be required to reduce misperceptions. However, they claimed that most people simply ignore misinformation when they see it on social media.

Tully et al [ 2 ] noted the promise in mobilizing users to engage in such correction, given the vast numbers of users on these sites, in comparison with professional fact-checkers and health authorities.

They considered whether the tone of a correction would influence perceptions of the credibility of the message. However, despite some mixed evidence, they concluded that overall, the tone was not a significant factor and that neutral, affirmative, and uncivil corrections were all effective at reducing misperceptions. They found that participants were generally unlikely to reply to the misinformation tweet. However, their content analysis of hypothetical replies suggested that when users did reply, they mainly provided correct information, particularly after seeing other corrections. Tully et al [ 2 ] concluded that user corrections offer “untapped potential” in responding to misinformation on social media, but further work is needed to consider how users can be mobilized to provide corrections, given their overall unwillingness to reply. However, a limitation of the experimental approach of Tully et al [ 2 ], acknowledged by the researchers, is that in asking individuals what they would hypothetically do, this may not reflect what they actually do in a real social media setting, especially in relation to an issue they care more strongly about. Although the experiment gauged attitudes, it did not delve into how strongly these attitudes were held. It is also not clear to what degree corrections were effective at reducing misperceptions and how reductions were measured.

By contrast, the results of experimental studies by Ittefaq [ 37 ] and Mourali and Drake [ 38 ] suggested that correcting misinformation is by no means a straightforward proposition. They noted the previous research on rebuttal, which suggested that properly designed corrections can mitigate the effects of misinformation. However, such studies have tended to compare responses to misinformation followed by correction with responses of a control group that receives no correction or receives an alternative correction. Mourali and Drake [ 38 ] argued that this static approach misses the dynamic nature of social media debate. They noted that the correction of misinformation is generally followed up with a rebuke by the original poster, inciting further correction and prolonged back-and-forth debate. Mourali and Drake [ 38 ] cited previous studies showing that exposure to conflicting information about health topics, including mammography, nutrition, and the human papillomavirus vaccine, may increase confusion and negative attitudes toward that particular health topic. The researchers found that initial exposure to misinformation had a negative impact on attitudes and intentions toward masking, consistent with previous studies that concluded that exposure to misinformation negatively impacts attitudes and intentions toward behaviors favored by science. Also consistent with previous research, they found that the first correction of the false claim improved attitudes and intentions toward masking. The authors suggested that this effect is partially explained by a decrease in the perceived strength of the argument underlying the false claim. However, this initial improvement diminished on further exposure to false claims and refutation attempts. This finding confirmed their hypothesis that extended exposure to false claims and refutation attempts appears to weaken belief in the possibility of objective knowledge, leading to less positive reactions toward masking as a science-based behavior. They concluded that the level of exposure to contradictory information needs to reach a certain threshold before it affects perceived truth objectivity. However, although people are more likely to share misinformation when its content is consistent with their existing beliefs or when its message is simple, direct, or sensational, correcting misinformation does reduce its likelihood of being shared on social media, an effect that persists even after multiple exposures.

Mourali and Drake [ 38 ] noted that each social media platform exhibits particular interaction norms, which may impact how users interpret the conversation. As their study was limited to a single platform, Reddit, and the debate was restricted to 4 exchanges between only 2 protagonists, the researchers acknowledged that these aspects limit the generalizability of the results. They suggested that future research could attempt to replicate their findings on different social media platforms, and to include more than 2 protagonists and more than 4 exchanges. They noted further that although extended debates are common on social media, it is not known how frequently they occur, echoing the comments by Suarez-Lledo and Alvarez-Galvez [ 16 ] that the extent of misinformation is not clear.

In contrast to the fairly sanguine view of Tully et al [ 2 ] about the potential of users to spread corrective information, Mourali and Drake [ 38 ] problematized the position, pointing to the potential for more complex and uncertain outcomes, whereas Larson and Broniatowski [ 19 ] argued that although the importance of correcting misinformation, item by item, should not be diminished, only if underlying issues driving misinformation are addressed can, for example, long-term vaccine confidence in populations be sustained. They argue that simply responding to misinformation with factual corrections is not likely to reverse the dissent that has been evident among antivaxxers or to necessarily persuade the more ambivalent vaccine-hesitant individuals. They identified deeper social and cultural issues at play, which have been discussed in this paper in the previous sections.

Research Into Effective Models to Accomplish the Automatic Detection of Health Misinformation in Online Health Communities

Here, we consider examples of research into the automatic detection of health misinformation in online health communities. Zhao et al [ 3 ] began from the premise that there is a vast amount of health misinformation, creating a challenge for health communities in identifying misinformation. Rather than relying on users’ ability to correct misinformation, they proposed that there is a need for an effective model to achieve automatic detection of health misinformation in online health communities. This view was also put forward by Weinzierl and Harabagiu [ 39 ]. Focusing specifically on COVID-19 vaccine misinformation, they argued that automatic detection of misinformation on social media is an essential first step in delivering interventions designed to address vaccine hesitancy.

Zhao et al [ 3 ] identified much of the existing analysis as concentrating on the linguistic features of communications only. They wanted to examine the underresearched area of whether integrating user behavioral features with linguistic features, sentiment features, and topic features could effectively distinguish misinformation from accurate information in online health communities. Their study combined the aforementioned features to build a detection model targeting misinformation in online health communities’ contexts. The behavioral features targeted were discussion initiation, interaction engagement, influential scope, relational mediation, and informational independence. Descriptions of these behavioral features are reproduced in Table 1 .

The authors tested their detection model on a data set collected from a real online health community, selecting as their data source Zibizheng Ba, an autism forum on the Baidu Tieba online health community site hosted by the Chinese web service Baidu. Baidu Tieba claims to be one of the largest interest-based discussion platforms in China. Users can generate topic-based discussion forums on the platform, share information, and make friends with other users. Posts on Baidu Tieba are indexed by Baidu, China’s most popular search engine, so users can readily find misinformation when searching for health-related information through the search engine. The authors developed a python-based web crawler to collect data from the forum. To train the health misinformation detection model, 5000 records were sampled from the whole data set by stratification according to 3 types of records (ie, thread, reply, and reply to reply) using stratified sampling methods. Therefore, the constituent types of the records (ie, thread, reply, and reply to reply) in the sample data set were consistent with the composition of the whole data set.

The researchers applied the elaboration likelihood model (ELM). The model, originally developed by Petty and Cacioppo [ 40 ] to explain attitude change, has been used extensively in advertising to try to influence consumers.

Overall, 4 types of misinformation were identified through their coding analysis, and the model correctly detected about 85% of the health misinformation. Their results also indicated that behavioral features were more informative than linguistic features in detecting misinformation. The authors concluded that their results not only demonstrated the efficacy of behavioral features in health misinformation detection but also offered both methodological and theoretical contributions to misinformation detection by integrating the features of messages as well as the features of message creators. Others have also highlighted the problems posed by misleading visual information [ 41 ].

It is worth noting that during the pandemic, the UK National Health Service (NHS) began using Twitter to promote provaccine messaging, which closely follows a combination of the features suggested by Zhao et al [ 3 ]. When users searched for the term “vaccine” or related terms, the top post was a message prominently displaying the NHS logo, identifying it as reputable and trustworthy. The tweets contained links to NHS websites providing information about vaccines and COVID-19. The posts differed in linguistic content and visual design. For example, one featured only written text on a white background and stated in bold, “Know the facts.” Another featured a large image of a happy minority ethnic family, washing dishes together, with the message that the COVID-19 vaccine decreases household transmission by up to half. The contrasting designs suggest that the message was targeted specifically to users’ timelines. It was also apparent that elements of ELM were being applied, combining the features identified by Zhao et al [ 3 ] in different ways.

Weinzierl and Harabagiu [ 39 ] adopted a different method than Zhao et al [ 3 ], reversing the more commonly used classification approach. The authors of each study claimed strong results in identifying health misinformation on social media platforms. However, Nabożny et al [ 42 ] argued that the current automatic systems for assessing the credibility of health information are not sufficiently precise to be used without supervision by human medical expert annotators.

Barve and Saini [ 43 ] have reported on their use of automated fact-checking using a coded content similarity measure (CSM). In this approach, the CSM showed improved accuracy (91.06%) compared to the accuracy of the Jaccard similarity measure (74.26%). Further, the algorithmic approach outperformed the feature-based method.

Neither Zhao et al [ 3 ] nor Weinzierl and Harabagiu [ 39 ] recorded what happens when misinformation is detected. Research from Broniatowksi et al [ 44 ] suggested that once detected, steps taken by social media platforms such as content removal or deplatforming may not be effective in stemming the spread of misinformation and may even be counterproductive. Social media platforms use a combination of “hard” and “soft” content remedies to reduce the spread of health misinformation. Soft remedies include warning labels attached to content and downranking of some content in web searches, whereas hard remedies include content removal and deplatforming of accounts. Hard remedies are controversial and have given rise to accusations of censorship. For the authors, short-term evidence for the effectiveness of hard remedies is in any case mixed, and long-term evidence is yet to be examined. Their study focused on Facebook and found that while hard remedies did reduce the number of antivaccine posts, they also produced unintended consequences. Provaccine content was removed, and engagement with the remaining antivaccine content repeatedly recovered to prepolicy levels. Worryingly, this content became more misinformative, more politically polarized, and more likely to be seen in users’ news feeds. The authors explain these results as a product of Facebook’s architecture, which is designed to promote community formation. Members of communities dedicated to vaccine refusal seek out misinformation. To meet this demand, and to circumvent content moderation efforts, antivaccine content producers post links to external sources of misinformative content, such as Bitchute, Rumble, Gab, and Telegram, in lieu of more mainstream platforms that had implemented similar content removal policies (eg, YouTube and Twitter). Broniatowski et al [ 44 ] argued that Facebook’s policy reduced the number of posts in antivaccine venues but was not successful in inducing a sustained reduction in engagement with antivaccine content, including misinformation. The authors noted that alternative platforms often host politically extreme right-wing content. Therefore, they argued that Facebook’s content removal policies may have the unintended consequence of radicalizing their audiences, and their findings suggested the need to address how social media platform architecture enables community formation and mobilization around misinformative topics when managing the spread of online content.

These studies advocate for the automatic detection of health misinformation. However, work that calls into question the ability of automatic detection to operate without human intervention has also been discussed. In addition, there are questions raised in the literature about what should be done when misinformation is detected and concerns about whether content removal or deplatforming of accounts are the most effective ways to reduce the spread of health misinformation or may even be counterproductive.

The Roles of Health Practitioners

The discussion so far has highlighted the complex and multifaceted dimensions of the context of online health misinformation in which health practitioners must operate. As noted in our introduction, a study of health practitioners in North Carolina found that nearly 95% had encountered patient health misinformation within the previous year [ 10 ]. There is very little research on the amount or effectiveness of training received by health professionals to prepare them for engaging with patients about health misinformation. Wood et al [ 10 ] found that most respondents had not received relevant training despite overwhelmingly reporting encountering health misinformation.

Nevertheless, within the literature, there is no shortage of advice from researchers and health professionals addressed to health practitioners on how to approach and correct health misinformation. This advice stems from both original research studies and reviews of best practices featured in peer-reviewed medical and health journals. Such advice centers on the need for health practitioners to understand misinformation and how to address it. Health practitioners are advised of the need to be aware of health myths and urged to dismantle them in providing accurate health guidance [ 45 , 46 ]. Practitioners are further advised that misinformation and pseudoscience are appealing to those seeking certainty because they present information in absolutes, whereas medical science is often ambiguous and contingent. Health practitioners are also encouraged to learn how to message more clearly and to mimic the strategies of misinformation [ 45 ]. One study recommends that “practitioners familiarize themselves with the tools of scientific enquiry and consider the pros and cons of various conspiracy evaluation guidelines” [ 47 ]. Thompson [ 48 ] reports on the activity of health professional influencers and pedagogues in combating misinformation. However, the effectiveness of such social media influencers who are also health professionals remains unclear. At the same time, there is some acknowledgment in this body of literature that misinformation cannot simply be offset with facts, confirming the challenges, discussed earlier, of simply engaging in online refutation. Addressing misinformation also depends on meeting patients’ emotional needs [ 45 , 49 ].

In this context, the one-to-one patient-provider relationship in the practice setting is perceived as paramount [ 45 ]. As suggested by much of the research, source credibility, or trust, is understood to be the strongest driver of effective correction strategies [ 50 ]. It is argued that health care practitioners have the unique opportunity to guide patients toward high-quality, evidence-based medical information [ 10 ]. However, it is also noted that practitioners will need patience in their efforts to persuade patients to abandon strongly held self-beliefs, however harmful. Doing so may mean patients relinquishing membership of online communities that have become integral in their lives and even their identities. As noted earlier, belief in misinformation is often persistent in the face of evidence. Success is more likely when individuals are encouraged to reexamine their information sources, alongside new information providing additional context, rather than simply characterizing the individual’s beliefs as wrong [ 51 ]. Kyabaggu et al [ 5 ] commented that good health communication needs to be tailored to the underlying cause of the misinformation problem, and efforts should be made to take on board inequalities within populations to create accurate, low-barrier, targeted health risk messaging. Skafle et al [ 24 ] contended that to challenge misconceptions, false claims need to be openly addressed and discussed with both cultural and religious awareness in mind. Guidance for practitioners noted that while responding to patient questions about alternative or unproven therapies may become laborious, a strong bond of trust between health practitioner and patient gives a patient a feeling of being supported and increases their adherence to treatment [ 52 ]. Rather than waiting for patients to raise misinformation issues, health care practitioners are advised to anticipate and proactively address potential misinformation and myths with patients. For example, the mortality rate for pediatric cancer has risen during the COVID-19 pandemic because of delayed access to medical care, but misinformation related to COVID-19 may also be a contributing factor [ 53 ]. The literature highlights the challenge of navigating the information and misinformation and the need for health practitioners to communicate with their patients more effectively. However, such efforts are not always successful. Some of the factors that may prevent effective communication of good health information have already been raised in this paper. They are revisited and discussed in the next section, along with other stressors for health practitioners.

Stressors for Health Practitioners

Challenges for health practitioners include time pressures and the additional burdens placed on them during the COVID-19 pandemic. These additional pressures add to the issues health practitioners face in trying to mitigate the impact of misinformation. The following is a brief overview of these issues.

On the one hand, administrative burdens placed on practitioners frequently deny them time for dialogue with their patients [ 52 ]. On the other, in different contexts, practitioners may be coping with a lack of proper facilities; poor infrastructure for patient care; insufficient or ineffective personal protective equipment; lack of awareness among the general population; poor compliance with preventive methods; and the fear of being infected with the virus, as they too are exposed to misinformation. During the COVID-19 pandemic, health practitioners were considered more vulnerable than other workers to developing psychological problems and other stress-related disorders, as they treated patients confirmed with COVID-19 while also dealing with misinformation [ 54 ].

As noted above, practitioners are recommended to invest in developing high levels of patient trust and to proactively correct health misinformation. However, recommendations presuppose that health practitioners necessarily have the resources to do these things well. Some of the materials produced to educate patients are not always reliable or evidence based, resulting ultimately in a loss of trust on the part of patients [ 52 ]. In addition, as noted previously, health practitioners themselves are not necessarily immune from accepting health misinformation as credible. Evidence about the level of knowledge and understanding of COVID-19 among practitioners reveals its unevenness. A study of dentists and oral health practitioners’ knowledge about COVID-19 suggested that their knowledge was at a relatively high level [ 55 ]. By contrast, a study of 310 eye care professionals in Nepal revealed some knowledge but also some acceptance of misinformation. Symptoms of COVID-19 were known to 94% of participants, but only 49% of participants were aware of how the disease is transmitted. More significantly, 41% of participants believed that the consumption of hot drinks helps to destroy the virus, in contradiction to World Health Organization information. The mean overall “knowledge” performance score, as measured by the benchmarks set by the researchers, was 69.65% [ 56 ].

A qualitative study to investigate primary health care practitioners’ perceptions and understanding of the COVID-19 pandemic was conducted in KwaZulu-Natal, South Africa. The study collected data from 15 participants at 2 different clinics situated in rural KwaZulu-Natal. Participants comprised nurses, physiotherapists, pharmacists, community caregivers, social workers, and clinical associates. Data were collected through individual, in-depth face-to-face interviews using a semistructured interview guide. The participants reported prepandemic and pandemic experiences of fear or denial. There was a perception of poor preparation for the COVID-19 outbreak. The findings also revealed participants’ misperceptions regarding the nature of the COVID-19 pandemic. Researchers concluded that respondents’ misunderstandings regarding the pandemic were primarily a result of misinformation found on social media [ 57 ].

The discussion in this section so far has highlighted the significant potential of health practitioners in mitigating the impact of online health misinformation. However, it has also underlined factors that may militate against health practitioners’ ability to do so effectively. Not least of these is the issue of health practitioners’ own knowledge, which coexists with other stressors for health practitioners in combating misinformation. The discussion will now consider health information management (HIM) as a tool for supporting health practitioners’ knowledge base as one element in a multifaceted strategy for combating misinformation on the web.

HIM as a Mitigation Strategy

We have seen there is a need for health practitioners to be supported with evidence-based knowledge that they can share with patients. Kyabaggu et al [ 5 ] argued that the COVID-19 pandemic has demonstrated that in an infectious health crisis, the gathering of accurate and reliable data to assist with the public health response is essential. They highlighted the importance of HIM professionals in supporting contact tracing and syndromic surveillance, as well as in mapping and forecasting health data. They noted that the generation of health information supports the continuum of care and the setting of targets and indicators and aids the planning, monitoring, and evaluation of health programs locally and globally. The health information produced also underpins the development of equitable, efficient, and accessible health care systems, contributing to improving public health initiatives and outcomes. Kyabaggu et al [ 5 ] emphasized the importance of an area of HIM, currently in its early stages, that deals with gathering and identifying evidence about the structural inequalities that underlie the disparities in vulnerability to health misinformation discussed in this paper. The collection of rich, high-quality information, including patient-reported experience, outcome measures, and culturally appropriate identity data, can enable health practitioners and public health advisers serving the most disadvantaged and underrepresented communities to use more tools of advocacy for patients.

The authors noted that advances in technology, including artificial intelligence, have the potential to relieve some of the pressures and constraints on health practitioners working on the front line during crises such as the COVID-19 pandemic, allowing more time for one-to-one engagement with patients. Kyabaggu et al [ 5 ] advocated for the content expertise of health information managers to serve health practitioners by delivering patient-facing information triaging services; constructing user-friendly knowledge representations, such as data visualizations; and developing information interpretation tools, such as decision aids, plain language summaries, and supplementary explanatory information and metadata. Kyabaggu et al [ 5 ] identified the interdisciplinary underpinnings of HIM as essential in contributing to the educational, informational, and decision-making support for addressing current and future infodemic management crises.

Summary of Results

Within the literature, there is a consensus that there exists a significant problem of online health misinformation disseminated via the internet on social network platforms, often by online health communities. It is apparent that while users seek trustworthy sources of health information, they are unequally equipped to assess its credibility. This is partly because some groups lack sufficient levels of health and digital literacies, which may be exacerbated by concomitant social and economic inequalities. Reception of, and response to, online health misinformation is also shaped by users’ cultural contexts, values, and experiences, which may hinder trust in scientific institutions and governments. Evidence suggests that some demographics are more vulnerable to accepting health misinformation as credible and that health practitioners are unevenly prepared in the context of new global health crises, such as the COVID-19 pandemic. Furthermore, the evidence of disparities in positive and negative attitudes toward vaccination highlights a need to pay specific attention to regional and national settings, even in the current global context. Preexisting levels of local trust in vaccine providers may be a significant factor to consider. While the validity and reliability of YouGov polls are limited, nevertheless, the data from an admittedly narrow range of sources suggests that vaccine confidence may have become more fluctuating and potentially vulnerable to destabilization in the digital era.

While online mitigation strategies such as user correction and automatic detection may have their uses, their effectiveness is contested, and some studies suggest they may even be counterproductive. Our analysis of the available literature indicates that the effectiveness of these strategies varies and needs further evaluation [ 42 , 58 ]. The issue of online health misinformation is further complicated by the operation of malicious actors and politicization of the issue, particularly during the COVID-19 pandemic, militating against the equitable and trusted dissemination of evidence-based knowledge. The role of health practitioners in this context is a challenging one. Research suggests that on the one hand, they are still best placed, at the front line of care, to combat health misinformation with science-based knowledge and advice. On the other hand, the stressors identified in this review create barriers to their abilities to do this well. Constraints of time and lack of supporting infrastructure add to the knowledge deficit noted earlier. Our review underlines the complexity of the environment in which health practitioners operate and calls for greater support and resources to enable effective mitigation of health misinformation [ 59 ]. Investment in HIM at local and global levels could address all 3 deficits, creating the potential for health practitioners to enhance their capacity to build trust via knowledgeable one-to-one communication with patients.

Limitations

The limitations of this study are the following: First, the constraints of time and space have necessarily limited the scale and scope of the survey. Second, the study of online health misinformation is a growing field, and inevitably, the nature of the issue means that new evidence is emerging at a rapid rate. In particular, new knowledge and further reflection in the wake of the COVID-19 pandemic will continue to shed new light on the subject. Our study acknowledges these limitations and emphasizes the dynamic nature of the field.

Conclusions

Our survey of the literature on online health misinformation has revealed a complex and multifaceted context in which health practitioners must operate. As the world renormalizes following the pandemic, a collaborative global interdisciplinary effort to provide equitable access to timely, accurate, and complete health information will be needed to support health practitioners in combating the impact of online health misinformation. Academic research will need to be disseminated into the public domain in a way that is accessible to the public to counter misinformation and educate populations concerning how science is carried out. Our conclusions drawn from this review stress the urgency of effective strategies and collaborative efforts to mitigate the prevalence and impact of health misinformation on a global scale. Without strategies for equipping populations with the health and digital literacies required to interpret and use information appropriately, the prevalence of online health misinformation will continue to pose a threat to global public health efforts, disproportionately affecting vulnerable and resource-limited populations. Although social media platforms have a responsibility to correct misinformation, governments will need to engage in evidence-informed decision-making and invest in HIM to support frontline health practitioners in their work, enhance population health literacy, and strengthen evidence-informed decision-making at all levels.

Several issues for further investigation arise from the findings of this review. These include the following:

• The long-term impact of COVID-19 vaccine hesitancy
• Whether the COVID-19 pandemic has intensified or diminished information literacy, and the related question of whether the pandemic will incentivize health information literacy
• The effects of social and cultural differences on the long-term traction of future health misinformation
• Whether social and economic inequalities will become less or more pronounced in the face of a global pandemic
• The comparative effectiveness of strategies to enhance populations’ media and digital literacies to facilitate the mitigation of health misinformation and its effects
• The influence of state actors on the propagation of health misinformation on the web
• The extent to which academic research has been disseminated into the public domain in a way that is accessible to the public, and the effectiveness of strategies to do so to counter misinformation and educate populations concerning how science is carried out

Acknowledgments

This research was funded by the School of Computing and Communications at the Open University. It allowed researchers across several faculties to collaborate and build a research team that focused on the experience of health practitioners with misinformation and its impact on their job practice. The authors would also like to thank Tracie Farrell and Nashwa Ismail for their invaluable suggestions and recommendations, as well as their assistance in the article screening process.

Data Availability

The data analyzed in this study are derived from published articles available on Google Scholar. All articles included in the review are cited in the reference list. No additional data or code were collected or generated as part of this study.

Authors' Contributions

The study was conceptualized by DK; funding acquisition was managed by DK; data were curated by DK, AK, MM, and IK; formal analysis was conducted by DK and MM; the investigation was carried out by AK and MM; the methodology was designed by DK and MM; project administration was overseen by DK; resources were provided by DK; supervision was carried out by DK; validation was conducted by DK, AK, MM, and IK; visualization was handled by DK and MM; writing (original draft preparation) was done by DK; and writing (review and editing) was carried out by DK, AK, and MM. All authors reviewed and approved the final version.

Conflicts of Interest

None declared.

Detailed search strategy.

• Ismail N, Kbaier D, Farrell T, Kane A. The experience of health professionals with misinformation and its impact on their job practice: qualitative interview study. JMIR Form Res. Nov 02, 2022;6(11):e38794. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tully M, Bode L, Vraga EK. Mobilizing users: does exposure to misinformation and its correction affect users’ responses to a health misinformation post? Soc Media Soc. Dec 10, 2020;6(4):205630512097837. [ CrossRef ]
• Zhao Y, Da J, Yan J. Detecting health misinformation in online health communities: Incorporating behavioral features into machine learning based approaches. Inf Process Manage. Jan 2021;58(1):102390. [ CrossRef ]
• Wang Y, McKee M, Torbica A, Stuckler D. Systematic literature review on the spread of health-related misinformation on social media. Soc Sci Med. Nov 2019;240:112552. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kyabaggu R, Marshall D, Ebuwei P, Ikenyei U. Health literacy, equity, and communication in the COVID-19 era of misinformation: emergence of health information professionals in infodemic management. JMIR Infodemiology. 2022;2(1):e35014. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sutton J. Health communication trolls and bots versus public health agencies' trusted voices. Am J Public Health. Oct 2018;108(10):1281-1282. [ CrossRef ] [ Medline ]
• Kim S, Capasso A, Ali SH, Headley T, DiClemente RJ, Tozan Y. What predicts people's belief in COVID-19 misinformation? A retrospective study using a nationwide online survey among adults residing in the United States. BMC Public Health. Nov 18, 2022;22(1):2114. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Morgan JC, Cappella JN. The effect of repetition on the perceived truth of tobacco-related health misinformation among U.S. adults. J Health Commun. Mar 04, 2023;28(3):182-189. [ CrossRef ] [ Medline ]
• Nan X, Wang Y, Thier K. Why do people believe health misinformation and who is at risk? A systematic review of individual differences in susceptibility to health misinformation. Soc Sci Med. Dec 2022;314:115398. [ CrossRef ] [ Medline ]
• Wood JL, Lee GY, Stinnett SS, Southwell BG. A pilot study of medical misinformation perceptions and training among practitioners in North Carolina (USA). Inquiry. 2021;58:469580211035742. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Broniatowski DA, Jamison AM, Qi S, AlKulaib L, Chen T, Benton A, et al. Weaponized health communication: Twitter bots and Russian trolls amplify the vaccine debate. Am J Public Health. Oct 2018;108(10):1378-1384. [ CrossRef ] [ Medline ]
• Mokhtari H, Mirzaei A. The tsunami of misinformation on COVID-19 challenged the health information literacy of the general public and the readability of educational material: a commentary. Public Health. Oct 2020;187:109-110. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Schmid P, Altay S, Scherer LD. The psychological impacts and message features of health misinformation - a systematic review of randomized controlled trials. Eur Psychol. Jul 2023;28(3):162-172. [ CrossRef ]
• Southwell BG, Otero Machuca J, Cherry ST, Burnside M, Barrett NJ. Health misinformation exposure and health disparities: observations and opportunities. Annu Rev Public Health. Apr 03, 2023;44(1):113-130. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Westberry C, Palmer XL, Potter L. Social media and health misinformation: a literature review. In: Arai K, editor. Proceedings of the Future Technologies Conference (FTC) 2023, Volume 3. FTC 2023. Lecture Notes in Networks and Systems, vol 815. Cham, Switzerland. Springer; 2023.
• Suarez-Lledo V, Alvarez-Galvez J. Prevalence of health misinformation on social media: systematic review. J Med Internet Res. Jan 20, 2021;23(1):e17187. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Borges do Nascimento IJ, Pizarro AB, Almeida J, Azzopardi-Muscat N, Gonçalves MA, Björklund M, et al. Infodemics and health misinformation: a systematic review of reviews. Bull World Health Organ. Sep 01, 2022;100(9):544-561. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lee JJ, Kang K, Wang MP, Zhao SZ, Wong JYH, O'Connor S, et al. Associations between COVID-19 misinformation exposure and belief with COVID-19 knowledge and preventive behaviors: cross-sectional online study. J Med Internet Res. Nov 13, 2020;22(11):e22205. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Larson HJ, Broniatowski DA. Why debunking misinformation is not enough to change people's minds about vaccines. Am J Public Health. Jun 2021;111(6):1058-1060. [ CrossRef ] [ Medline ]
• Bapaye JA, Bapaye HA. Demographic factors influencing the impact of coronavirus-related misinformation on WhatsApp: cross-sectional questionnaire study. JMIR Public Health Surveill. Jan 30, 2021;7(1):e19858. [ FREE Full text ] [ CrossRef ] [ Medline ]
• McKernon E. Fake news and the public. Harper's. Oct 1925. URL: https://harpers.org/archive/1925/10/fake-news-and-the-public/ [accessed 2024-07-30]
• Hotez P. The physician-scientist: defending vaccines and combating antiscience. J Clin Invest. Apr 29, 2019;129(6):2169-2171. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Dyer C. Lancet retracts Wakefield's MMR paper. BMJ. Mar 02, 2010;340:c696. [ CrossRef ] [ Medline ]
• Skafle I, Nordahl-Hansen A, Quintana DS, Wynn R, Gabarron E. Misinformation about COVID-19 vaccines on social media: rapid review. J Med Internet Res. Aug 04, 2022;24(8):e37367. [ FREE Full text ] [ CrossRef ] [ Medline ]
• New Yahoo News/YouGov coronavirus poll: almost 1 in 5 say they won't get vaccinated. Yahoo News. URL: https:/​/www.​yahoo.com/​news/​new-yahoo-news-you-gov-coronavirus-poll-almost-one-in-five-say-they-wont-get-vaccinated-143852222.​html [accessed 2024-08-06]
• Peretti-Watel P, Verger P, Raude J, Constant A, Gautier A, Jestin C, et al. Dramatic change in public attitudes towards vaccination during the 2009 influenza A(H1N1) pandemic in France. Euro Surveill. Oct 31, 2013;18(44):20623. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Reuters. ‘They’re killing people’: Biden slams Facebook for Covid disinformation. The Guardian. Jul 21, 2021. URL: https:/​/www.​theguardian.com/​media/​2021/​jul/​17/​theyre-killing-people-biden-slams-facebook-for-covid-misinformation [accessed 2024-07-30]
• Britain has had enough of experts, says Gove. Financial Times. URL: https://www.ft.com/content/3be49734-29cb-11e6-83e4-abc22d5d108c [accessed 2024-07-30]
• US news roundup: 19-25 August. Research Professional News. Aug 25, 2022. URL: https://www.researchprofessionalnews.com/rr-news-usa-2022-8-us-news-roundup-19-25-august/ [accessed 2024-07-30]
• Gruzd A, Mai P, Soares FB. How coordinated link sharing behavior and partisans' narrative framing fan the spread of COVID-19 misinformation and conspiracy theories. Soc Netw Anal Min. 2022;12(1):118. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Vosoughi S, Roy D, Aral S. The spread of true and false news online. Science. Mar 09, 2018;359(6380):1146-1151. [ CrossRef ] [ Medline ]
• Wang Y. Systematic review on the social mechanism of health misinformation dissemination in the internet era. Eur J Public Health. Nov 2018;28(suppl_4):cky213.194. [ CrossRef ]
• Framework for information literacy for higher education. American Library Association. 2016. URL: http://www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/infolit/framework1.pdf [accessed 2024-07-30]
• Quraishi Z. Addressing mental health, misinformation, and religious tensions among South Asian students across California higher education during the COVID-19 pandemic: A qualitative research study. Heliyon. Jun 2023;9(6):e16396. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zhou J, Xiang H, Xie B. Better safe than sorry: a study on older adults' credibility judgments and spreading of health misinformation. Univers Access Inf Soc. Aug 04, 2022;22(3):1-10. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Goldenberg MJ. Vaccine Hesitancy: Public Trust, Expertise, and the War on Science. Pittsburgh, PA. University of Pittsburgh Press; 2021.
• Ittefaq M. "It frustrates me beyond words that I can't fix that": health misinformation correction on Facebook during COVID-19. Health Commun. Nov 12, 2023;12:1-11. [ CrossRef ] [ Medline ]
• Mourali M, Drake C. The challenge of debunking health misinformation in dynamic social media conversations: online randomized study of public masking during COVID-19. J Med Internet Res. Mar 02, 2022;24(3):e34831. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Weinzierl MA, Harabagiu SM. Automatic detection of COVID-19 vaccine misinformation with graph link prediction. J Biomed Inform. Dec 2021;124:103955. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Petty RE, Cacioppo JT. Elaboration likelihood model of persuasion. In: Communication and Persuasion. Springer Series in Social Psychology. New York, NY. Springer; 1986.
• Cowles K, Miller R, Suppok R. When seeing isn't believing: navigating visual health misinformation through library instruction. Med Ref Serv Q. 2024;43(1):44-58. [ CrossRef ] [ Medline ]
• Nabożny A, Balcerzak B, Morzy M, Wierzbicki A, Savov P, Warpechowski K. Improving medical experts' efficiency of misinformation detection: an exploratory study. World Wide Web. 2023;26(2):773-798. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Barve Y, Saini JR. Detecting and classifying online health misinformation with 'Content Similarity Measure (CSM)' algorithm: an automated fact-checking-based approach. J Supercomput. 2023;79(8):9127-9156. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Broniatowski D, Gu J, Jamison AM, Simons JR. Facebook's architecture undermines vaccine misinformation removal efforts. arXiv. Preprint posted online Feb 4, 2022. [ FREE Full text ] [ CrossRef ]
• Russell N. Misinformation during COVID: how should nurse practitioners respond? J Nurse Pract. Jun 2021;17(6):763-764. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wilner T, Holton A. Breast cancer prevention and treatment: misinformation on Pinterest, 2018. Am J Public Health. Oct 2020;110(S3):S300-S304. [ CrossRef ] [ Medline ]
• MacFarlane D, Hurlstone MJ, Ecker UK. Protecting consumers from fraudulent health claims: A taxonomy of psychological drivers, interventions, barriers, and treatments. Soc Sci Med. Aug 2020;259:112790. [ CrossRef ] [ Medline ]
• Thompson JD. Public health pedagogy and digital misinformation: health professional influencers and the politics of expertise. J Sociol. Sep 28, 2022;59(3):646-663. [ CrossRef ]
• Gunter J. Medical misinformation and the internet: a call to arms. Lancet. Jun 08, 2019;393(10188):2294-2295. [ CrossRef ] [ Medline ]
• Sui Y, Zhang B. Determinants of the perceived credibility of rebuttals concerning health misinformation. Int J Environ Res Public Health. Mar 02, 2021;18(3):1345. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Collier R. Containing health myths in the age of viral misinformation. CMAJ. May 14, 2018;190(19):E578. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Peterson JS, Swire-Thompson B, Johnson SB. What is the alternative? Responding strategically to cancer misinformation. Future Oncol. Sep 2020;16(25):1883-1888. [ CrossRef ] [ Medline ]
• Guidry JP, Miller CA, Ksinan AJ, Rohan JM, Winter MA, Carlyle KE, et al. COVID-19-related misinformation among parents of patients with pediatric cancer. Emerg Infect Dis. Mar 2021;27(2):650-652. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ali S, Khalid A, Zahid E. Is COVID-19 immune to misinformation? A brief overview. Asian Bioeth Rev. Jun 2021;13(2):255-277. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jafari A, Mohammadpour M, Ghanbarzadegan A, Rossi-Fedele G, Bastani P. Oral health practitioners' knowledge, attitude, and awareness about coronavirus: A systematic review and meta-analysis. J Educ Health Promot. 2021;10:39. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sanyam SD, Sah SK, Chaudhary P, Burton MJ, Hoffman JJ. Knowledge and awareness-based survey of COVID-19 within the eye care profession in Nepal: Misinformation is hiding the truth. PLoS One. 2021;16(7):e0254761. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Nxumalo CT, Mchunu GG. A qualitative study to explore primary health care practitioners' perceptions and understanding regarding the COVID-19 pandemic in KwaZulu-Natal, South Africa. Afr J Prim Health Care Fam Med. Nov 26, 2021;13(1):e1-e11. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Abbasi K. Where now in the danse macabre of COVID-19 and misinformation? BMJ. Aug 17, 2023;382:1884. [ CrossRef ]
• Pesko MF, Cummings KM, Douglas CE, Foulds J, Miller T, Rigotti NA, et al. United States public health officials need to correct e-cigarette health misinformation. Addiction. May 2023;118(5):785-788. [ CrossRef ] [ Medline ]

Abbreviations

Edited by G Eysenbach, T Leung; submitted 15.04.22; peer-reviewed by G Nneji, S-F Tsao; comments to author 07.06.22; revised version received 29.09.22; accepted 12.07.24; published 19.08.24.

©Dhouha Kbaier, Annemarie Kane, Mark McJury, Ian Kenny. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 19.08.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

Exploring the factors driving AI adoption in production: a systematic literature review and future research agenda

• Open access
• Published: 23 August 2024

Cite this article

You have full access to this open access article

• Heidi Heimberger   ORCID: orcid.org/0000-0003-3390-0219 1 , 2 ,
• Djerdj Horvat   ORCID: orcid.org/0000-0003-3747-3402 1 &
• Frank Schultmann   ORCID: orcid.org/0000-0001-6405-9763 1  

Our paper analyzes the current state of research on artificial intelligence (AI) adoption from a production perspective. We represent a holistic view on the topic which is necessary to get a first understanding of AI in a production-context and to build a comprehensive view on the different dimensions as well as factors influencing its adoption. We review the scientific literature published between 2010 and May 2024 to analyze the current state of research on AI in production. Following a systematic approach to select relevant studies, our literature review is based on a sample of articles that contribute to production-specific AI adoption. Our results reveal that the topic has been emerging within the last years and that AI adoption research in production is to date still in an early stage. We are able to systematize and explain 35 factors with a significant role for AI adoption in production and classify the results in a framework. Based on the factor analysis, we establish a future research agenda that serves as a basis for future research and addresses open questions. Our paper provides an overview of the current state of the research on the adoption of AI in a production-specific context, which forms a basis for further studies as well as a starting point for a better understanding of the implementation of AI in practice.

Explore related subjects

• Artificial Intelligence

Avoid common mistakes on your manuscript.

1 Introduction

The technological change resulting from deep digitisation and the increasing use of digital technologies has reached and transformed many sectors [ 1 ]. In manufacturing, the development of a new industrial age, characterized by extensive automation and digitisation of processes [ 2 ], is changing the sector’s ‘technological reality’ [ 3 ] by integrating a wide range of information and communication technologies (such as Industry 4.0-related technologies) into production processes [ 4 ].

Although the evolution of AI traces back to the year 1956 (as part of the Dartmouth Conference) [ 5 ], its development has progressed rapidly, especially since the 2010s [ 6 ]. Driven by improvements, such as the fast and low-cost development of smart hardware, the enhancement of algorithms as well as the capability to manage big data [ 7 ], there is an increasing number of AI applications available for implementation today [ 8 ]. The integration of AI into production processes promises to boost the productivity, efficiency as well as automation of processes [ 9 ], but is currently still in its infancy [ 10 ] and manufacturing firms seem to still be hesitant to adopt AI in a production-context. This appears to be driven by the high complexity of AI combined with the lack of practical knowledge about its implementation in production and several other influencing factors [ 11 , 12 ].

In the literature, many contributions analyze AI from a technological perspective, mainly addressing underlying models, algorithms, and developments of AI tools. Various authors characterise both machine learning and deep learning as key technologies of AI [ 8 , 13 ], which are often applied in combination with other AI technologies, such as natural language recognition. While promising areas for AI application already exist in various domains such as marketing [ 14 ], procurement [ 15 ], supply chain management [ 16 ] or innovation management [ 17 ], the integration of AI into production processes also provides significant performance potentials, particularly in the areas of maintenance [ 18 ], quality control [ 19 ] and production planning and management [ 20 ]. However, AI adoption requires important technological foundations, such as the provision of data and the necessary infrastructure, which must be ensured [ 11 , 12 , 21 ]. Although the state of the art literature provides important insights into possible fields of application of AI in production, the question remains: To what extent are these versatile applications already in use and what is required for their successful adoption?

Besides the technology perspective of AI, a more human-oriented field of discussion is debated in scientific literature [ 22 ]. While new technologies play an essential role in driving business growth in the digital transformation of the production industry, the increasing interaction between humans and intelligent machines (also referred to as ‘augmentation’) creates stress challenges [ 23 ] and impacts work [ 24 ], which thus creates managerial challenges in organizations [ 25 , 26 ]. One of the widely discussed topics in this context is the fear of AI threatening jobs (including production jobs), which was triggered by e.g. a study of Frey, Osborne [ 27 ]. Another issue associated to the fear of machines replacing humans is the lack of acceptance resulting from the mistrust of technologies [ 28 , 29 ]. This can also be linked to the various ethical challenges involved in working with AI [ 22 ]. This perspective, which focuses on the interplay between AI and humans [ 30 ], reveals the tension triggered by AI. Although this is discussed from different angles, the question remains how these aspects influence the adoption of AI in production.

Another thematic stream of current literature can be observed in a series of contributions on the organizational aspects of the technology. In comparison to the two research areas discussed above, the number of publications in this area seems to be smaller. This perspective focuses on issues to implement AI, such as the importance of a profound management structure [ 31 , 32 ], leadership [ 33 ], implications on the organizational culture [ 34 ] as well as the need for digital capabilities and special organizational skills [ 33 ]. Although some studies on the general adoption of AI without a sectoral focus have already been conducted (such as by Chen, Tajdini [ 35 ] or Kinkel, Baumgartner, Cherubini [ 36 ]) and hence, some initial factors influencing the adoption of AI can be derived, the contributions from this perspective are still scarce, are usually not specifically analyzed in the context of production or lack a comprehensive view on the organization in AI adoption.

While non-industry specific AI issues have been researched in recent years, the current literature misses a production-specific analysis of AI adoption, providing an understanding of the possibilities and issues related to integrating AI into the production context. Moreover, the existing literature tells us little about relevant mechanisms and factors underlying the adoption of AI in production processes, which include both technical, human-centered as well as organizational issues. As organizational understanding of AI in a business context is currently still in its early stages, it is difficult to find an aggregate view on the factors that can support companies in implementing AI initiatives in production [ 37 , 38 ]. Addressing this gap, we aim to systematise the current scientific knowledge on AI adoption, with a focus on production. By drawing on a systematic literature review (SLR), we examine existing studies on AI adoption in production and explore the main issues regarding adoption that are covered in the analyzed articles. Building on these findings, we conduct a comprehensive analysis of the existing studies with the aim of systematically investigating the key factors influencing the adoption of AI in production. This systematic approach paves the way for the formulation of a future research agenda.

Our SLR addresses three research questions (RQs). RQ1: What are the statistical characteristics of existing research on AI adoption in production? To answer this RQ, we conduct descriptive statistics of the analyzed studies and provide information on time trends, methods used in the research, and country specifications. RQ2: What factors influence the adoption of AI in production? RQ2 specifies the adoption factors and forms the core component of our analysis. By adoption factors, we mean the factors that influence the use of AI in production (both positively and negatively) and that must therefore be analyzed and taken into account. RQ3: What research topics are of importance to advance the research field of AI adoption in production? We address this RQ by using the analyzed literature as well as the key factors of AI adoption as a starting point to derive RQs that are not addressed and thus provide an outlook on the topic.

2 Methodology

In order to create a sound information base for both policy makers and practitioners on the topic of AI adoption in production, this paper follows the systematic approach of a SLR. For many fields, including management research, a SLR is an important tool to capture the diversity of existing knowledge on a specific topic for a scientific investigation [ 39 ]. The investigator often pursues multiple goals, such as capturing and assessing the existing environment and advancing the existing body of knowledge with a proprietary RQ [ 39 ] or identifying key research topics [ 40 ].

Our SLR aims to select, analyze, and synthesize findings from the existing literature on AI adoption in production over the past 24 years. In order to identify relevant data for our literature synthesis, we follow the systematic approach of the Preferred Reporting Items for Systematic reviews (PRISMA) [ 41 ]. In evaluating the findings, we draw on a mixed-methods approach, combining some quantitative analyses, especially on the descriptive aspects of the selected publications, as well as qualitative analyses aimed at evaluating and comparing the contents of the papers. Figure  1 graphically summarizes the methodological approach that guides the content of the following sub-chapters.

Methodical procedure of our SLR following PRISMA [ 41 ]

2.1 Data identification

Following the development of the specific RQs, we searched for suitable publications. To locate relevant studies, we chose to conduct a publication analysis in the databases Scopus, Web of Science and ScienceDirect as these databases primarily contain international scientific articles and provide a broad overview of the interdisciplinary research field and its findings. To align the search with the RQs [ 42 ], we applied predefined key words to search the titles, abstracts, and keywords of Scopus, Web of Science and ScienceDirect articles. Our research team conducted several pre-tests to determine the final search commands for which the test results were on target and increased the efficiency of the search [ 42 ]. Using the combination of Boolean operators, we covered the three topics of AI, production, and adoption by searching combinations of ‘Artificial Intelligence’ AND ‘production or manufacturing’ AND ‘adopt*’ in the three scientific databases. Although ‘manufacturing’ tends to stand for the whole sector and ‘production’ refers to the process, the two terms are often used to describe the same context. We also follow the view of Burbidge, Falster, Riis, Svendsen [ 43 ] and use the terms synonymously in this paper and therefore also include both terms as keywords in the study location as well as in the analysis.

AI research has been credited with a resurgence since 2010 [ 6 ], which is the reason for our choice of time horizon. Due to the increase in publications within the last years, we selected articles published online from 2010 to May 8, 2024 for our analysis. As document types, we included conference papers, articles, reviews, book chapters, conference reviews as well as books, focusing exclusively on contributions in English in the final publication stage. The result of the study location is a list of 3,833 documents whose titles, abstracts, and keywords meet the search criteria and are therefore included in the next step of the analysis.

2.2 Data analysis

For these 3,833 documents, we then conducted an abstract analysis, ‘us[ing] a set of explicit selection criteria to assess the relevance of each study found to see if it actually does address the research question’ [ 42 ]. For this step, we again conducted double-blind screenings (including a minimum of two reviewers) as pilot searches so that all reviewers have the same understanding of the decision rules and make equal decisions regarding their inclusion for further analysis.

To ensure the paper’s focus on all three topics regarded in our research (AI, production, and adoption), we followed clearly defined rules of inclusion and exclusion that all reviewers had to follow in the review process. As a first requirement for inclusion, AI must be the technology in focus that is analysed in the publication. If AI was only mentioned and not further specified, we excluded the publication. With a second requirement, we checked the papers for the context of analysis, which in our case must be production. If the core focus is beyond production, the publication was also excluded from further analysis. The third prerequisite for further consideration of the publication is the analysis of the adoption of a technology in the paper. If technology adoption is not addressed or adoption factors are not considered, we excluded the paper. An article was only selected for full-text analysis if, after analyzing the titles, abstracts, and keywords, a clear focus on all three research areas was visible and the inclusion criteria were met for all three contexts.

By using this tripartite inclusion analysis, we were able to analyse the publications in a structured way and to reduce the 3,833 selected documents in our double-blind approach to 300 articles that were chosen for the full-text analysis. In the process of finding full versions of these publications, we had to exclude three papers as we could not access them. For the rest of the 297 articles we obtained full access and thus included them for further analysis. After a thorough examination of the full texts, we again had to exclude 249 publications because they did not meet our content-related inclusion criteria mentioned above, although the abstract analysis gave indications that they did. As a result, we finally obtained 47 selected papers on which we base the literature analysis and synthesis (see Fig.  1 ).

2.3 Descriptive analysis

Figure  2 summarises the results of the descriptive analysis on the selected literature regarding AI adoption in production that we analyse in our SLR. From Fig.  2 a), which illustrates annual publication trends (2010–2024), the increase in publications on AI adoption in production over the past 5 years is evident, yet slightly declining after a peak in 2022. After a steady increase until 2022, in which 11 articles are included in the final analysis, 2023 features ten articles, followed by three articles for 2024 until the cut-off date in May 2024. Of the 47 papers identified through our search, the majority (n = 33) are peer-reviewed journal articles and the remaining thirteen contributions conference proceedings and one book chapter (see Fig.  2 b)).

Descriptive analyses of the selected articles addressing AI adoption in production

The identified contributions reveal some additional characteristics in terms of the authors country base (Fig.  2 c)) and research methods used (Fig.  2 d)). Almost four out of ten of the publications were written in collaboration with authors from several countries (n = 19). Six of the papers were published by authors from the United States, five from Germany and four from India. In terms of the applied research methods used by the researchers, a wide range of methods is used (see Fig.  2 c), with qualitative methods (n = 22) being the most frequently used.

2.4 Factor analysis

In order to derive a comprehensive list of factors that influence the use of AI in production at different levels, we follow a qualitative content analysis. It is based on inductive category development, avoiding prefabricated categories in order to allow new categories to emerge based on the content at hand [ 44 , 45 ]. To do this, we first read the entire text to gain an understanding of the content and then derive codes [ 46 ] that seem to capture key ideas [ 45 ]. The codes are subsequently sorted into distinct categories, each of which is clearly defined and establishes meaningful connections between different codes. Based on an iterative process with feedback loops, the assigned categories are continuously reviewed and updated as revisions are made [ 44 ].

Various factors at different levels are of significance to AI and influence technology adoption [ 47 , 48 ]. To identify the specific factors that are of importance for AI adoption in production, we analyze the selected contributions in terms of the factors considered, compare them with each other and consequently obtain a list of factors through a bottom-up approach. While some of the factors are based on empirical findings, others are expected factors that result from the research findings of the respective studies. Through our analysis, a list of 35 factors emerges that influence AI adoption in production which occur with varying frequency in the studies analyzed by our SLR. Table 1 visualizes each factor in the respective contributions sorted by the frequency of occurrence.

The presence of skills is considered a particularly important factor in AI adoption in the studies analyzed (n = 35). The availability of data (n = 25) as well as the need for ethical guidelines (n = 24) are also seen as key drivers of AI adoption, as data is seen as the basis for the implementation of AI and ethical issues must be addressed in handling such an advanced technology. As such, these three factors make up the accelerants of AI adoption in production that are most frequently cited in the studies analyzed.

Also of importance are issues of managerial support (n = 22), as well as performance measures and IT infrastructure (n = 20). Some factors were also mentioned, but only addressed by one study at a time: government support, industrial sector, product complexity, batch size, and R&D Intensity. These factors are often used as quantitatively measurable adoption factors, especially in empirical surveys, such the study by Kinkel, Baumgartner, Cherubini [ 36 ].

3 Factors influencing AI adoption

The 35 factors presented characteristically in Sect.  2.4 serve as the basis for our in-depth analysis and for developing a framework of influences on AI adoption in production which are grouped into supercategories. A supercategory describes a cluster of topics to which various factors of AI adoption in production can be assigned. We were able to define seven categories that influence AI adoption in production: the internal influences of ‘business and structure’, ‘organizational effectiveness’, ‘technology and system’, ‘data management’ as well as the external influences of the ‘regulatory environment’, ‘business environment’ and ‘economic environment’ (see Fig.  3 ). The factors that were mentioned most frequently (occurrence in at least half of the papers analyzed) are marked accordingly (*) in Fig.  3 .

Framework of factors influencing AI adoption in production

3.1 Internal Environment

The internal influences on AI adoption in production refer to factors that an organization carries internally and that thus also influence adoption from within. Such factors can usually be influenced and clearly controlled by the organization itself.

3.1.1 Business and structure

The supercategory ‘business and structure’ includes the various factors and characteristics that impact a company’s performance, operations, and strategic decision-making. By considering and analyzing these business variables when implementing AI in production processes, companies can develop effective strategies to optimize their performance, increase their competitiveness, and adapt to changes in the business environment.

To understand and grasp the benefits in the use of AI, quantitative performance measures for the current and potential use of AI in industrial production systems help to clarify the value and potential benefits of AI use [ 49 , 54 , 74 , 79 , 91 ]. Assessing possible risks [ 77 ] as well as the monetary expected benefits for AI (e.g. Return on Investment (ROI)) in production plays an important role for adoption decisions in market-oriented companies [ 57 , 58 , 63 , 65 , 78 ]. Due to financial constraints, managers behave cautiously in their investments [ 78 ], so they need to evaluate AI adoption as financially viable to want to make the investment [ 61 , 63 , 93 ] and also drive acceptance [ 60 ]. AI systems can significantly improve cost–benefit structures in manufacturing, thereby increasing the profitability of production systems [ 73 ] and making companies more resilient [ 75 ]. However, in most cases, the adoption of AI requires high investments and the allocation of resources (s.a. personnel or financial) for this purpose [ 50 , 51 , 57 , 80 , 94 ]. Consequently, a lack of budgets and high expected transition costs often hinder the implementation of smart concepts [ 56 , 62 , 67 , 82 , 84 , 92 ]. It is up to management to provide necessary funding for AI adoption [ 53 , 59 , 79 ], which is required, for example, for skill development of employees [ 59 , 61 , 63 ], IT adaptation [ 62 , 66 ], AI development [ 74 ] or hardware deployment [ 68 ]. In their empirical study, Kinkel, Baumgartner, Cherubini [ 36 ] confirm a positive correlation between company size and the intensity in the use of AI technologies. Large companies generally stand out with a higher propensity to adopt [ 53 ] as they have less difficulties in comparison to small firms regarding the availability of resources [ 69 ], such as know-how, budget [ 68 , 84 ] and general data organization [ 68 ]. Others argue that small companies tend to be more open to change and are characterized by faster decision-making processes [ 68 , 93 ]. Product complexity also influences a company’s propensity for AI. Companies that produce rather simple products are more likely to digitize, which in turn offers good starting points for AI adoption. On the other hand, complex product manufacturers (often characterized by small batch sizes) are often less able to standardize and automate [ 36 ]. The company’s produced batch size has a similar influence on AI adoption. Small and medium batch sizes in particular hinder the integration of intelligent technologies, as less automation often prevails here as well. Nevertheless, even small and medium lot sizes can benefit economically from AI [ 36 ]. Since a high R&D intensity indicates a high innovation capability of a company, it is assumed to have a positive influence on AI adoption, as companies with a high R&D intensity already invest heavily in and use new innovations. This in turn speaks for existing competencies, know how and structures [ 36 ].

3.1.2 Organizational effectiveness

This supercategory focuses on the broader aspects that contribute to the effectiveness, development, and success of an organization when implementing AI in a production context. As the factors are interconnected and influence each other, decision makers should consider them carefully.

Users´ trust in AI is an essential factor to enable successful AI adoption and use in production [ 52 , 68 , 78 , 79 , 88 , 90 ]. From the users´ perspective, AI often exhibits the characteristics of a black box because its inherent processes are not fully understood [ 50 , 90 ] which can lead individuals to develop a fear towards the unknown [ 71 ]. Because of this lack of understanding, successful interaction between humans and AI is not guaranteed [ 90 ], as trust is a foundation for decisions that machines are intended to make autonomously [ 52 , 91 ]. To strengthen faith in AI systems [ 76 , 80 ], AI users can be involved in AI design processes in order to understand appropriate tools [ 54 , 90 ]. In this context, trust is also discussed in close connection with transparency and regulation [ 79 ]. User resistance is considered a barrier to implementing new information technologies, as adoption requires change [ 53 , 62 , 92 ]. Ignorance, as a kind of resistance to change, is a main obstacle to successful digital transformation [ 51 , 56 , 65 ]. Some employees may resist the change brought about by AI because they fear losing their jobs [ 52 ] or have other concerns [ 78 ]. Overcoming resistance to technology adoption requires organizational change and is critical for the success of adoption [ 50 , 51 , 62 , 67 , 71 , 80 ]. Therefore, change management is important to create awareness of the importance of AI adoption and increase acceptance of the workforce [ 66 , 68 , 74 , 83 ]. Management commitment is seen as a significant driver of technology adoption [ 53 , 59 , 81 , 82 , 86 ] and a lack of commitment can negatively impact user adoption and workforce trust and lead to skepticism towards technology [ 86 ]. The top management’s understanding and support for the benefits of the adopted technology [ 53 , 56 , 67 , 78 , 93 , 94 ] enhances AI adoption, can prioritize its implementation and also affects the performance of the AI-enabled application [ 55 , 60 , 83 ]. Preparing, enabling, and thus empowering the workforce, are considered the management’s responsibility in the adoption of digital technologies [ 59 , 75 ]. This requires intelligent leadership [ 52 ] as decision makers need to integrate their workforce into decision-making processes [ 75 ]. Guidelines can support managers by providing access to best practices that help in the adoption of AI [ 50 ]. Critical measures to manage organizational change include the empowerment of visionaries or appointed AI champions leading the change and the collaborative development of digital roadmaps [ 54 , 62 ]. To demonstrate management commitment, managers can create such a dedicated role, consisting of an individual or a small group that is actively and enthusiastically committed to AI adoption in production. This body is considered the adoption manager, point of contact and internal driver of adoption [ 62 , 74 , 80 ]. AI initiatives in production do not necessarily have to be initiated by management. Although management support is essential for successful AI adoption, employees can also actively drive integration initially and thus realize pilot projects or initial trials [ 66 , 80 ]. The development of strategies as well as roadmaps is considered another enabling and necessary factor for the adoption of AI in production [ 50 , 53 , 54 , 62 , 71 , 93 ]. While many major AI strategies already exist at country level to further promote research and development of AI [ 87 ], strategy development is also important at the firm level [ 76 , 77 , 81 ]. In this context, strategies should not be delegated top-down, but be developed in a collaborative manner, i.e. by engaging the workforce [ 75 ] and be in alignment with clear visions [ 91 , 94 ]. Roadmaps are used to improve planning, support implementation, facilitate the adoption of smart technologies in manufacturing [ 93 ] and should be integrated into both business and IT strategy [ 62 , 66 ]. In practice, clear adoption roadmaps that provide approaches on how to effectively integrate AI into existing strategies and businesses are often lacking [ 56 , 87 ]. The need for AI-related skills in organizations is a widely discussed topic in AI adoption analyses [ 79 ]. In this context, the literature points both at the need for specific skills in the development and design of AI applications [ 57 , 71 , 72 , 73 , 76 , 93 ] as well as the skills in using the technology [ 53 , 65 , 73 , 74 , 75 , 84 , 93 ] which availability in the firm is not always given [ 49 ]. AI requires new digital skills [ 36 , 50 , 52 , 55 , 56 , 59 , 61 , 63 , 66 , 78 , 80 ], where e.g. advanced analytics [ 64 , 75 , 81 ], programming skills [ 68 ] and cybersecurity skills [ 78 , 93 ] gain importance. The lack of skills required for AI is seen as a major challenge of digital transformation, as a skilled workforce is considered a key resource for companies [ 51 , 54 , 56 , 60 , 62 , 67 , 69 , 70 , 82 , 93 ]. This lack of a necessary skillset hinders the adoption of AI tools in production systems [ 58 , 77 ]. Closely related to skills is the need for new training concepts, which organizations need to consider when integrating digital technologies [ 49 , 50 , 51 , 56 , 59 , 63 , 71 , 74 , 75 ]. Firms must invest in qualification in order to create necessary competences [ 73 , 78 , 80 , 81 , 92 ]. Additionally, education must target and further develop the skills required for effectively integrating intelligent technologies into manufacturing processes [ 54 , 61 , 62 , 83 ]. Regarding this issue, academic institutions must develop fitting curricula for data driven manufacturing engineering [ 64 ]. Another driving factor of AI adoption is the innovation culture of an organization, which is influenced by various drivers. For example, companies that operate in an environment with high innovation rates, facing intense competitive pressures are considered more likely to see smart technologies as a tool for strategic change [ 83 , 91 , 93 ]. These firms often invest in more expensive and advanced smart technologies as the pressure and resulting competition forces them to innovate [ 93 ]. Another way of approach this is that innovation capability can also be supported and complemented by AI, for example by intelligent systems supporting humans in innovation or even innovating on their own [ 52 ].The entrepreneurial orientation of a firm is characterized in particular by innovativeness [ 66 ], productivity [ 63 ], risk-taking [ 86 ] as well as continuous improvement [ 50 ]. Such characteristics of an innovating culture are considered essential for companies to recognise dynamic changes in the market and make adoption decisions [ 51 , 71 , 81 , 84 , 86 , 94 ]. The prevalence of a digital mindset in companies is important for technology adoption, as digital transformation affects the entire organizational culture and behavior [ 59 , 80 , 92 ] and a lack of a digital culture [ 50 , 65 ] as well as a ‘passive mindset’ [ 78 ] can hinder the digital transformation of firms. Organizations need to develop a corresponding culture [ 66 , 67 , 71 ], also referred to as ‘AI-ready-culture’ [ 54 ], that promotes development and encourages people and data through the incorporation of technology [ 71 , 75 ]. With the increasing adoption of smart technologies, a ‘new digital normal’ is emerging, characterized by hybrid work models, more human–machine interactions and an increased use of digital technologies [ 75 , 83 ].

3.1.3 Technology and System

The ‘technology and system’ supercategory focuses on the broader issues related to the technology and infrastructure that support organizational operations and provide the technical foundation for AI deployment.

By IT infrastructure we refer to issues regarding the foundational systems and IT needed for AI adoption in production. Industrial firms and their IT systems must achieve a mature technological readiness in order to enable successful AI adoption [ 51 , 60 , 67 , 69 , 83 ]. A lack of appropriate IT infrastructure [ 68 , 71 , 78 , 91 ] or small maturity of Internet of Things (IoT) technologies [ 70 ]) hinders the efficient use of data in production firms [ 56 ] which is why firms must update their foundational information systems for successful AI adoption [ 53 , 54 , 62 , 66 , 72 , 75 ]. IT and data security are fundamental for AI adoption and must be provided [ 50 , 51 , 68 , 82 ]. This requires necessary developments that can ensure security during AI implementation while complying with legal requirements [ 52 , 72 , 78 ]. Generally, security concerns are common when implementing AI innovations [ 72 , 79 , 91 , 94 ]. This fear of a lack of security can also prevent the release of (e.g. customer) data in a production environment [ 56 ]. Additionally, as industrial production systems are vulnerable to failures as well as cyberattacks, companies need to address security and cybersecurity measures [ 49 , 76 , 88 , 89 ]. Developing user-friendly AI solutions can facilitate the adoption of smart solutions by increasing user understanding and making systems easy to use by employees as well as quick to integrate [ 50 , 72 , 84 ]. When developing user-friendly solutions which satisfy user needs [ 76 ], it is particularly important to understand and integrate the user perspective in the development process [ 90 ]. If employees find technical solutions easy to use, they are more confident in its use and perceived usefulness increases [ 53 , 67 , 68 ]. The compatibility of AI with a firm and its existing systems, i.e., the extent to which AI matches existing processes, structures, and infrastructures [ 53 , 54 , 56 , 60 , 78 , 80 , 82 , 83 , 93 , 94 ], is considered an important requirement for the adoption of AI in IT systems [ 91 ]. Along with compatibility also comes connectivity, which is intended to ensure the links within the overall network and avoid silo thinking [ 59 ]. Connectivity and interoperability of AI-based processes within the company’s IT manufacturing systems must be ensured at different system levels and are considered key factors in the development of AI applications for production [ 50 , 72 , 89 ]. The design of modular AI solutions can increase system compatibility [ 84 ]. Firms deciding for AI adoption must address safety issues [ 51 , 54 , 59 , 72 , 73 , 78 ]. This includes both safety in the use and operation of AI [ 60 , 69 ]. In order to address safety concerns of integrating AI solutions in industrial systems [ 49 ], systems must secure high reliability [ 71 ]. AI can also be integrated as a safety enabler, for example, by providing technologies to monitor health and safety in the workplace to prevent fatigue and injury [ 75 ].

3.1.4 Data management

Since AI adoption in the organization is strongly data-driven, the ‘data management’ supercategory is dedicated to the comprehensive aspects related to the effective and responsible management of data within the organization.

Data privacy must be guaranteed when creating AI applications based on industrial production data [ 49 , 58 , 59 , 60 , 72 , 76 , 78 , 79 , 82 , 88 , 89 , 91 , 94 ] as ‘[M]anufacturing industries generate large volumes of unstructured and sensitive data during their daily operations’ [ 89 ]. Closely related to this is the need for anonymization and confidentiality of data [ 61 , 69 , 70 , 78 ]. The availability of large, heterogeneous data sets is essential for the digital transformation of organizations [ 52 , 59 , 78 , 80 , 88 , 89 ] and is considered one of the key drivers of AI innovation [ 62 , 68 , 72 , 86 ]. In production systems, lack of data availability is often a barrier to AI adoption [ 58 , 70 , 77 ]. In order to enable AI to establish relationships between data, the availability of large input data that is critical [ 62 , 76 , 81 ]. New AI models are trained with this data and can adapt as well as improve as they receive new data [ 59 , 62 ]. Big data can thus significantly improve the quality of AI applications [ 59 , 71 ]. As more and more data is generated in manufacturing [ 85 ], AI opens up new opportunities for companies to make use of it [ 62 ]. However, operational data are often unstructured, as they come from different sources and exist in diverse formats [ 85 , 87 ]. This challenges data processing, as data quality and origin are key factors in the management of data [ 78 , 79 , 80 , 88 , 89 , 91 ]. To make production data valuable and usable for AI, consistency of data and thus data integrity is required across manufacturing systems [ 50 , 62 , 77 , 84 ]. Another key prerequisites for AI adoption is data governance [ 56 , 59 , 67 , 68 , 71 , 78 , 88 ] which is an important asset to make use of data in production [ 50 ] and ensure the complex management of heterogenous data sets [ 89 ]. The interoperability of data and thus the foundation for the compatibility of AI with existing systems, i.e., the extent to which AI matches existing processes, structures, and infrastructures [ 53 , 56 , 84 , 93 ], is considered another important requirement for the adoption of AI in IT systems. Data interoperability in production systems can be hindered by missing data standards as different machines use different formats [ 87 ]. Data processing refers to techniques used to preparing data for analysis which is essential to obtain consistent results from data analytics in production [ 58 , 72 , 80 , 81 , 84 ]. In this process, the numerous, heterogeneous data from different sensors are processed in such a way that they can be used for further analyses [ 87 ]. The capability of production firms to process data and information is thus important to enable AI adoption [ 77 , 86 , 93 ]. With the increasing data generation in the smart and connected factory, the strategic relevance of data analytics is gaining importance [ 55 , 69 , 78 ], as it is essential for AI systems in performing advanced data analyses [ 49 , 67 , 72 , 86 , 88 ]. Using analytics, valuable insights can be gained from the production data obtained using AI systems [ 58 , 77 , 87 ]. In order to enable the processing of big data, a profound data infrastructure is necessary [ 65 , 75 , 87 ]. Facilities must be equipped with sensors, that collect data and model information, which requires investments from firms [ 72 ]. In addition, production firms must build the necessary skills, culture and capabilities for data analytics [ 54 , 75 , 87 , 93 ]. Data storage, one of the foundations and prerequisites for smart manufacturing [ 54 , 68 , 71 , 74 ], must be ensured in order to manage the larg amounts of data and thus realize the adoption of intelligent technologies in production [ 50 , 59 , 72 , 78 , 84 , 87 , 88 , 89 ].

3.2 External environment

The external drivers of AI adoption in production influence the organization through conditions and events from outside the firm and are therefore difficult to control by the organization itself.

3.2.1 Regulatory environment

This supercategory captures the broader concept of establishing rules, standards, and frameworks that guide the behavior, actions, and operations of individuals, organizations, and societies when implementing AI.

AI adoption in production faces many ethical challenges [ 70 , 72 , 79 ]. AI applications must be compliant with the requirements of organizational ethical standards and laws [ 49 , 50 , 59 , 60 , 62 , 75 ] which is why certain issues must be examined in AI adoption and AI design [ 62 , 73 , 82 , 91 ] so that fairness and justice are guaranteed [ 78 , 79 , 92 ]. Social rights, cultural values and norms must not be violated in the process [ 49 , 52 , 53 , 81 ]. In this context, the explainability and transparency of AI decisions also plays an important role [ 50 , 54 , 58 , 70 , 78 , 89 ] and can address the characteristic of AI of a black box [ 90 ]. In addition, AI applications must be compliant with legal and regulatory requirements [ 51 , 52 , 59 , 77 , 81 , 82 , 91 ] and be developed accordingly [ 49 , 76 ] in order to make organization processes using AI clear and effective [ 65 ]. At present, policies and regulation of AI are still in its infancy [ 49 ] and missing federal regulatory guidelines, standards as well as incentives hinder the adoption of AI [ 67 ] which should be expanded simultaneously to the expansion of AI technology [ 60 ]. This also includes regulations on the handling of data (e.g. anonymization of data) [ 61 , 72 ].

3.2.2 Business environment

The factors in the ‘business environment’ supercategory refer to the external conditions and influences that affect the operations, decision making, and performance of the company seeking to implement AI in a production context.

Cooperation and collaboration can influence the success of digital technology adoption [ 52 , 53 , 59 , 72 ], which is why partnerships are important for adoption [ 53 , 59 ] and can positively influence its future success [ 52 , 67 ]. Both intraorganizational and interorganizational knowledge sharing can positively influence AI adoption [ 49 ]. In collaborations, companies can use a shared knowledge base where data and process sharing [ 51 , 59 , 94 ] as well as social support systems strengthen feedback loops between departments [ 79 , 80 ]. With regard to AI adoption in firms, vendors as well as service providers need to collaborate closely to improve the compatibility and operational capability of smart technologies across different industries [ 82 , 93 ]. Without external IT support, companies can rarely integrate AI into their production processes [ 66 ], which is why thorough support from vendors can significantly facilitate the integration of AI into existing manufacturing processes [ 80 , 91 ]. Public–private collaborations can also add value and governments can target AI dissemination [ 60 , 74 ]. The support of the government also positively influences AI adoption. This includes investing in research projects and policies, building a regulatory setting as well as creating a collaborative environment [ 60 ]. Production companies are constantly exposed to changing conditions, which is why the dynamics of the environment is another factor influencing the adoption of AI [ 52 , 63 , 72 , 86 ]. Environmental dynamics influence the operational performance of firms and can favor an entrepreneurial orientation of firms [ 86 ]. In order to respond to dynamics, companies need to develop certain capabilities and resources (i.e. dynamic capabilities) [ 86 ]. This requires the development of transparency, agility, as well as resilience to unpredictable changes, which was important in the case of the COVID-19 pandemic, for example, where companies had to adapt quickly to changing environments [ 75 ]. A firm’s environment (e.g. governments, partners or customers) can also pressure companies to adopt digital technologies [ 53 , 67 , 82 , 91 ]. Companies facing intense competition are considered more likely to invest in smart technologies, as rivalry pushes them to innovate and they hope to gain competitive advantages from adoption [ 36 , 66 , 82 , 93 ].

3.2.3 Economic environment

By considering both the industrial sector and country within the subcategory ‘economic environment’, production firms can analyze the interplay between the two and understand how drivers can influence the AI adoption process in their industrial sector’s performance within a particular country.

The industrial sector of a firm influences AI adoption in production from a structural perspective, as it indicates variations in product characteristics, governmental support, the general digitalization status, the production environment as well as the use of AI technologies within the sector [ 36 ]. Another factor that influences AI adoption is the country in which a company is located. This influences not only cultural aspects, the availability of know-how and technology orientation, but also regulations, laws, standards and subsidies [ 36 ]. From another perspective, AI can also contribute to the wider socio-economic growth of economies by making new opportunities easily available and thus equipping e.g. more rural areas with advanced capabilities [ 78 ].

3.3 Future research directions

The analysis of AI adoption in production requires a comprehensive analysis of the various factors that influence the introduction of the innovation. As discussed by Kinkel, Baumgartner, Cherubini [ 36 ], our research also concludes that organizational factors have a particularly important role to play. After evaluating the individual drivers of AI adoption in production in detail in this qualitative synthesis, we draw a conclusion from the results and derive a research agenda from the analysis to serve as a basis for future research. The RQs emerged from the analyzed factors and are presented in Table  2 . We developed the questions based on the literature review and identified research gaps for every factor that was most frequently mentioned. From the factors analyzed and RQs developed, the internal environment has a strong influence on AI adoption in production, and organizational factors play a major role here.

Looking at the supercategory ‘business and environment’, performance indicators and investments are considered drivers of AI adoption in production. Indicators to measure the performance of AI innovations are necessary here so that managers can perform cost–benefit analyses and make the right decision for their company. There is a need for research here to support possible calculations and show managers a comprehensive view of the costs and benefits of technology in production. In terms of budget, it should be noted that AI adoption involves a considerable financial outlay that must be carefully weighed and some capital must be available to carry out the necessary implementation efforts (e.g., staffing costs, machine retrofits, change management, and external IT service costs). Since AI adoption is a complex process and turnkey solutions can seldom be implemented easily and quickly, but require many changes (not only technologically but also on an organizational level), it is currently difficult to estimate the necessary budgets and thus make them available. Especially the factors of the supercategory ‘organizational effectiveness’ drive AI adoption in production. Trust of the workforce is considered an important driver, which must be created in order to successfully implement AI. This requires measures that can support management in building trust. Closely related to this are the necessary change management processes that must be initiated to accompany the changes in a targeted manner. Management itself must also play a clear role in the introduction of AI and communicate its support, as this also influences the adoption. The development of clear processes and measures can help here. Developing roadmaps for AI adoption can facilitate the adoption process and promote strategic integration with existing IT and business strategy. Here, best practice roadmaps and necessary action steps can be helpful for companies. Skills are considered the most important driver for AI adoption in manufacturing. Here, there is a lack of clear approaches that support companies in identifying the range of necessary skills and, associated with this, also opportunities to further develop these skills in the existing workforce. Also, building a culture of innovation requires closer research that can help companies foster a conducive environment for AI adoption and the integration of other smart technologies. Steps for developing a positive mindset require further research that can provide approaches for necessary action steps and measures in creating a positive digital culture. With regard to ‘technology and system’, the factors of IT infrastructure and security in particular are driving AI adoption in production. Existing IT systems must reach a certain maturity to enable AI adoption on a technical level. This calls for clear requirements that visualize for companies which systems and standards are in place and where developments are needed. Security must be continuously ensured, for which certain standards and action catalogs must be developed. With regard to the supercategory ‘data management’, the availability of data is considered the basis for successful AI adoption, as no AI can be successfully deployed without data. In the production context in particular, this requires developments that support companies in the provision of data, which usually arises from very heterogeneous sources and forms. Data analytics must also be closely examined, and production companies usually need external support in doing so. The multitude of data also requires big data storage capabilities. Here, groundwork is needed to show companies options about the possibilities of different storage options (e.g., on premis vs. cloud-based).

In the ‘regulatory environment’, ethics in particular is considered a driver of AI adoption in production. Here, fundamental ethical factors and frameworks need to be developed that companies can use as a guideline to ensure ethical standards throughout the process. Cooperations and environmental dynamism drive the supercategory ‘business environment’. Collaborations are necessary to successfully implement AI adoption and action is needed to create the necessary contact facilitation bodies. In a competitive environment, companies have to make quick decisions under strong pressure, which also affects AI adoption. Here, guidelines and also best practice approaches can help to simplify decisions and quickly demonstrate the advantage of the solutions. There is a need for research in this context.

4 Conclusions

The use of AI technologies in production continues to gain momentum as managers hope to increase efficiency, productivity and reduce costs [ 9 , 13 , 20 ]. Although the benefits of AI adoption speak for themselves, implementing AI is a complex decision that requires a lot of knowledge, capital and change [ 95 ] and is influenced by various internal and external factors. Therefore, managers are still cautious about implementing the technology in a production context. Our SLR seeks to examine the emergent phenomenon of AI in production with the precise aim of understanding the factors influencing AI adoption and the key topics discussed in the literature when analyzing AI in a production context. For this purpose, we use the current state of research and examine the existing studies based on the methodology of a systematic literature analysis and respond to three RQs.

We answer RQ1 by closely analyzing the literature selected in our SLR to identify trends in current research on AI adoption in production. In this process, it becomes clear that the topic is gaining importance and that research has increased over the last few years. In the field of production, AI is being examined from various angles and current research addresses aspects from a business, human and technical perspective. In our response to RQ2 we synthesized the existing literature to derive 35 factors that influence AI adoption in production at different levels from inside or outside the organization. In doing so, we find that AI adoption in production poses particularly significant challenges to organizational effectiveness compared to other digital technologies and that the relevance of data management takes on a new dimension. Production companies often operate more traditionally and are sometimes rigid when it comes to change [ 96 , 97 ], which can pose organizational challenges when adopting AI. In addition, the existing machines and systems are typically rather heterogeneous and are subject to different digitalization standards, which in turn can hinder the availability of the necessary data for AI implementation [ 98 , 99 ]. We address RQ3 by deriving a research agenda, which lays a foundation for further scientific research and deepening the understanding of AI adoption in production. The results of our analysis can further help managers to better understand AI adoption and to pay attention to the different factors that influence the adoption of this complex technology.

4.1 Contributions

Our paper takes the first step towards analysing the current state of the research on AI adoption from a production perspective. We represent a holistic view on the topic, which is necessary to get a better understanding of AI in a production-context and build a comprehensive view on the different dimensions as well as factors influencing its adoption. To the best of our knowledge, this is the first contribution that systematises research about the adoption of AI in production. As such, it makes an important contribution to current AI and production research, which is threefold:

First, we highlight the characteristics of studies conducted in recent years on the topic of AI adoption in production, from which several features and developments can be deduced. Our results confirm the topicality of the issue and the increasing relevance of research in the field.

Having laid the foundations for understanding AI in production, we focused our research on the identification and systematization of the most relevant factors influencing AI adoption in production at different levels. This brings us to the second contribution, our comprehensive factor analysis of AI adoption in production provides a framework for further research as well as a potential basis for managers to draw upon when adopting AI. By systematizing the relevant factors influencing AI adoption in production, we derived a set of 35 researched factors associated with AI adoption in production. These factors can be clustered in two areas of analysis and seven respective supercategories. The internal environment area includes four levels of analysis: ‘business and structure’ (focusing on financial aspects and firm characteristics), ‘organizational effectiveness’ (focusing on human-centred factors), ‘technology and system’ (based on the IT infrastructure and systems) as well as ‘data management’ (including all data related factors). Three categories are assigned to the external environment: the ‘regulatory environment’ (such as ethics and the regulatory forms), the ‘business environment’ (focused on cooperation activities and dynamics in the firm environment) and the ‘economic environment’ (related to sectoral and country specifics).

Third, the developed research plan as outlined in Table  2 serves as an additional outcome of the SLR, identifying key RQs in the analyzed areas that can serve as a foundation for researchers to expand the research area of AI adoption in production. These RQs are related to the mostly cited factors analyzed in our SLR and aim to broaden the understanding on the emerging topic.

The resulting insights can serve as the basis for strategic decisions by production companies looking to integrate AI into their processes. Our findings on the factors influencing AI adoption as well as the developed research agenda enhance the practical understanding of a production-specific adoption. Hence, they can serve as the basis for strategic decisions for companies on the path to an effective AI adoption. Managers can, for example, analyse the individual factors in light of their company as well as take necessary steps to develop further aspects in a targeted manner. Researchers, on the other hand, can use the future research agenda in order to assess open RQs and can expand the state of research on AI adoption in production.

4.2 Limitations

Since a literature review must be restricted in its scope in order to make the analyses feasible, our study provides a starting point for further research. Hence, there is a need for further qualitative and quantitative empirical research on the heterogeneous nature of how firms configure their AI adoption process. Along these lines, the following aspects would be of particular interest for future research to improve and further validate the analytical power of the proposed framework.

First, the lack of research on AI adoption in production leads to a limited number of papers included in this SLR. As visualized in Fig.  2 , the number of publications related to the adoption of AI in production has been increasing since 2018 but is, to date, still at an early stage. For this reason, only 47 papers published until May 2024 addressing the production-specific adoption of AI were identified and therefore included in our analysis for in-depth investigation. This rather small number of papers included in the full-text analysis gives a limited view on AI adoption in production but allows a more detailed analysis. As the number of publications in this research field increases, there seems to be a lot of research happening in this field which is why new findings might be constantly added and developed as relevant in the future [ 39 ]. Moreover, in order to research AI adoption from a more practical perspective and thus to build up a broader, continuously updated view on AI adoption in production, future literature analyses could include other publication formats, e.g. study reports of research institutions and companies, as well discussion papers.

Second, the scope of the application areas of AI in production has been increasing rapidly. Even though our overview of the three main areas covered in the recent literature serves as a good basis for identifying the most dominant fields for AI adoption in production, a more detailed analysis could provide a better overview of possibilities for manufacturing companies. Hence, a further systematisation as well as evaluation of application areas for AI in production can provide managers with the information needed to decide where AI applications might be of interest for the specific company needs.

Third, the systematisation of the 35 factors influencing AI adoption in production serve as a good ground for identifying relevant areas influenced by and in turn influencing the adoption of AI. Further analyses should be conducted in order to extend this view and extend the framework. For example, our review could be combined with explorative research methods (such as case studies in production firms) in order to add the practical insights from firms adopting AI. This integration of practical experiences can also help exploit and monitor more AI-specific factors by observing AI adoption processes. In enriching the factors through in-depth analyses, the results of the identified AI adoption factors could also be examined in light of theoretical contributions like the technology-organization-environment (TOE) framework [ 47 ] and other adoption theories.

Fourth, in order to examine the special relevance of identified factors for AI adoption process and thus to distinguish it from the common factors influencing the adoption of more general digital technologies, there is a further need for more in-depth (ethnographic) research into their impacts on the adoption processes, particularly in the production context. Similarly, further research could use the framework introduced in this paper as a basis to develop new indicators and measurement concepts as well as to examine their impacts on production performance using quantitative methods.

Benner MJ, Waldfogel J (2020) Changing the channel: digitization and the rise of “middle tail” strategies. Strat Mgmt J 86:1–24. https://doi.org/10.1002/smj.3130

Article   Google Scholar  

Roblek V, Meško M, Krapež A (2016) A complex view of industry 4.0. SAGE Open. https://doi.org/10.1177/2158244016653987

Oliveira BG, Liboni LB, Cezarino LO et al (2020) Industry 4.0 in systems thinking: from a narrow to a broad spectrum. Syst Res Behav Sci 37:593–606. https://doi.org/10.1002/sres.2703

Li B, Hou B, Yu W et al (2017) Applications of artificial intelligence in intelligent manufacturing: a review. Frontiers Inf Technol Electronic Eng 18:86–96. https://doi.org/10.1631/FITEE.1601885

Dhamija P, Bag S (2020) Role of artificial intelligence in operations environment: a review and bibliometric analysis. TQM 32:869–896. https://doi.org/10.1108/TQM-10-2019-0243

Collins C, Dennehy D, Conboy K et al (2021) Artificial intelligence in information systems research: a systematic literature review and research agenda. Int J Inf Manage 60:102383. https://doi.org/10.1016/j.ijinfomgt.2021.102383

Chien C-F, Dauzère-Pérès S, Huh WT et al (2020) Artificial intelligence in manufacturing and logistics systems: algorithms, applications, and case studies. Int J Prod Res 58:2730–2731. https://doi.org/10.1080/00207543.2020.1752488

Chen H (2019) Success factors impacting artificial intelligence adoption: perspective from the telecom industry in China, Old Dominion University

Sanchez M, Exposito E, Aguilar J (2020) Autonomic computing in manufacturing process coordination in industry 4.0 context. J Industrial Inf Integr. https://doi.org/10.1016/j.jii.2020.100159

Lee J, Davari H, Singh J et al (2018) Industrial artificial intelligence for industry 4.0-based manufacturing systems. Manufacturing Letters 18:20–23. https://doi.org/10.1016/j.mfglet.2018.09.002

Heimberger H, Horvat D, Schultmann F (2023) Assessing AI-readiness in production—A conceptual approach. In: Huang C-Y, Dekkers R, Chiu SF et al. (eds) intelligent and transformative production in pandemic times. Springer, Cham, pp 249–257

Horvat D, Heimberger H (2023) AI Readiness: An Integrated Socio-technical Framework. In: Deschamps F, Pinheiro de Lima E, Da Gouvêa Costa SE et al. (eds) Proceedings of the 11 th international conference on production research—Americas: ICPR Americas 2022, 1 st ed. 2023. Springer Nature Switzerland; Imprint Springer, Cham, pp 548–557

Wang J, Ma Y, Zhang L et al (2018) Deep learning for smart manufacturing: methods and applications. J Manuf Syst 48:144–156. https://doi.org/10.1016/J.JMSY.2018.01.003

Davenport T, Guha A, Grewal D et al (2020) How artificial intelligence will change the future of marketing. J Acad Mark Sci 48:24–42. https://doi.org/10.1007/s11747-019-00696-0

Cui R, Li M, Zhang S (2022) AI and procurement. Manufacturing Serv Operations Manag 24(691):706. https://doi.org/10.1287/msom.2021.0989

Pournader M, Ghaderi H, Hassanzadegan A et al (2021) Artificial intelligence applications in supply chain management. Int J Prod Econ 241:108250. https://doi.org/10.1016/j.ijpe.2021.108250

Su H, Li L, Tian S et al (2024) Innovation mechanism of AI empowering manufacturing enterprises: case study of an industrial internet platform. Inf Technol Manag. https://doi.org/10.1007/s10799-024-00423-4

Venkatesh V, Raman R, Cruz-Jesus F (2024) AI and emerging technology adoption: a research agenda for operations management. Int J Prod Res 62:5367–5377. https://doi.org/10.1080/00207543.2023.2192309

Senoner J, Netland T, Feuerriegel S (2022) Using explainable artificial intelligence to improve process quality: evidence from semiconductor manufacturing. Manage Sci 68:5704–5723. https://doi.org/10.1287/mnsc.2021.4190

Fosso Wamba S, Queiroz MM, Ngai EWT et al (2024) The interplay between artificial intelligence, production systems, and operations management resilience. Int J Prod Res 62:5361–5366. https://doi.org/10.1080/00207543.2024.2321826

Uren V, Edwards JS (2023) Technology readiness and the organizational journey towards AI adoption: an empirical study. Int J Inf Manage 68:102588. https://doi.org/10.1016/j.ijinfomgt.2022.102588

Berente N, Gu B, Recker J (2021) Managing artificial intelligence special issue managing AI. MIS Quarterly 45:1433–1450

Google Scholar  

Scafà M, Papetti A, Brunzini A et al (2019) How to improve worker’s well-being and company performance: a method to identify effective corrective actions. Procedia CIRP 81:162–167. https://doi.org/10.1016/j.procir.2019.03.029

Wang H, Qiu F (2023) AI adoption and labor cost stickiness: based on natural language and machine learning. Inf Technol Manag. https://doi.org/10.1007/s10799-023-00408-9

Lindebaum D, Vesa M, den Hond F (2020) Insights from “the machine stops ” to better understand rational assumptions in algorithmic decision making and its implications for organizations. Acad Manag Rev 45:247–263. https://doi.org/10.5465/amr.2018.0181

Baskerville RL, Myers MD, Yoo Y (2020) Digital first: the ontological reversal and new challenges for information systems research. MIS Quarterly 44:509–523

Frey CB, Osborne MA (2017) The future of employment: How susceptible are jobs to computerisation? Technol Forecast Soc Chang 114:254–280. https://doi.org/10.1016/J.TECHFORE.2016.08.019

Jarrahi MH (2018) Artificial intelligence and the future of work: human-AI symbiosis in organizational decision making. Bus Horiz 61:577–586. https://doi.org/10.1016/j.bushor.2018.03.007

Fügener A, Grahl J, Gupta A et al (2021) Will humans-in-the-loop become borgs? Merits and pitfalls of working with AI. MIS Quarterly 45:1527–1556

Klumpp M (2018) Automation and artificial intelligence in business logistics systems: human reactions and collaboration requirements. Int J Log Res Appl 21:224–242. https://doi.org/10.1080/13675567.2017.1384451

Schrettenbrunnner MB (2020) Artificial-Intelligence-driven management. IEEE Eng Manag Rev 48:15–19. https://doi.org/10.1109/EMR.2020.2990933

Li J, Li M, Wang X et al (2021) Strategic directions for AI: the role of CIOs and boards of directors. MIS Quarterly 45:1603–1644

Brock JK-U, von Wangenheim F (2019) Demystifying AI: What digital transformation leaders can teach you about realistic artificial intelligence. Calif Manage Rev 61:110–134. https://doi.org/10.1177/1536504219865226

Lee J, Suh T, Roy D et al (2019) Emerging technology and business model innovation: the case of artificial intelligence. JOItmC 5:44. https://doi.org/10.3390/joitmc5030044

Chen J, Tajdini S (2024) A moderated model of artificial intelligence adoption in firms and its effects on their performance. Inf Technol Manag. https://doi.org/10.1007/s10799-024-00422-5

Kinkel S, Baumgartner M, Cherubini E (2022) Prerequisites for the adoption of AI technologies in manufacturing—evidence from a worldwide sample of manufacturing companies. Technovation 110:102375. https://doi.org/10.1016/j.technovation.2021.102375

Mikalef P, Gupta M (2021) Artificial intelligence capability: Conceptualization, measurement calibration, and empirical study on its impact on organizational creativity and firm performance. Inf Manag 58:103434. https://doi.org/10.1016/j.im.2021.103434

McElheran K, Li JF, Brynjolfsson E et al (2024) AI adoption in America: Who, what, and where. Economics Manag Strategy 33:375–415. https://doi.org/10.1111/jems.12576

Tranfield D, Denyer D, Smart P (2003) Towards a methodology for developing evidence-informed management knowledge by means of systematic review. Br J Manag 14:207–222. https://doi.org/10.1111/1467-8551.00375

Cooper H, Hedges LV, Valentine JC (2009) Handbook of research synthesis and meta-analysis. Russell Sage Foundation, New York

Page MJ, McKenzie JE, Bossuyt PM et al (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372:n71. https://doi.org/10.1136/bmj.n71

Denyer D, Tranfield D (2011) Producing a systematic review. In: Buchanan DA, Bryman A (eds) The Sage handbook of organizational research methods. Sage Publications Inc, Thousand Oaks, CA, pp 671–689

Burbidge JL, Falster P, Riis JO et al (1987) Integration in manufacturing. Comput Ind 9:297–305. https://doi.org/10.1016/0166-3615(87)90103-5

Mayring P (2000) Qualitative content analysis. Forum qualitative Sozialforschung/Forum: Qualitative social research, Vol 1, No 2 (2000): Qualitative methods in various disciplines I: Psychology. https://doi.org/10.17169/fqs-1.2.1089

Hsieh H-F, Shannon SE (2005) Three approaches to qualitative content analysis. Qual Health Res 15:1277–1288. https://doi.org/10.1177/1049732305276687

Miles MB, Huberman AM (2009) Qualitative data analysis: An expanded sourcebook, 2nd edn. Sage, Thousand Oaks, Calif

Tornatzky LG, Fleischer M (1990) The processes of technological innovation. Issues in organization and management series. Lexington Books, Lexington, Mass.

Alsheibani S, Cheung Y, Messom C (2018) Artificial Intelligence Adoption: AI-readiness at Firm-Level: Research-in-Progress. Twenty-Second Pacific Asia Conference on Information Systems

Akinsolu MO (2023) Applied artificial intelligence in manufacturing and industrial production systems: PEST considerations for engineering managers. IEEE Eng Manag Rev 51:52–62. https://doi.org/10.1109/EMR.2022.3209891

Bettoni A, Matteri D, Montini E et al (2021) An AI adoption model for SMEs: a conceptual framework. IFAC-PapersOnLine 54:702–708. https://doi.org/10.1016/j.ifacol.2021.08.082

Boavida N, Candeias M (2021) Recent automation trends in portugal: implications on industrial productivity and employment in automotive sector. Societies 11:101. https://doi.org/10.3390/soc11030101

Botha AP (2019) A mind model for intelligent machine innovation using future thinking principles. Jnl of Manu Tech Mnagmnt 30:1250–1264. https://doi.org/10.1108/JMTM-01-2018-0021

Chatterjee S, Rana NP, Dwivedi YK et al (2021) Understanding AI adoption in manufacturing and production firms using an integrated TAM-TOE model. Technol Forecast Soc Chang 170:120880. https://doi.org/10.1016/j.techfore.2021.120880

Chiang LH, Braun B, Wang Z et al (2022) Towards artificial intelligence at scale in the chemical industry. AIChE J. https://doi.org/10.1002/aic.17644

Chouchene A, Carvalho A, Lima TM et al. (2020) Artificial intelligence for product quality inspection toward smart industries: quality control of vehicle Non-conformities. In: Garengo P (ed) 2020 9th International Conference on Industrial Technology and Management: ICITM 2020 February 11–13, 2020, Oxford, United Kingdom. IEEE, pp 127–131

Corti D, Masiero S, Gladysz B (2021) Impact of Industry 4.0 on Quality Management: identification of main challenges towards a Quality 4.0 approach. In: 2021 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC). IEEE, pp 1–8

Demlehner Q, Schoemer D, Laumer S (2021) How can artificial intelligence enhance car manufacturing? A Delphi study-based identification and assessment of general use cases. Int J Inf Manage 58:102317. https://doi.org/10.1016/j.ijinfomgt.2021.102317

Dohale V, Akarte M, Gunasekaran A et al (2022) (2022) Exploring the role of artificial intelligence in building production resilience: learnings from the COVID-19 pandemic. Int J Prod Res 10(1080/00207543):2127961

Drobot AT (2020) Industrial Transformation and the Digital Revolution: A Focus on artificial intelligence, data science and data engineering. In: 2020 ITU Kaleidoscope: Industry-Driven Digital Transformation (ITU K). IEEE, pp 1–11

Ghani EK, Ariffin N, Sukmadilaga C (2022) Factors influencing artificial intelligence adoption in publicly listed manufacturing companies: a technology, organisation, and environment approach. IJAEFA 14:108–117

Hammer A, Karmakar S (2021) Automation, AI and the future of work in India. ER 43:1327–1341. https://doi.org/10.1108/ER-12-2019-0452

Hartley JL, Sawaya WJ (2019) Tortoise, not the hare: digital transformation of supply chain business processes. Bus Horiz 62:707–715. https://doi.org/10.1016/j.bushor.2019.07.006

Kyvik Nordås H, Klügl F (2021) Drivers of automation and consequences for jobs in engineering services: an agent-based modelling approach. Front Robot AI 8:637125. https://doi.org/10.3389/frobt.2021.637125

Mubarok K, Arriaga EF (2020) Building a smart and intelligent factory of the future with industry 4.0 technologies. J Phys Conf Ser. https://doi.org/10.1088/1742-6596/1569/3/032031

Muriel-Pera YdJ, Diaz-Piraquive FN, Rodriguez-Bernal LP et al. (2018) Adoption of strategies the fourth industrial revolution by micro, small and medium enterprises in bogota D.C. In: Lozano Garzón CA (ed) 2018 Congreso Internacional de Innovación y Tendencias en Ingeniería (CONIITI). IEEE, pp 1–6

Olsowski S, Schlögl S, Richter E et al. (2022) Investigating the Potential of AutoML as an Instrument for Fostering AI Adoption in SMEs. In: Uden L, Ting I-H, Feldmann B (eds) Knowledge Management in Organisations: 16th International Conference, KMO 2022, Hagen, Germany, July 11–14, 2022, Proceedings, 1st ed. 2022, vol 1593. Springer, Cham, pp 360–371

Rodríguez-Espíndola O, Chowdhury S, Dey PK et al (2022) Analysis of the adoption of emergent technologies for risk management in the era of digital manufacturing. Technol Forecast Soc Chang 178:121562. https://doi.org/10.1016/j.techfore.2022.121562

Schkarin T, Dobhan A (2022) Prerequisites for Applying Artificial Intelligence for Scheduling in Small- and Medium-sized Enterprises. In: Proceedings of the 24 th International Conference on Enterprise Information Systems. SCITEPRESS—Science and Technology Publications, pp 529–536

Sharma P, Shah J, Patel R (2022) Artificial intelligence framework for MSME sectors with focus on design and manufacturing industries. Mater Today: Proc 62:6962–6966. https://doi.org/10.1016/j.matpr.2021.12.360

Siaterlis G, Nikolakis N, Alexopoulos K et al. (2022) Adoption of AI in EU Manufacturing. Gaps and Challenges. In: Katalinic B (ed) Proceedings of the 33 rd International DAAAM Symposium 2022, vol 1. DAAAM International Vienna, pp 547–550

Tariq MU, Poulin M, Abonamah AA (2021) Achieving operational excellence through artificial intelligence: driving forces and barriers. Front Psychol 12:686624. https://doi.org/10.3389/fpsyg.2021.686624

Trakadas P, Simoens P, Gkonis P et al (2020) An artificial intelligence-based collaboration approach in industrial IoT manufacturing: key concepts. Architectural Ext Potential Applications Sens. https://doi.org/10.3390/s20195480

Vernim S, Bauer H, Rauch E et al (2022) A value sensitive design approach for designing AI-based worker assistance systems in manufacturing. Procedia Computer Sci 200:505–516. https://doi.org/10.1016/j.procs.2022.01.248

Williams G, Meisel NA, Simpson TW et al (2022) Design for artificial intelligence: proposing a conceptual framework grounded in data wrangling. J Computing Inf Sci Eng 10(1115/1):4055854

Wuest T, Romero D, Cavuoto LA et al (2020) Empowering the workforce in Post–COVID-19 smart manufacturing systems. Smart Sustain Manuf Syst 4:20200043. https://doi.org/10.1520/SSMS20200043

Javaid M, Haleem A, Singh RP (2023) A study on ChatGPT for Industry 4.0: background, potentials, challenges, and eventualities. J Economy Technol 1:127–143. https://doi.org/10.1016/j.ject.2023.08.001

Rathore AS, Nikita S, Thakur G et al (2023) Artificial intelligence and machine learning applications in biopharmaceutical manufacturing. Trends Biotechnol 41:497–510. https://doi.org/10.1016/j.tibtech.2022.08.007

Jan Z, Ahamed F, Mayer W et al (2023) Artificial intelligence for industry 4.0: systematic review of applications, challenges, and opportunities. Expert Syst Applications 216:119456

Waschull S, Emmanouilidis C (2023) Assessing human-centricity in AI enabled manufacturing systems: a socio-technical evaluation methodology. IFAC-PapersOnLine 56:1791–1796. https://doi.org/10.1016/j.ifacol.2023.10.1891

Stohr A, Ollig P, Keller R et al (2024) Generative mechanisms of AI implementation: a critical realist perspective on predictive maintenance. Inf Organ 34:100503. https://doi.org/10.1016/j.infoandorg.2024.100503

Pazhayattil AB, Konyu-Fogel G (2023) ML and AI Implementation Insights for Bio/Pharma Manufacturing. BioPharm International 36:24–29

Ronaghi MH (2023) The influence of artificial intelligence adoption on circular economy practices in manufacturing industries. Environ Dev Sustain 25:14355–14380. https://doi.org/10.1007/s10668-022-02670-3

Rath SP, Tripathy R, Jain NK (2024) Assessing the factors influencing the adoption of generative artificial intelligence (GenAI) in the manufacturing sector. In: Sharma SK, Dwivedi YK, Metri B et al (eds) Transfer, diffusion and adoption of next-generation digital technologies, vol 697. Springer Nature Switzerland, Cham

Bonnard R, Da Arantes MS, Lorbieski R et al (2021) Big data/analytics platform for Industry 4.0 implementation in advanced manufacturing context. Int J Adv Manuf Technol 117:1959–1973. https://doi.org/10.1007/s00170-021-07834-5

Confalonieri M, Barni A, Valente A et al. (2015) An AI based decision support system for preventive maintenance and production optimization in energy intensive manufacturing plants. In: 2015 IEEE international conference on engineering, technology and innovation/ international technology management conference (ICE/ITMC). IEEE, pp 1–8

Dubey R, Gunasekaran A, Childe SJ et al (2020) Big data analytics and artificial intelligence pathway to operational performance under the effects of entrepreneurial orientation and environmental dynamism: a study of manufacturing organisations. Int J Prod Econ 226:107599. https://doi.org/10.1016/j.ijpe.2019.107599

Lee J, Singh J, Azamfar M et al (2020) Industrial AI: a systematic framework for AI in industrial applications. China Mechanical Eng 31:37–48

Turner CJ, Emmanouilidis C, Tomiyama T et al (2019) Intelligent decision support for maintenance: an overview and future trends. Int J Comput Integr Manuf 32:936–959. https://doi.org/10.1080/0951192X.2019.1667033

Agostinho C, Dikopoulou Z, Lavasa E et al (2023) Explainability as the key ingredient for AI adoption in Industry 5.0 settings. Front Artif Intell. https://doi.org/10.3389/frai.2023.1264372

Csiszar A, Hein P, Wachter M et al. (2020) Towards a user-centered development process of machine learning applications for manufacturing domain experts. In: 2020 third international conference on artificial intelligence for industries (AI4I). IEEE, pp 36–39

Merhi MI (2023) Harfouche A (2023) Enablers of artificial intelligence adoption and implementation in production systems. Int J Prod Res. https://doi.org/10.1080/00207543.2023.2167014

Demlehner Q, Laumer S (2024) How the terminator might affect the car manufacturing industry: examining the role of pre-announcement bias for AI-based IS adoptions. Inf Manag 61:103881. https://doi.org/10.1016/j.im.2023.103881

Ghobakhloo M, Ching NT (2019) Adoption of digital technologies of smart manufacturing in SMEs. J Ind Inf Integr 16:100107. https://doi.org/10.1016/j.jii.2019.100107

Binsaeed RH, Yousaf Z, Grigorescu A et al (2023) Knowledge sharing key issue for digital technology and artificial intelligence adoption. Systems 11:316. https://doi.org/10.3390/systems11070316

Papadopoulos T, Sivarajah U, Spanaki K et al (2022) Editorial: artificial Intelligence (AI) and data sharing in manufacturing, production and operations management research. Int J Prod Res 60:4361–4364. https://doi.org/10.1080/00207543.2021.2010979

Chirumalla K (2021) Building digitally-enabled process innovation in the process industries: a dynamic capabilities approach. Technovation 105:102256. https://doi.org/10.1016/j.technovation.2021.102256

Fragapane G, Ivanov D, Peron M et al (2022) Increasing flexibility and productivity in Industry 4.0 production networks with autonomous mobile robots and smart intralogistics. Ann Oper Res 308:125–143. https://doi.org/10.1007/s10479-020-03526-7

Shahbazi Z, Byun Y-C (2021) Integration of Blockchain, IoT and machine learning for multistage quality control and enhancing security in smart manufacturing. Sensors (Basel). https://doi.org/10.3390/s21041467

Javaid M, Haleem A, Singh RP et al (2021) Significance of sensors for industry 4.0: roles, capabilities, and applications. Sensors Int 2:100110. https://doi.org/10.1016/j.sintl.2021.100110

Download references

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

Business Unit Industrial Change and New Business Models, Competence Center Innovation and Knowledge Economy, Fraunhofer Institute for Systems and Innovation Research ISI, Breslauer Straße 48, 76139, Karlsruhe, Germany

Heidi Heimberger, Djerdj Horvat & Frank Schultmann

Karlsruhe Institute for Technology KIT, Institute for Industrial Production (IIP) - Chair of Business Administration, Production and Operations Management, Hertzstraße 16, 76187, Karlsruhe, Germany

Heidi Heimberger

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Heidi Heimberger .

Ethics declarations

Conflict of interest.

The authors report no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Heimberger, H., Horvat, D. & Schultmann, F. Exploring the factors driving AI adoption in production: a systematic literature review and future research agenda. Inf Technol Manag (2024). https://doi.org/10.1007/s10799-024-00436-z

Download citation

Accepted : 10 August 2024

Published : 23 August 2024

DOI : https://doi.org/10.1007/s10799-024-00436-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Artificial intelligence
• Technology adoption
• AI adoption
• Adoption factors
• Systematic literature review
• Find a journal
• Publish with us
• Track your research

Protocol for Systematic Review and Meta-Analysis of Prehospital Large Vessel Occlusion Screening Scales

• Find this author on Google Scholar
• Find this author on PubMed
• Search for this author on this site
• ORCID record for Hidetaka Suzuki
• For correspondence: [email protected]
• ORCID record for Yohei Okada
• ORCID record for Yamagami Hiroshi
• ORCID record for Hitoshi Kobata
• Info/History
• Preview PDF

Background Large Vessel Occlusion (LVO) is a serious condition that causes approximately 24-46% of acute ischemic strokes (AIS). LVO strokes tend to have higher mortality rates and result in more severe longterm disabilities compared to non LVO ischemic strokes. Early intervention with endovascular therapy (EVT) is recommended; however, EVT is limited to tertiary care hospitals with specialized facilities. Therefore, identifying patients with a high probability of LVO in prehospital settings and ensuring their rapid transfer to appropriate hospitals is crucial. While LVO diagnosis typically requires advanced imaging like MRI or CT scans, various scoring systems based on neurological symptoms have been developed for prehospital use. Although previous systematic reviews have addressed some of these scales, recent studies have introduced new scales and additional data on their accuracy. This systematic review and meta-analysis aim to summarize the current evidence on the diagnostic accuracy of these prehospital LVO screening scales.

Methods This systematic review and meta-analysis will be conducted in accordance with the PRISMA-DTA Statement and the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. We will include observational studies and randomized controlled trials that assess the utility of LVO scales in suspected stroke patients in prehospital settings. Eligible studies must provide sufficient data to calculate sensitivity and specificity, and those lacking such data or being case reports will be excluded. The literature search will cover CENTRAL, MEDLINE, and Ichushi databases, including studies in English and Japanese. Bias will be assessed using QUADAS-2, and meta-analysis will be conducted using a random effects model, with subgroup and sensitivity analyses to explore heterogeneity.

Competing Interest Statement

The authors have declared no competing interest.

Funding Statement

This study did not receive any funding

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

We will search the following databases CENTRAL, MEDLINE, and Ichushi.

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Data Availability

All data produced in the present study are available upon reasonable request to the authors.

View the discussion thread.

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Citation Manager Formats

• EndNote (tagged)
• EndNote 8 (xml)
• RefWorks Tagged
• Ref Manager
• Tweet Widget
• Facebook Like
• Google Plus One

Subject Area

• Addiction Medicine (342)
• Allergy and Immunology (665)
• Anesthesia (180)
• Cardiovascular Medicine (2625)
• Dentistry and Oral Medicine (314)
• Dermatology (222)
• Emergency Medicine (397)
• Endocrinology (including Diabetes Mellitus and Metabolic Disease) (930)
• Epidemiology (12175)
• Forensic Medicine (10)
• Gastroenterology (756)
• Genetic and Genomic Medicine (4064)
• Geriatric Medicine (385)
• Health Economics (676)
• Health Informatics (2625)
• Health Policy (997)
• Health Systems and Quality Improvement (979)
• Hematology (360)
• HIV/AIDS (845)
• Infectious Diseases (except HIV/AIDS) (13659)
• Intensive Care and Critical Care Medicine (790)
• Medical Education (398)
• Medical Ethics (109)
• Nephrology (430)
• Neurology (3832)
• Nursing (209)
• Nutrition (570)
• Obstetrics and Gynecology (734)
• Occupational and Environmental Health (690)
• Oncology (2008)
• Ophthalmology (581)
• Orthopedics (238)
• Otolaryngology (304)
• Pain Medicine (250)
• Palliative Medicine (73)
• Pathology (471)
• Pediatrics (1107)
• Pharmacology and Therapeutics (459)
• Primary Care Research (447)
• Psychiatry and Clinical Psychology (3400)
• Public and Global Health (6499)
• Radiology and Imaging (1390)
• Rehabilitation Medicine and Physical Therapy (806)
• Respiratory Medicine (869)
• Rheumatology (400)
• Sexual and Reproductive Health (407)
• Sports Medicine (338)
• Surgery (441)
• Toxicology (52)
• Transplantation (185)
• Urology (165)