Logo for British Columbia/Yukon Open Authoring Platform

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

The Research Proposal

83 Components of the Literature Review

Krathwohl (2005) suggests and describes a variety of components to include in a research proposal.  The following sections present these components in a suggested template for you to follow in the preparation of your research proposal.

Introduction

The introduction sets the tone for what follows in your research proposal – treat it as the initial pitch of your idea.  After reading the introduction your reader should:

  • Understand what it is you want to do;
  • Have a sense of your passion for the topic;
  • Be excited about the study´s possible outcomes.

As you begin writing your research proposal it is helpful to think of the introduction as a narrative of what it is you want to do, written in one to three paragraphs.  Within those one to three paragraphs, it is important to briefly answer the following questions:

  • What is the central research problem?
  • How is the topic of your research proposal related to the problem?
  • What methods will you utilize to analyze the research problem?
  • Why is it important to undertake this research? What is the significance of your proposed research?  Why are the outcomes of your proposed research important, and to whom or to what are they important?

Note : You may be asked by your instructor to include an abstract with your research proposal.  In such cases, an abstract should provide an overview of what it is you plan to study, your main research question, a brief explanation of your methods to answer the research question, and your expected findings. All of this information must be carefully crafted in 150 to 250 words.  A word of advice is to save the writing of your abstract until the very end of your research proposal preparation.  If you are asked to provide an abstract, you should include 5-7 key words that are of most relevance to your study. List these in order of relevance.

Background and significance

The purpose of this section is to explain the context of your proposal and to describe, in detail, why it is important to undertake this research. Assume that the person or people who will read your research proposal know nothing or very little about the research problem.  While you do not need to include all knowledge you have learned about your topic in this section, it is important to ensure that you include the most relevant material that will help to explain the goals of your research.

While there are no hard and fast rules, you should attempt to address some or all of the following key points:

  • State the research problem and provide a more thorough explanation about the purpose of the study than what you stated in the introduction.
  • Present the rationale for the proposed research study. Clearly indicate why this research is worth doing.  Answer the “so what?” question.
  • Describe the major issues or problems to be addressed by your research. Do not forget to explain how and in what ways your proposed research builds upon previous related research.
  • Explain how you plan to go about conducting your research.
  • Clearly identify the key or most relevant sources of research you intend to use and explain how they will contribute to your analysis of the topic.
  • Set the boundaries of your proposed research, in order to provide a clear focus. Where appropriate, state not only what you will study, but what will be excluded from your study.
  • Provide clear definitions of key concepts and terms. As key concepts and terms often have numerous definitions, make sure you state which definition you will be utilizing in your research.

Literature Review

This is the most time-consuming aspect in the preparation of your research proposal and it is a key component of the research proposal. As described in Chapter 5 , the literature review provides the background to your study and demonstrates the significance of the proposed research. Specifically, it is a review and synthesis of prior research that is related to the problem you are setting forth to investigate.  Essentially, your goal in the literature review is to place your research study within the larger whole of what has been studied in the past, while demonstrating to your reader that your work is original, innovative, and adds to the larger whole.

As the literature review is information dense, it is essential that this section be intelligently structured to enable your reader to grasp the key arguments underpinning your study. However, this can be easier to state and harder to do, simply due to the fact there is usually a plethora of related research to sift through. Consequently, a good strategy for writing the literature review is to break the literature into conceptual categories or themes, rather than attempting to describe various groups of literature you reviewed.  Chapter V, “ The Literature Review ,” describes a variety of methods to help you organize the themes.

Here are some suggestions on how to approach the writing of your literature review:

  • Think about what questions other researchers have asked, what methods they used, what they found, and what they recommended based upon their findings.
  • Do not be afraid to challenge previous related research findings and/or conclusions.
  • Assess what you believe to be missing from previous research and explain how your research fills in this gap and/or extends previous research

It is important to note that a significant challenge related to undertaking a literature review is knowing when to stop.  As such, it is important to know how to know when you have uncovered the key conceptual categories underlying your research topic.  Generally, when you start to see repetition in the conclusions or recommendations, you can have confidence that you have covered all of the significant conceptual categories in your literature review.  However, it is also important to acknowledge that researchers often find themselves returning to the literature as they collect and analyze their data.  For example, an unexpected finding may develop as one collects and/or analyzes the data and it is important to take the time to step back and review the literature again, to ensure that no other researchers have found a similar finding.  This may include looking to research outside your field.

This situation occurred with one of the authors of this textbook´s research related to community resilience.  During the interviews, the researchers heard many participants discuss individual resilience factors and how they believed these individual factors helped make the community more resilient, overall.  Sheppard and Williams (2016) had not discovered these individual factors in their original literature review on community and environmental resilience. However, when they returned to the literature to search for individual resilience factors, they discovered a small body of literature in the child and youth psychology field. Consequently, Sheppard and Williams had to go back and add a new section to their literature review on individual resilience factors. Interestingly, their research appeared to be the first research to link individual resilience factors with community resilience factors.

Research design and methods

The objective of this section of the research proposal is to convince the reader that your overall research design and methods of analysis will enable you to solve the research problem you have identified and also enable you to accurately and effectively interpret the results of your research. Consequently, it is critical that the research design and methods section is well-written, clear, and logically organized.  This demonstrates to your reader that you know what you are going to do and how you are going to do it.  Overall, you want to leave your reader feeling confident that you have what it takes to get this research study completed in a timely fashion.

Essentially, this section of the research proposal should be clearly tied to the specific objectives of your study; however, it is also important to draw upon and include examples from the literature review that relate to your design and intended methods.  In other words, you must clearly demonstrate how your study utilizes and builds upon past studies, as it relates to the research design and intended methods.  For example, what methods have been used by other researchers in similar studies?

While it is important to consider the methods that other researchers have employed, it is equally important, if not more so, to consider what methods have not been employed but could be.  Remember, the methods section is not simply a list of tasks to be undertaken. It is also an argument as to why and how the tasks you have outlined will help you investigate the research problem and answer your research question(s).

Tips for writing the research design and methods section:

  • Specify the methodological approaches you intend to employ to obtain information and the techniques you will use to analyze the data.
  • Specify the research operations you will undertake and he way you will interpret the results of those operations in relation to the research problem.
  • Go beyond stating what you hope to achieve through the methods you have chosen. State how you will actually do the methods (i.e. coding interview text, running regression analysis, etc.).
  • Anticipate and acknowledge any potential barriers you may encounter when undertaking your research and describe how you will address these barriers.
  • Explain where you believe you will find challenges related to data collection, including access to participants and information.

Preliminary suppositions and implications

The purpose of this section is to argue how and in what ways you anticipate that your research will refine, revise, or extend existing knowledge in the area of your study. Depending upon the aims and objectives of your study, you should also discuss how your anticipated findings may impact future research.  For example, is it possible that your research may lead to a new policy, new theoretical understanding, or a new method for analyzing data?  How might your study influence future studies?  What might your study mean for future practitioners working in the field?  Who or what may benefit from your study?  How might your study contribute to social, economic, environmental issues?  While it is important to think about and discuss possibilities such as these, it is equally important to be realistic in stating your anticipated findings.  In other words, you do not want to delve into idle speculation.  Rather, the purpose here is to reflect upon gaps in the current body of literature and to describe how and in what ways you anticipate your research will begin to fill in some or all of those gaps.

The conclusion reiterates the importance and significance of your research proposal and it provides a brief summary of the entire proposed study.  Essentially, this section should only be one or two paragraphs in length. Here is a potential outline for your conclusion:

  • Discuss why the study should be done. Specifically discuss how you expect your study will advance existing knowledge and how your study is unique.
  • Explain the specific purpose of the study and the research questions that the study will answer.
  • Explain why the research design and methods chosen for this study are appropriate, and why other design and methods were not chosen.
  • State the potential implications you expect to emerge from your proposed study,
  • Provide a sense of how your study fits within the broader scholarship currently in existence related to the research problem.

As with any scholarly research paper, you must cite the sources you used in composing your research proposal.  In a research proposal, this can take two forms: a reference list or a bibliography.  A reference list does what the name suggests, it lists the literature you referenced in the body of your research proposal.  All references in the reference list, must appear in the body of the research proposal.  Remember, it is not acceptable to say “as cited in …”  As a researcher you must always go to the original source and check it for yourself.  Many errors are made in referencing, even by top researchers, and so it is important not to perpetuate an error made by someone else. While this can be time consuming, it is the proper way to undertake a literature review.

In contrast, a bibliography , is a list of everything you used or cited in your research proposal, with additional citations to any key sources relevant to understanding the research problem.  In other words, sources cited in your bibliography may not necessarily appear in the body of your research proposal.  Make sure you check with your instructor to see which of the two you are expected to produce.

Overall, your list of citations should be a testament to the fact that you have done a sufficient level of preliminary research to ensure that your project will complement, but not duplicate, previous research efforts. For social sciences, the reference list or bibliography should be prepared in American Psychological Association (APA) referencing format. Usually, the reference list (or bibliography) is not included in the word count of the research proposal. Again, make sure you check with your instructor to confirm.

An Introduction to Research Methods in Sociology Copyright © 2019 by Valerie A. Sheppard is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

OWL logo

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

  • An introductory paragraph that explains what your working topic and thesis is
  • A forecast of key topics or texts that will appear in the review
  • Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
  • Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
  • Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
  • Critically Evaluate: Mention the strengths and weaknesses of your sources
  • Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

  • Summarize the key findings you have taken from the literature and emphasize their significance
  • Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

  • Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
  • Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
  • Qualitative versus quantitative research
  • Empirical versus theoretical scholarship
  • Divide the research by sociological, historical, or cultural sources
  • Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

  • It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
  • Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
  • Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
  • Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

The Writing Center • University of North Carolina at Chapel Hill

Literature Reviews

What this handout is about.

This handout will explain what literature reviews are and offer insights into the form and construction of literature reviews in the humanities, social sciences, and sciences.

Introduction

OK. You’ve got to write a literature review. You dust off a novel and a book of poetry, settle down in your chair, and get ready to issue a “thumbs up” or “thumbs down” as you leaf through the pages. “Literature review” done. Right?

Wrong! The “literature” of a literature review refers to any collection of materials on a topic, not necessarily the great literary texts of the world. “Literature” could be anything from a set of government pamphlets on British colonial methods in Africa to scholarly articles on the treatment of a torn ACL. And a review does not necessarily mean that your reader wants you to give your personal opinion on whether or not you liked these sources.

What is a literature review, then?

A literature review discusses published information in a particular subject area, and sometimes information in a particular subject area within a certain time period.

A literature review can be just a simple summary of the sources, but it usually has an organizational pattern and combines both summary and synthesis. A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information. It might give a new interpretation of old material or combine new with old interpretations. Or it might trace the intellectual progression of the field, including major debates. And depending on the situation, the literature review may evaluate the sources and advise the reader on the most pertinent or relevant.

But how is a literature review different from an academic research paper?

The main focus of an academic research paper is to develop a new argument, and a research paper is likely to contain a literature review as one of its parts. In a research paper, you use the literature as a foundation and as support for a new insight that you contribute. The focus of a literature review, however, is to summarize and synthesize the arguments and ideas of others without adding new contributions.

Why do we write literature reviews?

Literature reviews provide you with a handy guide to a particular topic. If you have limited time to conduct research, literature reviews can give you an overview or act as a stepping stone. For professionals, they are useful reports that keep them up to date with what is current in the field. For scholars, the depth and breadth of the literature review emphasizes the credibility of the writer in his or her field. Literature reviews also provide a solid background for a research paper’s investigation. Comprehensive knowledge of the literature of the field is essential to most research papers.

Who writes these things, anyway?

Literature reviews are written occasionally in the humanities, but mostly in the sciences and social sciences; in experiment and lab reports, they constitute a section of the paper. Sometimes a literature review is written as a paper in itself.

Let’s get to it! What should I do before writing the literature review?

If your assignment is not very specific, seek clarification from your instructor:

  • Roughly how many sources should you include?
  • What types of sources (books, journal articles, websites)?
  • Should you summarize, synthesize, or critique your sources by discussing a common theme or issue?
  • Should you evaluate your sources?
  • Should you provide subheadings and other background information, such as definitions and/or a history?

Find models

Look for other literature reviews in your area of interest or in the discipline and read them to get a sense of the types of themes you might want to look for in your own research or ways to organize your final review. You can simply put the word “review” in your search engine along with your other topic terms to find articles of this type on the Internet or in an electronic database. The bibliography or reference section of sources you’ve already read are also excellent entry points into your own research.

Narrow your topic

There are hundreds or even thousands of articles and books on most areas of study. The narrower your topic, the easier it will be to limit the number of sources you need to read in order to get a good survey of the material. Your instructor will probably not expect you to read everything that’s out there on the topic, but you’ll make your job easier if you first limit your scope.

Keep in mind that UNC Libraries have research guides and to databases relevant to many fields of study. You can reach out to the subject librarian for a consultation: https://library.unc.edu/support/consultations/ .

And don’t forget to tap into your professor’s (or other professors’) knowledge in the field. Ask your professor questions such as: “If you had to read only one book from the 90’s on topic X, what would it be?” Questions such as this help you to find and determine quickly the most seminal pieces in the field.

Consider whether your sources are current

Some disciplines require that you use information that is as current as possible. In the sciences, for instance, treatments for medical problems are constantly changing according to the latest studies. Information even two years old could be obsolete. However, if you are writing a review in the humanities, history, or social sciences, a survey of the history of the literature may be what is needed, because what is important is how perspectives have changed through the years or within a certain time period. Try sorting through some other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to consider what is currently of interest to scholars in this field and what is not.

Strategies for writing the literature review

Find a focus.

A literature review, like a term paper, is usually organized around ideas, not the sources themselves as an annotated bibliography would be organized. This means that you will not just simply list your sources and go into detail about each one of them, one at a time. No. As you read widely but selectively in your topic area, consider instead what themes or issues connect your sources together. Do they present one or different solutions? Is there an aspect of the field that is missing? How well do they present the material and do they portray it according to an appropriate theory? Do they reveal a trend in the field? A raging debate? Pick one of these themes to focus the organization of your review.

Convey it to your reader

A literature review may not have a traditional thesis statement (one that makes an argument), but you do need to tell readers what to expect. Try writing a simple statement that lets the reader know what is your main organizing principle. Here are a couple of examples:

The current trend in treatment for congestive heart failure combines surgery and medicine. More and more cultural studies scholars are accepting popular media as a subject worthy of academic consideration.

Consider organization

You’ve got a focus, and you’ve stated it clearly and directly. Now what is the most effective way of presenting the information? What are the most important topics, subtopics, etc., that your review needs to include? And in what order should you present them? Develop an organization for your review at both a global and local level:

First, cover the basic categories

Just like most academic papers, literature reviews also must contain at least three basic elements: an introduction or background information section; the body of the review containing the discussion of sources; and, finally, a conclusion and/or recommendations section to end the paper. The following provides a brief description of the content of each:

  • Introduction: Gives a quick idea of the topic of the literature review, such as the central theme or organizational pattern.
  • Body: Contains your discussion of sources and is organized either chronologically, thematically, or methodologically (see below for more information on each).
  • Conclusions/Recommendations: Discuss what you have drawn from reviewing literature so far. Where might the discussion proceed?

Organizing the body

Once you have the basic categories in place, then you must consider how you will present the sources themselves within the body of your paper. Create an organizational method to focus this section even further.

To help you come up with an overall organizational framework for your review, consider the following scenario:

You’ve decided to focus your literature review on materials dealing with sperm whales. This is because you’ve just finished reading Moby Dick, and you wonder if that whale’s portrayal is really real. You start with some articles about the physiology of sperm whales in biology journals written in the 1980’s. But these articles refer to some British biological studies performed on whales in the early 18th century. So you check those out. Then you look up a book written in 1968 with information on how sperm whales have been portrayed in other forms of art, such as in Alaskan poetry, in French painting, or on whale bone, as the whale hunters in the late 19th century used to do. This makes you wonder about American whaling methods during the time portrayed in Moby Dick, so you find some academic articles published in the last five years on how accurately Herman Melville portrayed the whaling scene in his novel.

Now consider some typical ways of organizing the sources into a review:

  • Chronological: If your review follows the chronological method, you could write about the materials above according to when they were published. For instance, first you would talk about the British biological studies of the 18th century, then about Moby Dick, published in 1851, then the book on sperm whales in other art (1968), and finally the biology articles (1980s) and the recent articles on American whaling of the 19th century. But there is relatively no continuity among subjects here. And notice that even though the sources on sperm whales in other art and on American whaling are written recently, they are about other subjects/objects that were created much earlier. Thus, the review loses its chronological focus.
  • By publication: Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on biological studies of sperm whales if the progression revealed a change in dissection practices of the researchers who wrote and/or conducted the studies.
  • By trend: A better way to organize the above sources chronologically is to examine the sources under another trend, such as the history of whaling. Then your review would have subsections according to eras within this period. For instance, the review might examine whaling from pre-1600-1699, 1700-1799, and 1800-1899. Under this method, you would combine the recent studies on American whaling in the 19th century with Moby Dick itself in the 1800-1899 category, even though the authors wrote a century apart.
  • Thematic: Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For instance, the sperm whale review could focus on the development of the harpoon for whale hunting. While the study focuses on one topic, harpoon technology, it will still be organized chronologically. The only difference here between a “chronological” and a “thematic” approach is what is emphasized the most: the development of the harpoon or the harpoon technology.But more authentic thematic reviews tend to break away from chronological order. For instance, a thematic review of material on sperm whales might examine how they are portrayed as “evil” in cultural documents. The subsections might include how they are personified, how their proportions are exaggerated, and their behaviors misunderstood. A review organized in this manner would shift between time periods within each section according to the point made.
  • Methodological: A methodological approach differs from the two above in that the focusing factor usually does not have to do with the content of the material. Instead, it focuses on the “methods” of the researcher or writer. For the sperm whale project, one methodological approach would be to look at cultural differences between the portrayal of whales in American, British, and French art work. Or the review might focus on the economic impact of whaling on a community. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed. Once you’ve decided on the organizational method for the body of the review, the sections you need to include in the paper should be easy to figure out. They should arise out of your organizational strategy. In other words, a chronological review would have subsections for each vital time period. A thematic review would have subtopics based upon factors that relate to the theme or issue.

Sometimes, though, you might need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. Put in only what is necessary. Here are a few other sections you might want to consider:

  • Current Situation: Information necessary to understand the topic or focus of the literature review.
  • History: The chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
  • Methods and/or Standards: The criteria you used to select the sources in your literature review or the way in which you present your information. For instance, you might explain that your review includes only peer-reviewed articles and journals.

Questions for Further Research: What questions about the field has the review sparked? How will you further your research as a result of the review?

Begin composing

Once you’ve settled on a general pattern of organization, you’re ready to write each section. There are a few guidelines you should follow during the writing stage as well. Here is a sample paragraph from a literature review about sexism and language to illuminate the following discussion:

However, other studies have shown that even gender-neutral antecedents are more likely to produce masculine images than feminine ones (Gastil, 1990). Hamilton (1988) asked students to complete sentences that required them to fill in pronouns that agreed with gender-neutral antecedents such as “writer,” “pedestrian,” and “persons.” The students were asked to describe any image they had when writing the sentence. Hamilton found that people imagined 3.3 men to each woman in the masculine “generic” condition and 1.5 men per woman in the unbiased condition. Thus, while ambient sexism accounted for some of the masculine bias, sexist language amplified the effect. (Source: Erika Falk and Jordan Mills, “Why Sexist Language Affects Persuasion: The Role of Homophily, Intended Audience, and Offense,” Women and Language19:2).

Use evidence

In the example above, the writers refer to several other sources when making their point. A literature review in this sense is just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence to show that what you are saying is valid.

Be selective

Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the review’s focus, whether it is thematic, methodological, or chronological.

Use quotes sparingly

Falk and Mills do not use any direct quotes. That is because the survey nature of the literature review does not allow for in-depth discussion or detailed quotes from the text. Some short quotes here and there are okay, though, if you want to emphasize a point, or if what the author said just cannot be rewritten in your own words. Notice that Falk and Mills do quote certain terms that were coined by the author, not common knowledge, or taken directly from the study. But if you find yourself wanting to put in more quotes, check with your instructor.

Summarize and synthesize

Remember to summarize and synthesize your sources within each paragraph as well as throughout the review. The authors here recapitulate important features of Hamilton’s study, but then synthesize it by rephrasing the study’s significance and relating it to their own work.

Keep your own voice

While the literature review presents others’ ideas, your voice (the writer’s) should remain front and center. Notice that Falk and Mills weave references to other sources into their own text, but they still maintain their own voice by starting and ending the paragraph with their own ideas and their own words. The sources support what Falk and Mills are saying.

Use caution when paraphrasing

When paraphrasing a source that is not your own, be sure to represent the author’s information or opinions accurately and in your own words. In the preceding example, Falk and Mills either directly refer in the text to the author of their source, such as Hamilton, or they provide ample notation in the text when the ideas they are mentioning are not their own, for example, Gastil’s. For more information, please see our handout on plagiarism .

Revise, revise, revise

Draft in hand? Now you’re ready to revise. Spending a lot of time revising is a wise idea, because your main objective is to present the material, not the argument. So check over your review again to make sure it follows the assignment and/or your outline. Then, just as you would for most other academic forms of writing, rewrite or rework the language of your review so that you’ve presented your information in the most concise manner possible. Be sure to use terminology familiar to your audience; get rid of unnecessary jargon or slang. Finally, double check that you’ve documented your sources and formatted the review appropriately for your discipline. For tips on the revising and editing process, see our handout on revising drafts .

Works consulted

We consulted these works while writing this handout. This is not a comprehensive list of resources on the handout’s topic, and we encourage you to do your own research to find additional publications. Please do not use this list as a model for the format of your own reference list, as it may not match the citation style you are using. For guidance on formatting citations, please see the UNC Libraries citation tutorial . We revise these tips periodically and welcome feedback.

Anson, Chris M., and Robert A. Schwegler. 2010. The Longman Handbook for Writers and Readers , 6th ed. New York: Longman.

Jones, Robert, Patrick Bizzaro, and Cynthia Selfe. 1997. The Harcourt Brace Guide to Writing in the Disciplines . New York: Harcourt Brace.

Lamb, Sandra E. 1998. How to Write It: A Complete Guide to Everything You’ll Ever Write . Berkeley: Ten Speed Press.

Rosen, Leonard J., and Laurence Behrens. 2003. The Allyn & Bacon Handbook , 5th ed. New York: Longman.

Troyka, Lynn Quittman, and Doug Hesse. 2016. Simon and Schuster Handbook for Writers , 11th ed. London: Pearson.

You may reproduce it for non-commercial use if you use the entire handout and attribute the source: The Writing Center, University of North Carolina at Chapel Hill

Make a Gift

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

  • Knowledge Base
  • Dissertation
  • What is a Literature Review? | Guide, Template, & Examples

What is a Literature Review? | Guide, Template, & Examples

Published on 22 February 2022 by Shona McCombes . Revised on 7 June 2022.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research.

There are five key steps to writing a literature review:

  • Search for relevant literature
  • Evaluate sources
  • Identify themes, debates and gaps
  • Outline the structure
  • Write your literature review

A good literature review doesn’t just summarise sources – it analyses, synthesises, and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

upload-your-document-ai-proofreader

Table of contents

Why write a literature review, examples of literature reviews, step 1: search for relevant literature, step 2: evaluate and select sources, step 3: identify themes, debates and gaps, step 4: outline your literature review’s structure, step 5: write your literature review, frequently asked questions about literature reviews, introduction.

  • Quick Run-through
  • Step 1 & 2

When you write a dissertation or thesis, you will have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

  • Demonstrate your familiarity with the topic and scholarly context
  • Develop a theoretical framework and methodology for your research
  • Position yourself in relation to other researchers and theorists
  • Show how your dissertation addresses a gap or contributes to a debate

You might also have to write a literature review as a stand-alone assignment. In this case, the purpose is to evaluate the current state of research and demonstrate your knowledge of scholarly debates around a topic.

The content will look slightly different in each case, but the process of conducting a literature review follows the same steps. We’ve written a step-by-step guide that you can follow below.

Literature review guide

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

four components of a literature review

Correct my document today

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

  • Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
  • Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
  • Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
  • Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research objectives and questions .

If you are writing a literature review as a stand-alone assignment, you will have to choose a focus and develop a central question to direct your search. Unlike a dissertation research question, this question has to be answerable without collecting original data. You should be able to answer it based only on a review of existing publications.

Make a list of keywords

Start by creating a list of keywords related to your research topic. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list if you discover new keywords in the process of your literature search.

  • Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
  • Body image, self-perception, self-esteem, mental health
  • Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some databases to search for journals and articles include:

  • Your university’s library catalogue
  • Google Scholar
  • Project Muse (humanities and social sciences)
  • Medline (life sciences and biomedicine)
  • EconLit (economics)
  • Inspec (physics, engineering and computer science)

You can use boolean operators to help narrow down your search:

Read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

To identify the most important publications on your topic, take note of recurring citations. If the same authors, books or articles keep appearing in your reading, make sure to seek them out.

You probably won’t be able to read absolutely everything that has been written on the topic – you’ll have to evaluate which sources are most relevant to your questions.

For each publication, ask yourself:

  • What question or problem is the author addressing?
  • What are the key concepts and how are they defined?
  • What are the key theories, models and methods? Does the research use established frameworks or take an innovative approach?
  • What are the results and conclusions of the study?
  • How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
  • How does the publication contribute to your understanding of the topic? What are its key insights and arguments?
  • What are the strengths and weaknesses of the research?

Make sure the sources you use are credible, and make sure you read any landmark studies and major theories in your field of research.

You can find out how many times an article has been cited on Google Scholar – a high citation count means the article has been influential in the field, and should certainly be included in your literature review.

The scope of your review will depend on your topic and discipline: in the sciences you usually only review recent literature, but in the humanities you might take a long historical perspective (for example, to trace how a concept has changed in meaning over time).

Remember that you can use our template to summarise and evaluate sources you’re thinking about using!

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It’s important to keep track of your sources with references to avoid plagiarism . It can be helpful to make an annotated bibliography, where you compile full reference information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

You can use our free APA Reference Generator for quick, correct, consistent citations.

Prevent plagiarism, run a free check.

To begin organising your literature review’s argument and structure, you need to understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

  • Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
  • Themes: what questions or concepts recur across the literature?
  • Debates, conflicts and contradictions: where do sources disagree?
  • Pivotal publications: are there any influential theories or studies that changed the direction of the field?
  • Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

  • Most research has focused on young women.
  • There is an increasing interest in the visual aspects of social media.
  • But there is still a lack of robust research on highly-visual platforms like Instagram and Snapchat – this is a gap that you could address in your own research.

There are various approaches to organising the body of a literature review. You should have a rough idea of your strategy before you start writing.

Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarising sources in order.

Try to analyse patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organise your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

  • Look at what results have emerged in qualitative versus quantitative research
  • Discuss how the topic has been approached by empirical versus theoretical scholarship
  • Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text, your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

If you are writing the literature review as part of your dissertation or thesis, reiterate your central problem or research question and give a brief summary of the scholarly context. You can emphasise the timeliness of the topic (“many recent studies have focused on the problem of x”) or highlight a gap in the literature (“while there has been much research on x, few researchers have taken y into consideration”).

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, make sure to follow these tips:

  • Summarise and synthesise: give an overview of the main points of each source and combine them into a coherent whole.
  • Analyse and interpret: don’t just paraphrase other researchers – add your own interpretations, discussing the significance of findings in relation to the literature as a whole.
  • Critically evaluate: mention the strengths and weaknesses of your sources.
  • Write in well-structured paragraphs: use transitions and topic sentences to draw connections, comparisons and contrasts.

In the conclusion, you should summarise the key findings you have taken from the literature and emphasise their significance.

If the literature review is part of your dissertation or thesis, reiterate how your research addresses gaps and contributes new knowledge, or discuss how you have drawn on existing theories and methods to build a framework for your research. This can lead directly into your methodology section.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

  • To familiarise yourself with the current state of knowledge on your topic
  • To ensure that you’re not just repeating what others have already done
  • To identify gaps in knowledge and unresolved problems that your research can address
  • To develop your theoretical framework and methodology
  • To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your  dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2022, June 07). What is a Literature Review? | Guide, Template, & Examples. Scribbr. Retrieved 27 May 2024, from https://www.scribbr.co.uk/thesis-dissertation/literature-review/

Is this article helpful?

Shona McCombes

Shona McCombes

Other students also liked, how to write a dissertation proposal | a step-by-step guide, what is a theoretical framework | a step-by-step guide, what is a research methodology | steps & tips.

University Library

Write a literature review.

  • Examples and Further Information

1. Introduction

Not to be confused with a book review, a literature review surveys scholarly articles, books and other sources (e.g. dissertations, conference proceedings) relevant to a particular issue, area of research, or theory, providing a description, summary, and critical evaluation of each work. The purpose is to offer an overview of significant literature published on a topic.

2. Components

Similar to primary research, development of the literature review requires four stages:

  • Problem formulation—which topic or field is being examined and what are its component issues?
  • Literature search—finding materials relevant to the subject being explored
  • Data evaluation—determining which literature makes a significant contribution to the understanding of the topic
  • Analysis and interpretation—discussing the findings and conclusions of pertinent literature

Literature reviews should comprise the following elements:

  • An overview of the subject, issue or theory under consideration, along with the objectives of the literature review
  • Division of works under review into categories (e.g. those in support of a particular position, those against, and those offering alternative theses entirely)
  • Explanation of how each work is similar to and how it varies from the others
  • Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research

In assessing each piece, consideration should be given to:

  • Provenance—What are the author's credentials? Are the author's arguments supported by evidence (e.g. primary historical material, case studies, narratives, statistics, recent scientific findings)?
  • Objectivity—Is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
  • Persuasiveness—Which of the author's theses are most/least convincing?
  • Value—Are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

3. Definition and Use/Purpose

A literature review may constitute an essential chapter of a thesis or dissertation, or may be a self-contained review of writings on a subject. In either case, its purpose is to:

  • Place each work in the context of its contribution to the understanding of the subject under review
  • Describe the relationship of each work to the others under consideration
  • Identify new ways to interpret, and shed light on any gaps in, previous research
  • Resolve conflicts amongst seemingly contradictory previous studies
  • Identify areas of prior scholarship to prevent duplication of effort
  • Point the way forward for further research
  • Place one's original work (in the case of theses or dissertations) in the context of existing literature

The literature review itself, however, does not present new primary scholarship.

  • Next: Examples and Further Information >>

spacer bullet

Creative Commons Attribution 3.0 License except where otherwise noted.

Library Twitter page

Land Acknowledgement

The land on which we gather is the unceded territory of the Awaswas-speaking Uypi Tribe. The Amah Mutsun Tribal Band, comprised of the descendants of indigenous people taken to missions Santa Cruz and San Juan Bautista during Spanish colonization of the Central Coast, is today working hard to restore traditional stewardship practices on these lands and heal from historical trauma.

The land acknowledgement used at UC Santa Cruz was developed in partnership with the Amah Mutsun Tribal Band Chairman and the Amah Mutsun Relearning Program at the UCSC Arboretum .

  • UConn Library
  • Literature Review: The What, Why and How-to Guide
  • Introduction

Literature Review: The What, Why and How-to Guide — Introduction

  • Getting Started
  • How to Pick a Topic
  • Strategies to Find Sources
  • Evaluating Sources & Lit. Reviews
  • Tips for Writing Literature Reviews
  • Writing Literature Review: Useful Sites
  • Citation Resources
  • Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

  • To develop a theory or evaluate an existing theory
  • To summarize the historical or existing state of a research topic
  • Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

  • A research paper assigned in a course
  • A thesis or dissertation
  • A grant proposal
  • An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

  • Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

  • Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

  • Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

  • Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

  • UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
  • << Previous: Getting Started
  • Next: How to Pick a Topic >>
  • Last Updated: Sep 21, 2022 2:16 PM
  • URL: https://guides.lib.uconn.edu/literaturereview

Creative Commons

Next generation now

  • Study resources
  • Calendar - Graduate
  • Calendar - Undergraduate
  • Class schedules
  • Class cancellations
  • Course registration
  • Important academic dates
  • More academic resources
  • Campus services
  • IT services
  • Job opportunities
  • Mental health support
  • Student Service Centre (Birks)
  • Calendar of events
  • Latest news
  • Media Relations
  • Faculties, Schools & Colleges
  • Arts and Science
  • Gina Cody School of Engineering and Computer Science
  • John Molson School of Business
  • School of Graduate Studies
  • All Schools, Colleges & Departments
  • Directories

Concordia University logo

  • My Library Account (Sofia) View checkouts, fees, place requests and more
  • Interlibrary Loans Request books from external libraries
  • Zotero Manage your citations and create bibliographies
  • E-journals via BrowZine Browse & read journals through a friendly interface
  • Article/Chapter Scan & Deliver Request a PDF of an article/chapter we have in our physical collection
  • Course Reserves Online course readings
  • Spectrum Deposit a thesis or article
  • WebPrint Upload documents to print with DPrint
  • Sofia Discovery tool
  • Databases by subject
  • Course Reserves
  • E-journals via BrowZine
  • E-journals via Sofia
  • Article/Chapter Scan & Deliver
  • Intercampus Delivery of Bound Periodicals/Microforms
  • Interlibrary Loans
  • Spectrum Research Repository
  • Special Collections
  • Additional resources & services
  • Loans & Returns (Circulation)
  • Subject & course guides
  • Open Educational Resources Guide
  • General guides for users
  • Evaluating...
  • Ask a librarian
  • Research Skills Tutorial
  • Quick Things for Digital Knowledge
  • Bibliometrics & research impact guide
  • Concordia University Press
  • Copyright Guide
  • Copyright Guide for Thesis Preparation
  • Digital Scholarship
  • Digital Preservation
  • Open Access
  • ORCID at Concordia
  • Research data management guide
  • Scholarship of Teaching & Learning
  • Systematic Reviews
  • How to get published speaker series
  • Borrow (laptops, tablets, equipment)
  • Connect (netname, Wi-Fi, guest accounts)
  • Desktop computers, software & availability maps
  • Group study, presentation practice & classrooms
  • Printers, copiers & scanners
  • Technology Sandbox
  • Visualization Studio
  • Webster Library
  • Vanier Library
  • Grey Nuns Reading Room
  • Book a group study room/scanner
  • Study spaces
  • Floor plans
  • Room booking for academic events
  • Exhibitions
  • Librarians & staff
  • University Librarian
  • Memberships & collaborations
  • Indigenous Student Librarian program
  • Wikipedian in residence
  • Researcher-in-Residence
  • Feedback & improvement
  • Annual reports & fast facts
  • Annual Plan
  • Library Services Fund
  • Giving to the Library
  • Webster Transformation blog
  • Policies & Code of Conduct

The Campaign for Concordia

Library Research Skills Tutorial

Log into...

  • My Library account (Sofia)
  • Interlibrary loans
  • Article/chapter scan
  • Course reserves

Quick links

How to write a literature review

What is a literature review.

The literature review is a written overview of major writings and other sources on a selected topic. Sources covered in the review may include scholarly journal articles, books, government reports, Web sites, etc. The literature review provides a description, summary and evaluation of each source. It is usually presented as a distinct section of a graduate thesis or dissertation.

Back to top

Purpose of the literature review

The purpose of the literature review is to provide a critical written account of the current state of research on a selected topic:

  • Identifies areas of prior scholarship
  • Places each source in the context of its contribution to the understanding of the specific issue, area of research, or theory under review.
  • Describes the relationship of each source to the others that you have selected
  • Identifies new ways to interpret, and shed light on any gaps in, previous research
  • Points the way forward for further research.

Components of the literature review

The literature review should include the following:

  • Objective of the literature review
  • Overview of the subject under consideration.
  • particular position, those opposed, and those offering completely different arguments.
  • Discussion of both the distinctiveness of each source and its similarities with the others.

Steps in the literature review process

Preparation of a literature review may be divided into four steps:

  • Define your subject and the scope of the review.
  • Search the library catalogue, subject specific databases and other search tools to find sources that are relevant to your topic.
  • Read and evaluate the sources and to determine their suitability to the understanding of topic at hand (see the Evaluating sources section).
  • Analyse, interpret and discuss the findings and conclusions of the sources you selected.

Evaluating sources

In assessing each source, consideration should be given to:

  • What is the author's expertise in this particular field of study (credentials)?
  • Are the author's arguments supported by empirical evidence (e.g. quantitative/qualitative studies)?
  • Is the author's perspective too biased in one direction or are opposing studies and viewpoints also considered?
  • Does the selected source contribute to a more profound understanding of the subject?

Examples of a published literature review

Literature reviews are often published as scholarly articles, books, and reports. Here is an example of a recent literature review published as a scholarly journal article:

Ledesma, M. C., & Calderón, D. (2015). Critical race theory in education: A review of past literature and a look to the future. Qualitative Inquiry, 21(3), 206-222. Link to the article

Additional sources on writing literature reviews

Further information on the literature review process may be found below:

  • Booth, A., Papaioannou, D., & Sutton, A. (2012). Systematic approaches to a successful literature review
  • Fink, A. (2010). Conducting research literature reviews: From the Internet to paper
  • Galvin, J. (2006). Writing literature reviews: A guide for students of the social and behavioral sciences
  • Machi, L. A., & McEvoy, B. T. (2012). The literature review: Six steps to success

Adapted with permission and thanks from How to Write a Literature Review originally created by Kenneth Lyons, McHenry Library, University of California, Santa Cruz.

arrow up, go to top of page

  • USC Libraries
  • Research Guides

Organizing Your Social Sciences Research Paper

  • 5. The Literature Review
  • Purpose of Guide
  • Design Flaws to Avoid
  • Independent and Dependent Variables
  • Glossary of Research Terms
  • Reading Research Effectively
  • Narrowing a Topic Idea
  • Broadening a Topic Idea
  • Extending the Timeliness of a Topic Idea
  • Academic Writing Style
  • Applying Critical Thinking
  • Choosing a Title
  • Making an Outline
  • Paragraph Development
  • Research Process Video Series
  • Executive Summary
  • The C.A.R.S. Model
  • Background Information
  • The Research Problem/Question
  • Theoretical Framework
  • Citation Tracking
  • Content Alert Services
  • Evaluating Sources
  • Primary Sources
  • Secondary Sources
  • Tiertiary Sources
  • Scholarly vs. Popular Publications
  • Qualitative Methods
  • Quantitative Methods
  • Insiderness
  • Using Non-Textual Elements
  • Limitations of the Study
  • Common Grammar Mistakes
  • Writing Concisely
  • Avoiding Plagiarism
  • Footnotes or Endnotes?
  • Further Readings
  • Generative AI and Writing
  • USC Libraries Tutorials and Other Guides
  • Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

  • Give a new interpretation of old material or combine new with old interpretations,
  • Trace the intellectual progression of the field, including major debates,
  • Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
  • Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

  • Place each work in the context of its contribution to understanding the research problem being studied.
  • Describe the relationship of each work to the others under consideration.
  • Identify new ways to interpret prior research.
  • Reveal any gaps that exist in the literature.
  • Resolve conflicts amongst seemingly contradictory previous studies.
  • Identify areas of prior scholarship to prevent duplication of effort.
  • Point the way in fulfilling a need for additional research.
  • Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE: Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

  • An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
  • Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
  • An explanation of how each work is similar to and how it varies from the others,
  • Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

  • Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
  • Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
  • Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
  • Persuasiveness -- which of the author's theses are most convincing or least convincing?
  • Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

  • Current Situation : Information necessary to understand the current topic or focus of the literature review.
  • Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
  • History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
  • Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
  • Standards : Description of the way in which you present your information.
  • Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

  • Sources in your literature review do not clearly relate to the research problem;
  • You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
  • Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
  • Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
  • Does not describe the search procedures that were used in identifying the literature to review;
  • Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
  • Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

  • How are they organizing their ideas?
  • What methods have they used to study the problem?
  • What theories have been used to explain, predict, or understand their research problem?
  • What sources have they cited to support their conclusions?
  • How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

  • Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
  • Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
  • Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

  • << Previous: Theoretical Framework
  • Next: Citation Tracking >>
  • Last Updated: May 25, 2024 4:09 PM
  • URL: https://libguides.usc.edu/writingguide

Grad Coach

Literature Review: 3 Essential Ingredients

The theoretical framework, empirical research and research gap

By: Derek Jansen (MBA) | Reviewer: Eunice Rautenbach (DTech) | July 2023

Writing a comprehensive but concise literature review is no simple task. There’s a lot of ground to cover and it can be challenging to figure out what’s important and what’s not. In this post, we’ll unpack three essential ingredients that need to be woven into your literature review to lay a rock-solid foundation for your study.

This post is based on our popular online course, Literature Review Bootcamp . In the course, we walk you through the full process of developing a literature review, step by step. If it’s your first time writing a literature review, you definitely want to use this link to get 50% off the course (limited-time offer).

Overview: Essential Ingredients

  • Ingredients vs structure
  • The theoretical framework (foundation of theory)
  • The empirical research
  • The research gap
  • Summary & key takeaways

Ingredients vs Structure

As a starting point, it’s important to clarify that the three ingredients we’ll cover in this video are things that need to feature within your literature review, as opposed to a set structure for your chapter . In other words, there are different ways you can weave these three ingredients into your literature review. Regardless of which structure you opt for, each of the three components will make an appearance in some shape or form. If you’re keen to learn more about structural options, we’ve got a dedicated post about that here .

Free Webinar: Literature Review 101

1. The Theoretical Framework

Let’s kick off with the first essential ingredient – that is the theoretical framework , also called the foundation of theory . 

The foundation of theory, as the name suggests, is where you’ll lay down the foundational building blocks for your literature review so that your reader can get a clear idea of the core concepts, theories and assumptions (in relation to your research aims and questions) that will guide your study. Note that this is not the same as a conceptual framework .

Typically you’ll cover a few things within the theoretical framework:

Firstly, you’ll need to clearly define the key constructs and variables that will feature within your study. In many cases, any given term can have multiple different definitions or interpretations – for example, different people will define the concept of “integrity” in different ways. This variation in interpretation can, of course, wreak havoc on how your study is understood. So, this section is where you’ll pin down what exactly you mean when you refer to X, Y or Z in your study, as well as why you chose that specific definition. It’s also a good idea to state any assumptions that are inherent in these definitions and why these are acceptable, given the purpose of your study.

Related to this, the second thing you’ll need to cover in your theoretical framework is the relationships between these variables and/or constructs . For example, how does one variable potentially affect another variable – does A have an impact on B, B on A, and so on? In other words, you want to connect the dots between the different “things” of interest that you’ll be exploring in your study. Note that you only need to focus on the key items of interest here (i.e. those most central to your research aims and questions) – not every possible construct or variable.

Lastly, and very importantly, you need to discuss the existing theories that are relevant to your research aims and research questions . For example, if you’re investigating the uptake/adoption of a certain application or software, you might discuss Davis’ Technology Acceptance Model and unpack what it has to say about the factors that influence technology adoption. More importantly, though, you need to explain how this impacts your expectations about what you will find in your own study . In other words, your theoretical framework should reveal some insights about what answers you might expect to find to your research questions .

If this sounds a bit fluffy, don’t worry. We deep dive into the theoretical framework (as well as the conceptual framework) and look at practical examples in Literature Review Bootcamp . If you’d like to learn more, take advantage of the limited-time offer (60% off the standard price).

Need a helping hand?

four components of a literature review

2. The Empirical Research

Onto the second essential ingredient, which is  empirical research . This section is where you’ll present a critical discussion of the existing empirical research that is relevant to your research aims and questions.

But what exactly is empirical research?

Simply put, empirical research includes any study that involves actual data collection and analysis , whether that’s qualitative data, quantitative data, or a mix of both . This contrasts against purely theoretical literature (the previous ingredient), which draws its conclusions based exclusively on logic and reason , as opposed to an analysis of real-world data.

In other words, theoretical literature provides a prediction or expectation of what one might find based on reason and logic, whereas empirical research tests the accuracy of those predictions using actual real-world data . This reflects the broader process of knowledge creation – in other words, first developing a theory and then testing it out in the field.

Long story short, the second essential ingredient of a high-quality literature review is a critical discussion of the existing empirical research . Here, it’s important to go beyond description . You’ll need to present a critical analysis that addresses some (if not all) of the following questions:

  • What have different studies found in relation to your research questions ?
  • What contexts have (and haven’t been covered)? For example, certain countries, cities, cultures, etc.
  • Are the findings across the studies similar or is there a lot of variation ? If so, why might this be the case?
  • What sorts of research methodologies have been used and how could these help me develop my own methodology?
  • What were the noteworthy limitations of these studies?

Simply put, your task here is to present a synthesis of what’s been done (and found) within the empirical research, so that you can clearly assess the current state of knowledge and identify potential research gaps , which leads us to our third essential ingredient.

Theoretical literature provides predictions, whereas empirical research tests the accuracy of those predictions using real-world data.

The Research Gap

The third essential ingredient of a high-quality literature review is a discussion of the research gap (or gaps).

But what exactly is a research gap?

Simply put, a research gap is any unaddressed or inadequately explored area within the existing body of academic knowledge. In other words, a research gap emerges whenever there’s still some uncertainty regarding a certain topic or question.

For example, it might be the case that there are mixed findings regarding the relationship between two variables (e.g., job performance and work-from-home policies). Similarly, there might be a lack of research regarding the impact of a specific new technology on people’s mental health. On the other end of the spectrum, there might be a wealth of research regarding a certain topic within one country (say the US), but very little research on that same topic in a different social context (say, China).

These are just random examples, but as you can see, research gaps can emerge from many different places. What’s important to understand is that the research gap (or gaps) needs to emerge from your previous discussion of the theoretical and empirical literature . In other words, your discussion in those sections needs to start laying the foundation for the research gap.

For example, when discussing empirical research, you might mention that most studies have focused on a certain context , yet very few (or none) have focused on another context, and there’s reason to believe that findings may differ. Or you might highlight how there’s a fair deal of mixed findings and disagreement regarding a certain matter. In other words, you want to start laying a little breadcrumb trail in those sections so that your discussion of the research gap is firmly rooted in the rest of the literature review.

But why does all of this matter?

Well, the research gap should serve as the core justification for your study . Through your literature review, you’ll show what gaps exist in the current body of knowledge, and then your study will then attempt to fill (or contribute towards filling) one of those gaps. In other words, you’re first explaining what the problem is (some sort of gap) and then proposing how you’ll solve it.

 A research gap exists whenever there’s still a  reasonable level of uncertainty or disagreement regarding a certain topic or question.

Key Takeaways

To recap, the three ingredients that need to be mixed into your literature review are:

  • The foundation of theory or theoretical framework
  • The empirical or evidence-based research

As we mentioned earlier, these are components of a literature review and not (necessarily) a structure for your literature review chapter. Of course, you can structure your chapter in a way that reflects these three components (in fact, in some cases that works very well), but it’s certainly not the only option. The right structure will vary from study to study , depending on various factors.

If you’d like to get hands-on help developing your literature review, be sure to check out our private coaching service , where we hold your hand through the entire research journey, step by step. 

Literature Review Course

Psst… there’s more!

This post is an extract from our bestselling short course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

You Might Also Like:

How to synthesise literature for a literature review

very good , as the first writer of the thesis i will need ur advise . please give me a piece of idea on topic -impact of national standardized exam on students learning engagement . Thank you .

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

  • Print Friendly

University of Texas

  • University of Texas Libraries

Literature Reviews

  • What is a literature review?
  • Steps in the Literature Review Process
  • Define your research question
  • Determine inclusion and exclusion criteria
  • Choose databases and search
  • Review Results
  • Synthesize Results
  • Analyze Results
  • Librarian Support

What is a Literature Review?

A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important past and current research and practices. It provides background and context, and shows how your research will contribute to the field. 

A literature review should: 

  • Provide a comprehensive and updated review of the literature;
  • Explain why this review has taken place;
  • Articulate a position or hypothesis;
  • Acknowledge and account for conflicting and corroborating points of view

From  S age Research Methods

Purpose of a Literature Review

A literature review can be written as an introduction to a study to:

  • Demonstrate how a study fills a gap in research
  • Compare a study with other research that's been done

Or it can be a separate work (a research article on its own) which:

  • Organizes or describes a topic
  • Describes variables within a particular issue/problem

Limitations of a Literature Review

Some of the limitations of a literature review are:

  • It's a snapshot in time. Unlike other reviews, this one has beginning, a middle and an end. There may be future developments that could make your work less relevant.
  • It may be too focused. Some niche studies may miss the bigger picture.
  • It can be difficult to be comprehensive. There is no way to make sure all the literature on a topic was considered.
  • It is easy to be biased if you stick to top tier journals. There may be other places where people are publishing exemplary research. Look to open access publications and conferences to reflect a more inclusive collection. Also, make sure to include opposing views (and not just supporting evidence).

Source: Grant, Maria J., and Andrew Booth. “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies.” Health Information & Libraries Journal, vol. 26, no. 2, June 2009, pp. 91–108. Wiley Online Library, doi:10.1111/j.1471-1842.2009.00848.x.

Meryl Brodsky : Communication and Information Studies

Hannah Chapman Tripp : Biology, Neuroscience

Carolyn Cunningham : Human Development & Family Sciences, Psychology, Sociology

Larayne Dallas : Engineering

Janelle Hedstrom : Special Education, Curriculum & Instruction, Ed Leadership & Policy ​

Susan Macicak : Linguistics

Imelda Vetter : Dell Medical School

For help in other subject areas, please see the guide to library specialists by subject .

Periodically, UT Libraries runs a workshop covering the basics and library support for literature reviews. While we try to offer these once per academic year, we find providing the recording to be helpful to community members who have missed the session. Following is the most recent recording of the workshop, Conducting a Literature Review. To view the recording, a UT login is required.

  • October 26, 2022 recording
  • Last Updated: Oct 26, 2022 2:49 PM
  • URL: https://guides.lib.utexas.edu/literaturereviews

Creative Commons License

Los Angeles Mission College logo

Literature Review

  • What is a Literature Review?
  • What is a good literature review?
  • Types of Literature Reviews
  • What are the parts of a Literature Review?
  • What is the difference between a Systematic Review and a Literature Review?

Parts of a Literature Review

Introduction      .

  • To explain the focus and establish the importance of the subject
  • provide the framework, selection criteria, or parameters of your literature review
  • provide background or history
  • outline what kind of work has been done on the topic
  • briefly identify any controversies within the field or any recent research that has raised questions about earlier assumptions
  • In a stand-alone literature review, this statement will sum up and evaluate the current state of this field of research
  • In a review that is an introduction or preparatory to a thesis or research report, it will suggest how the review findings will lead to the research the writer proposes to undertake.
  • To summarize and evaluate the current state of knowledge in the field
  • To note major themes or topics, the most important trends, and any findings about which researchers agree or disagree
  • Often divided by headings/subheadings
  • If the review is preliminary to your own thesis or research project, its purpose is to make an argument that will justify your proposed research. Therefore, the literature review will discuss only that research which leads directly to your own project.
  • To summarize the evidence presented and show its significance
  • Rather than restating your thesis or purpose statement, explain what your review tells you about the current state of the field
  • If the review is an introduction to your own research, the conclusion highlights gaps and indicates how previous research leads to your own research project and chosen methodology. 
  • If the review is a stand-alone assignment for a course, the conclusion should suggest any practical applications of the research as well as the implications and possibilities for future research.
  • Find out what style guide you are required to follow (e.g., APA, MLA, ASA)
  • Follow the guidelines to format citations and create a reference list or bibliography
  • Cite Your Sources

This work is licensed under a  Creative Commons Attribution-NonCommercial-ShareAlike 4.0  International License. adapted from UofG,McLaughlin Library

  • << Previous: Types of Literature Reviews
  • Next: What is the difference between a Systematic Review and a Literature Review? >>
  • Last Updated: Nov 21, 2023 12:49 PM
  • URL: https://libguides.lamission.edu/c.php?g=1190903

Los Angeles Mission College. All rights reserved. - 13356 Eldridge Avenue, Sylmar, CA 91342. 818-364-7600 | LACCD.edu | ADA Compliance Questions or comments about this web site? Please leave Feedback

Banner Image

Research Process :: Step by Step

  • Introduction
  • Select Topic
  • Identify Keywords
  • Background Information
  • Develop Research Questions
  • Refine Topic
  • Search Strategy
  • Popular Databases
  • Evaluate Sources
  • Types of Periodicals
  • Reading Scholarly Articles
  • Primary & Secondary Sources
  • Organize / Take Notes
  • Writing & Grammar Resources
  • Annotated Bibliography
  • Literature Review
  • Citation Styles
  • Paraphrasing
  • Privacy / Confidentiality
  • Research Process
  • Selecting Your Topic
  • Identifying Keywords
  • Gathering Background Info
  • Evaluating Sources

four components of a literature review

Organize the literature review into sections that present themes or identify trends, including relevant theory. You are not trying to list all the material published, but to synthesize and evaluate it according to the guiding concept of your thesis or research question.  

What is a literature review?

A literature review is an account of what has been published on a topic by accredited scholars and researchers. Occasionally you will be asked to write one as a separate assignment, but more often it is part of the introduction to an essay, research report, or thesis. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries

A literature review must do these things:

  • be organized around and related directly to the thesis or research question you are developing
  • synthesize results into a summary of what is and is not known
  • identify areas of controversy in the literature
  • formulate questions that need further research

Ask yourself questions like these:

  • What is the specific thesis, problem, or research question that my literature review helps to define?
  • What type of literature review am I conducting? Am I looking at issues of theory? methodology? policy? quantitative research (e.g. on the effectiveness of a new procedure)? qualitative research (e.g., studies of loneliness among migrant workers)?
  • What is the scope of my literature review? What types of publications am I using (e.g., journals, books, government documents, popular media)? What discipline am I working in (e.g., nursing psychology, sociology, medicine)?
  • How good was my information seeking? Has my search been wide enough to ensure I've found all the relevant material? Has it been narrow enough to exclude irrelevant material? Is the number of sources I've used appropriate for the length of my paper?
  • Have I critically analyzed the literature I use? Do I follow through a set of concepts and questions, comparing items to each other in the ways they deal with them? Instead of just listing and summarizing items, do I assess them, discussing strengths and weaknesses?
  • Have I cited and discussed studies contrary to my perspective?
  • Will the reader find my literature review relevant, appropriate, and useful?

Ask yourself questions like these about each book or article you include:

  • Has the author formulated a problem/issue?
  • Is it clearly defined? Is its significance (scope, severity, relevance) clearly established?
  • Could the problem have been approached more effectively from another perspective?
  • What is the author's research orientation (e.g., interpretive, critical science, combination)?
  • What is the author's theoretical framework (e.g., psychological, developmental, feminist)?
  • What is the relationship between the theoretical and research perspectives?
  • Has the author evaluated the literature relevant to the problem/issue? Does the author include literature taking positions she or he does not agree with?
  • In a research study, how good are the basic components of the study design (e.g., population, intervention, outcome)? How accurate and valid are the measurements? Is the analysis of the data accurate and relevant to the research question? Are the conclusions validly based upon the data and analysis?
  • In material written for a popular readership, does the author use appeals to emotion, one-sided examples, or rhetorically-charged language and tone? Is there an objective basis to the reasoning, or is the author merely "proving" what he or she already believes?
  • How does the author structure the argument? Can you "deconstruct" the flow of the argument to see whether or where it breaks down logically (e.g., in establishing cause-effect relationships)?
  • In what ways does this book or article contribute to our understanding of the problem under study, and in what ways is it useful for practice? What are the strengths and limitations?
  • How does this book or article relate to the specific thesis or question I am developing?

Text written by Dena Taylor, Health Sciences Writing Centre, University of Toronto

http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review

  • << Previous: Annotated Bibliography
  • Next: Step 5: Cite Sources >>
  • Last Updated: May 28, 2024 4:14 PM
  • URL: https://libguides.uta.edu/researchprocess

University of Texas Arlington Libraries 702 Planetarium Place · Arlington, TX 76019 · 817-272-3000

  • Internet Privacy
  • Accessibility
  • Problems with a guide? Contact Us.

Banner

Literature Reviews

  • Introduction

Problem formulation

Conducting your literature review, evaluating the data, analysing the material.

  • Resources on writing and research
  • Citing and referencing
  • You should think about your research topic and identify central areas and issues.
  • Next you should compile a list of keywords to help you when searching for materials on your topic.

This involves sourcing the literature pertinent to your research topic. Material could be in any format such as books, journals, websites, multimedia sources etc. It is at this stage that the library will be most useful to you.

There are a number of library resources that might be useful to you when conducting your search:

  • The library catalogue  – You can search the catalogue to find materials (books, journals etc) that are available in the library.
  • Library Search - simultaneously searches the library catalogue PLUS online databases, online journals, ebooks, articles, and material freely available online. The library recommends that you use this comprehensive discovery search tool.
  • Online databases– the library subscribes to a number of online databases covering a variety of subjects. These can be accessed through the library website on  Databases A-Z
  • Journals. Journals in print format covering a number of subject disciplines are shelved to the left of the entrance to the library.
  • Journals in electronic format and print can be searched for online by logging on to Publication Finder .   You can search by journal title, search within a particular journal for your topic, or browse by discipline.
  • Library PC’s– there are a number of PC’s on the mezzanine level of the library where you can access the internet and look at web resources related to your topic.
  • Library laptops are available to borrow for three hours within the library.

The resources that you find in the library or electronically could help you to find more material on your topic. Make sure to consult the bibliographies in books, journal article references and links pages on websites which will point you towards other useful material.

Before including any material that you have found in your literature review you must evaluate your results to ensure that the information you have found is relevant, accurate, reliable and current. There are a number of criteria you can use to decide this:

  • what is the content of the source? ( look at contents pages, indexes, abstracts etc.)
  • who are the intended audience? (books aimed at the general public may not be specific enough)
  • Who is the author?
  • What is the edition and publication date? (i.e. is the information recent)
  • Is the source from a well regarded journal?
  • Has the work been reviewed and what do they say?

You need to carefully evaluate web sources as they are not always reliable or accurate. Some other things to keep in mind when evaluating web sources are:

  • what is the domain of the site? (i.e. is it an educational or government site or just someone’s home page),
  • when was it last updated?
  • Have reputable sites got links to this site?

At this stage you must read, interpret and structure the data that you have gathered and finally you must write the review. The review must consist of:

  • An Introduction– here the topic should be specified, overall trends and conflicts in the literature should be outlined and gaps in previous research identified. It is also very important at this point to justify your reasons for writing the review.
  • A body– this will be the bulk of the review and here you will discuss each piece of literature in turn. Research studies should be presented in a logical order e.g. chronological, thematically etc. Previous studies should be summarised and critically evaluated.
  • A conclusion– discuss which studies have made the greatest contribution to the subject. Evaluate the current general state of research in this area and finally discuss the research opportunities in this area.
  • << Previous: Introduction
  • Next: Resources on writing and research >>
  • Last Updated: Jul 17, 2023 2:11 PM
  • URL: https://tudublin.libguides.com/c.php?g=694824

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

  • Advanced Search
  • Journal List
  • J Mark Access Health Policy
  • v.11(1); 2023
  • PMC10392303

Logo of jmaph

Rapid literature review: definition and methodology

Beata smela.

a Assignity, Cracow, Poland

Mondher Toumi

b Public Health Department, Aix-Marseille University, Marseille, France

Karolina Świerk

Clement francois, małgorzata biernikiewicz.

c Studio Slowa, Wroclaw, Poland

Emilie Clay

d Clever-Access, Paris, France

Laurent Boyer

Introduction: A rapid literature review (RLR) is an alternative to systematic literature review (SLR) that can speed up the analysis of newly published data. The objective was to identify and summarize available information regarding different approaches to defining RLR and the methodology applied to the conduct of such reviews.

Methods: The Medline and EMBASE databases, as well as the grey literature, were searched using the set of keywords and their combination related to the targeted and rapid review, as well as design, approach, and methodology. Of the 3,898 records retrieved, 12 articles were included.

Results: Specific definition of RLRs has only been developed in 2021. In terms of methodology, the RLR should be completed within shorter timeframes using simplified procedures in comparison to SLRs, while maintaining a similar level of transparency and minimizing bias. Inherent components of the RLR process should be a clear research question, search protocol, simplified process of study selection, data extraction, and quality assurance.

Conclusions: There is a lack of consensus on the formal definition of the RLR and the best approaches to perform it. The evidence-based supporting methods are evolving, and more work is needed to define the most robust approaches.

Introduction

A systematic literature review (SLR) summarizes the results of all available studies on a specific topic and provides a high level of evidence. Authors of the SLR have to follow an advanced plan that covers defining a priori information regarding the research question, sources they are going to search, inclusion criteria applied to choose studies answering the research question, and information regarding how they are going to summarize findings [ 1 ].

The rigor and transparency of SLRs make them the most reliable form of literature review [ 2 ], providing a comprehensive, objective summary of the evidence for a given topic [ 3 , 4 ]. On the other hand, the SLR process is usually very time-consuming and requires a lot of human resources. Taking into account a high increase of newly published data and a growing need to analyze information in the fastest possible way, rapid literature reviews (RLRs) often replace standard SLRs.

There are several guidelines on the methodology of RLRs [ 5–11 ]; however, only recently, one publication from 2021 attempted to construct a unified definition [ 11 ]. Generally, by RLRs, researchers understand evidence synthesis during which some of the components of the systematic approach are being used to facilitate answering a focused research question; however, scope restrictions and a narrower search strategy help to make the project manageable in a shorter time and to get the key conclusions faster [ 4 ].

The objective of this research was to collect and summarize available information on different approaches to the definition and methodology of RLRs. An RLR has been run to capture publications providing data that fit the project objective.

To find publications reporting information on the methodology of RLRs, searches were run in the Medline and EMBASE databases in November 2022. The following keywords were searched for in titles and abstracts: ‘targeted adj2 review’ OR ‘focused adj2 review’ OR ‘rapid adj2 review’, and ‘methodology’ OR ‘design’ OR ‘scheme’ OR ‘approach’. The grey literature was identified using Google Scholar with keywords including ‘targeted review methodology’ OR ‘focused review methodology’ OR ‘rapid review methodology’. Only publications in English were included, and the date of publication was restricted to year 2016 onward in order to identify the most up-to-date literature. The reference lists of each included article were searched manually to obtain the potentially eligible articles. Titles and abstracts of the retrieved records were first screened to exclude articles that were evidently irrelevant. The full texts of potentially relevant papers were further reviewed to examine their eligibility.

A pre-defined Excel grid was developed to extract the following information related to the methodology of RLR from guidelines:

  • Definition,
  • Research question and searches,
  • Studies selection,
  • Data extraction and quality assessment,
  • Additional information.

There was no restriction on the study types to be analyzed; any study reporting on the methodology of RLRs could be included: reviews, practice guidelines, commentaries, and expert opinions on RLR relevant to healthcare policymakers or practitioners. The data extraction and evidence summary were conducted by one analyst and further examined by a senior analyst to ensure that relevant information was not omitted. Disagreements were resolved by discussion and consensus.

Studies selection

A total of 3,898 records (3,864 articles from a database search and 34 grey literature from Google Scholar) were retrieved. After removing duplicates, titles and abstracts of 3,813 articles were uploaded and screened. The full texts of 43 articles were analyzed resulting in 12 articles selected for this review, including 7 guidelines [ 5–11 ] on the methodology of RLRs, together with 2 papers summarizing the results of the Delphi consensus on the topic [ 12 , 13 ], and 3 publications analyzing and assessing different approaches to RLRs [ 4 , 14 , 15 ].

Overall, seven guidelines were identified: from the World Health Organization (WHO) [ 5 ], National Collaborating Centre for Methods and Tools (NCCMT) [ 7 ], the UK government [ 8 ], the Oxford Centre for Evidence Based Medicine [ 9 ], the Cochrane group [ 6 , 11 ], and one multi-national review [ 10 ]. Among the papers that did not describe the guidelines, Gordon et al. [ 4 ] proposed 12 tips for conducting a rapid review in the right settings and discussed why these reviews may be more beneficial in some circumstances. The objective of work conducted by Tricco et al. [ 13 ] and Pandor et al. [ 12 ] was to collect and compare perceptions of rapid reviews from stakeholders, including researchers, policymakers, industry, journal editors, and healthcare providers, and to reach a consensus outlining the domains to consider when deciding on approaches for RLRs. Haby et al. [ 14 ] run a rapid review of systematic reviews and primary studies to find out the best way to conduct an RLR in health policy and practice. In Tricco et al. (2022) [ 15 ], JBI position statement for RLRs is presented.

From all the seven identified guidelines information regarding definitions the authors used for RLRs, approach to the PICOS criteria and search strategy development, studies selection, data extractions, quality assessment, and reporting were extracted.

Cochrane Rapid Reviews Methods Group developed methods guidance based on scoping review of the underlying evidence, primary methods studies conducted, as well as surveys sent to Cochrane representative and discussion among those with expertise [ 11 ]. They analyzed over 300 RLRs or RLR method papers and based on the methodology of those studies, constructed a broad definition RLR, one that meets a minimum set of requirements identified in the thematic analysis: ‘ A rapid review is a form of knowledge synthesis that accelerates the process of conducting a traditional systematic review through streamlining or omitting a variety of methods to produce evidence in a resource-efficient manner .’ This interpretation aligns with more than 50% of RLRs identified in this study. The authors additionally provided several other definitions, depending on specific situations or requirements (e.g., when RLR is produced on stakeholder’s request). It was additionally underlined that RLRs should be driven by the need of timely evidence for decision-making purposes [ 11 ].

Rapid reviews vary in their objective, format, and methods used for evidence synthesis. This is a quite new area, and still no agreement on optimal methods can be found [ 5 ]. All of the definitions are highlighting that RLRs are completed within shorter timeframes than SLRs, and also lack of time is one of the main reasons they are conducted. It has been suggested that most rapid reviews are conducted within 12 weeks; however, some of the resources suggest time between a few weeks to no more than 6 months [ 5 , 6 ]. Some of the definitions are highlighting that RLRs follow the SLR process, but certain phases of the process are simplified or omitted to retrieve information in a time-saving way [ 6 , 7 ]. Different mechanisms are used to enhance the timeliness of reviews. They can be used independently or concurrently: increasing the intensity of work by intensifying the efforts of multiple analysts by parallelization of tasks, using review shortcuts whereby one or more systematic review steps may be reduced, automatizing review steps by using new technologies [ 5 ]. The UK government report [ 8 ] referred to two different RLRs: in the form of quick scoping reviews (QSR) or rapid evidence assessments (REA). While being less resource and time-consuming compared to standard SLRs, QSRs and REAs are designed to be similarly transparent and to minimize bias. QSRs can be applied to rather open-ended questions, e.g., ‘what do we know about something’ but both, QSRs and REAs, provide an understanding of the volume and characteristics of evidence on a specific topic, allowing answering questions by maximizing the use of existing data, and providing a clear picture of the adequacy of existing evidence [ 8 ].

Research questions and searches

The guidelines suggest creating a clear research question and search protocol at the beginning of the project. Additionally, to not duplicate RLRs, the Cochrane Rapid Reviews Methods Group encourages all people working on RLRs to consider registering their search protocol with PROSPERO, the international prospective register of reviews; however, so far they are not formally registered in most cases [ 5 , 6 ]. They also recommend involving key stakeholders (review users) to set and refine the review question, criteria, and outcomes, as well as consulting them through the entire process [ 11 ].

Regarding research questions, it is better to structure them in a neutral way rather than focus on a specific direction for the outcome. By doing so, the researcher is in a better position to identify all the relevant evidence [ 7 ]. Authors can add a second, supportive research question when needed [ 8 ]. It is encouraged to limit the number of interventions, comparators and outcomes, to focus on the ones that are most important for decision-making [ 11 ]. Useful could be also reviewing additional materials, e.g., SLRs on the topic, as well as conducting a quick literature search to better understand the topic before starting with RLRs [ 7 ]. In SLRs researchers usually do not need to care a lot about time spent on creating PICOS, they need to make sure that the scope is broad enough, and they cannot use many restrictions. When working on RLRs, a reviewer may spend more or less time defining each of the components of the study question, and the main step is making sure that PICOS addresses the needs of those who requested the rapid review, and at the same time, it is feasible within the required time frame [ 7 ]. Search protocol should contain an outline of how the following review steps are to be carried out, including selected search keywords and a full strategy, a list of data sources, precise inclusion and exclusion criteria, a strategy for data extraction and critical appraisal, and a plan of how the information will be synthesized [ 8 ].

In terms of searches running, in most cases, an exhaustive process will not be feasible. Researchers should make sure that the search is effective and efficient to produce results in a timely manner. Cochrane Rapid Reviews Methods Group recommends involving an information specialist and conducting peer review of at least one search strategy [ 11 ]. According to the rapid review guidebook by McMaster University [ 7 ], it is important that RLRs, especially those that support policy and program decisions, are being fed by the results of a body of literature, rather than single studies, when possible. It would result in more generalizable findings applied at the level of a population and serve more realistic findings for program decisions [ 7 ]. It is important to document the search strategy, together with a record of the date and any date limits of the search, so that it can easily be run again, modified, or updated. Furthermore, the information on the individual databases included in platform services should always be reported, as this depends on organizations’ subscriptions and must be included for transparency and repeatability [ 7 , 8 ]. Good solution for RLRs is narrowing the scope or searching a limited number of databases and other sources [ 7 ]. Often, the authors use the PubMed/MEDLINE, Cochrane Library, and Embase databases. In most reviews, two or more databases are searched, and common limits are language (usually restricted to English), date, study design, and geographical area. Some RLRs include searching of grey literature; however, contact with authors is rather uncommon [ 5 , 8 ]. According to the flexible framework for restricted systematic review published by the University of Oxford, the search should be run in at least one major scientific database such as PubMed, and one other source, e.g., Google Scholar [ 9 ]. Grey literature and unpublished evidence may be particularly needed and important for intervention questions. It is related to the fact that studies that do not report the effects of interventions are less likely to be published [ 8 ]. If there is any type of evidence that will not be considered by the RLRs, e.g., reviews or theoretical and conceptual studies, it should also be stated in the protocol together with justification [ 8 ]. Additionally, authors of a practical guide published by WHO suggest using a staged search to identify existing SLRs at the beginning, and then focusing on studies with other designs [ 5 ]. If a low number of citations have been retrieved, it is acceptable to expand searches, remove some of the limits, and add additional databases and sources [ 7 ].

Searching for RLRs is an iterative process, and revising the approach is usually needed [ 7 ]. Changes should be confirmed with stakeholders and should be tracked and reflected in the final report [ 5 ].

The next step in the rapid review is the selection of studies consisting of two phases: screening of titles and abstracts, and analysis of full texts. Prior to screening initiation, it is recommended to conduct a pilot exercise using the same 30–50 abstracts and 5–10 full-texts for the entire screening team in order to calibrate and test the review form [ 11 ]. In contrast to SLRs, it can be done by one reviewer with or without verification by a second one. If verification is performed, usually the second reviewer checks only a subset of records and compares them. Cochrane Group, in contrast, recommends a stricter approach: at least 20% of references should be double-screened at titles and abstracts stage, and while the rest of the references may be screened by one reviewer, the excluded items need to be re-examined by second reviewer; similar approach is used in full-text screening [ 11 ]. This helps to ensure that bias was reduced and that the PICOS criteria are applied in a relevant way [ 5 , 8 , 9 , 11 ]. During the analysis of titles and abstracts, there is no need to report reasons for exclusion; however, they should be tracked for all excluded full texts [ 7 ].

Data extraction and quality assessment

According to the WHO guide, the most common method for data extraction in RLRs is extraction done by a single reviewer with or without partial verification. The authors point out that a reasonable approach is to use a second reviewer to check a random sample of at least 10% of the extractions for accuracy. Dual performance is more necessary for the extraction of quantitative results than for descriptive study information. In contrast, Cochrane group recommends that second reviewer should check the correctness and completeness of all data [ 11 ]. When possible, extractions should be limited to key characteristics and outcomes of the study. The same approach to data extraction is also suggested for a quality assessment process within rapid reviews [ 5 , 9 , 11 ]. Authors of the guidebook from McMaster University highlight that data extraction should be done ideally by two reviewers independently and consensus on the discrepancies should always be reached [ 7 ]. The final decision on the approach to this important step of review should depend on the available time and should also reflect the complexity of the research question [ 9 ].

For screening, analysis of full texts, extractions, and quality assessments, researchers can use information technologies to support them by making these review steps more efficient [ 5 ].

Before data reporting, a reviewer should prepare a document with key message headings, executive summary, background related to the topic and status of the current knowledge, project question, synthesis of findings, conclusions, and recommendations. According to the McMaster University guidebook, a report should be structured in a 1:2:20 format, that is, one page for key messages, two pages for an executive summary, and a full report of up to 20 pages [ 7 ]. All the limitations of the RLRs should be analyzed, and conclusions should be drawn with caution [ 5 ]. The quality of the accumulated evidence and the strength of recommendations can be assessed using, e.g., the GRADE system [ 5 ]. When working on references quoting, researchers should remember to use a primary source, not secondary references [ 7 ]. It would be worth considering the support of some software tools to automate reporting steps. Additionally, any standardization of the process and the usage of templates can support report development and enhance the transparency of the review [ 5 ].

Ideally, all the review steps should be completed during RLRs; however, often some steps may need skipping or will not be completed as thoroughly as should because of time constraints. It is always crucial to decide which steps may be skipped, and which are the key ones, depending on the project [ 7 ]. Guidelines suggest that it may be helpful to invite researchers with experience in the operations of SLRs to participate in the rapid review development [ 5 , 9 ]. As some of the steps will be completed by one reviewer only, it is important to provide them with relevant training at the beginning of the process, as well as during the review, to minimize the risk of mistakes [ 5 ].

Additional information

Depending on the policy goal and available resources and deadlines, methodology of the RLRs may be modified. Wilson et al. [ 10 ] provided extensive guidelines for performing RLR within days (e.g., to inform urgent internal policy discussions and/or management decisions), weeks (e.g., to inform public debates), or months (e.g., to inform policy development cycles that have a longer timeline, but that cannot wait for a traditional full systematic review). These approaches vary in terms of data synthesis, types of considered evidence and project management considerations.

In shortest timeframes, focused questions and subquestions should be formulated, typically to conduct a policy analysis; the report should consist of tables along with a brief narrative summary. Evidence from SLRs is often considered, as well as key informant interviews may be conducted to identify additional literature and insights about the topic, while primary studies and other types of evidence are not typically feasible due to time restrictions. The review would be best conducted with 1–2 reviewers sharing the work, enabling rapid iterations of the review. As for RLRs with longer timeline (weeks), these may use a mix of policy, systems and political analysis. Structure of the review would be similar to shorter RLRs – tabular with short narrative summary, as the timeline does not allow for comprehensive synthesis of data. Besides SLRs, primary studies and other evidence may be feasible in this timeframe, if obtained using the targeted searches in the most relevant databases. The review team should be larger, and standardized procedures for reviewing of the results and data extraction should be applied. In contrast to previous timeframe, merit review process may be feasible. For both timeframes, brief consultations with small transdisciplinary team should be conducted at the beginning and in the final stage of the review to discuss important matters.

For RLRs spanning several months, more comprehensive methodology may be adapted in terms of data synthesis and types of evidence. However, authors advise that review may be best conducted with a small review team in order to allow for more in-depth interpretation and iteration.

Studies analyzing methodology

There have been two interesting publications summarizing the results of Delphi consensus on the RLR methodology identified and included in this review [ 12 , 13 ].

Tricco et al. [ 13 ] first conducted an international survey and scoping review to collect information on the possible approaches to the running of rapid reviews, based on which, they employed a modified Delphi method that included inputs from 113 stakeholders to explore the most optimized approach. Among the six most frequent rapid review approaches (not all detailed here) being evaluated, the approach that combines inclusion of published literature only, a search of more than one database and limitations by date and language, study selection by one analyst, data extraction, and quality assessment by one analyst and one verifier, was perceived as the most feasible approach (72%, 81/113 responses) with the potentially lowest risk of bias (12%, 12/103). The approach ranked as the first one when considering timelines assumes updating of the search from a previously published review, no additional limits on search, studies selection and data extraction done by one reviewer, and no quality assessment. Finally, based on the publication, the most comprehensive RLRs can be made by moving on with the following rules: searching more than one database and grey literature and using date restriction, and assigning one reviewer working on screening, data extraction, and risk of bias assessment ( Table 1 ). Pandor et al. [ 12 ] introduced a decision tool for SelecTing Approaches for Rapid Reviews (STARR) that were produced through the Delphi consensus of international experts through an iterative and rigorous process. Participants were asked to assess the importance of predefined items in four domains related to the rapid review process: interaction with commissioners, understanding the evidence base, data extraction and synthesis methods, and reporting of rapid review methods. All items assigned to four domains achieved > 70% of consensus, and in that way, the first consensus-driven tool has been created that supports authors of RLRs in planning and deciding on approaches.

Six most frequent approaches to RLRs (adapted from Tricco et al. [ 13 ]).

Haby et al. [ 14 ] run searches of 11 databases and two websites and developed a comprehensive overview of the methodology of RLRs. With five SLRs and one RCT being finally included, they identified the following approaches used in RLRs to make them faster than full SLRs: limiting the number and scope of questions, searching fewer databases, limited searching of grey literature, restrictions on language and date (e.g., English only, most recent publications), updating the existing SLRs, eliminating or limiting hand searches of reference lists, noniterative search strategies, eliminating consultation with experts, limiting dual study selection, data extraction and quality assessment, minimal data synthesis with short concise conclusions or recommendations. All the SLRs included in this review were consistent in stating that no agreed definition of rapid reviews is available, and there is still no final agreement on the best methodological rules to be followed.

Gordon et al. [ 4 ] explained the advantages of performing a focused review and provided 12 tips for its conduction. They define focused reviews as ‘a form of knowledge synthesis in which the components of the systematic process are applied to facilitate the analysis of a focused research question’. The first tip presented by the authors is related to deciding if a focused review is a right solution for the considered project. RLRs will suit emerging topics, approaches, or assessments where early synthesis can support doctors, policymakers, etc., but also can direct future research. The second, third, and fourth tips highlight the importance of running preliminary searches and considering narrowing the results by using reasonable constraints taking into account the local context, problems, efficiency perspectives, and available time. Further tips include creating a team of experienced reviewers working on the RLRs, thinking about the target journal from the beginning of work on the rapid review, registering the search protocol on the PROSPERO registry, and the need for contacting authors of papers when data available in publications are missing or incongruent. The last three tips are related to the choice of evidence synthesis method, using the visual presentation of data, and considering and describing all the limitations of the focused review.

Finally, a new publication by Tricco et al. from 2022, describing JBI position statement [ 15 ] underlined that for the time being, there is no specific tool for critical appraisal of the RLR’s methodological quality. Instead, reviewers may use available tools to assess the risk of bias or quality of SLRs, like ROBIS, the JBI critical appraisal tools, or the assessment of multiple systematic reviews (AMSTAR).

Inconsistency in the definitions and methodologies of RLR

Although RLR was broadly perceived as an approach to quicken the conduct of conventional SLR, there is a lack of consensus on the formal definition of the RLR, so as to the best approaches to perform it. Only in 2021, a study proposing unified definition was published; however, it is important to note that the most accurate definition was only matching slightly over 50% of papers analysed by the authors, which underlines the lack of homogeneity in the field [ 11 ]. The evidence-based supporting methods are evolving, and more evidence is needed to define the most robust approaches [ 5 ].

Diverse terms are used to describe the RLR, including ‘rapid review’, focused systematic review’, ‘quick scoping reviews’, and ‘rapid evidence assessments’. Although the general principles of conducting RLR are to accelerate the whole process, complexity was seen in the methodologies used for RLRs, as reflected in this study. Also, inconsistencies related to the scope of the questions, search strategies, inclusion criteria, study screening, full-text review, quality assessment, and evidence presentation were implied. All these factors may hamper decision-making about optimal methodologies for conducting rapid reviews, and as a result, the efficiency of RLR might be decreased. Additionally, researchers may tend to report the methodology of their reviews without a sufficient level of detail, making it difficult to appraise the quality and robustness of their work.

Advantages and weaknesses of RLR

Although RLR used simplified approaches for evidence synthesis compared with SLR, the methodologies for RLR should be replicable, rigorous, and transparent to the greatest extent [ 16 ]. When time and resources are limited, RLR could be a practical and efficient tool to provide the summary of evidence that is critical for making rapid clinical or policy-related decisions [ 5 ]. Focusing on specific questions that are of controversy or special interest could be powerful in reaffirming whether the existing recommendation statements are still appropriate [ 17 ].

The weakness of RLR should also be borne in mind, and the trade-off of using RLR should be carefully considered regarding the thoroughness of the search, breadth of a research question, and depth of analysis [ 18 ]. If allowed, SLR is preferred over RLR considering that some relevant studies might be omitted with narrowed search strategies and simplified screening process [ 14 ]. Additionally, omitting the quality assessment of included studies could result in an increased risk of bias, making the comprehensiveness of RLR compromised [ 13 ]. Furthermore, in situations that require high accuracy, for example, where a small relative difference in an intervention has great impacts, for the purpose of drafting clinical guidelines, or making licensing decisions, a comprehensive SLR may remain the priority [ 19 ]. Therefore, clear communications with policymakers are recommended to reach an agreement on whether an RLR is justified and whether the methodologies of RLR are acceptable to address the unanswered questions [ 18 ].

Disclosure statement

No potential conflict of interest was reported by the author(s).

  • Open access
  • Published: 28 May 2024

Composite outcome measures in high-impact critical care randomised controlled trials: a systematic review

  • Humphrey G. M. Walker 1 , 2   na1 ,
  • Alastair J. Brown 1 , 2 , 3 , 4   na1 ,
  • Ines P. Vaz 2 ,
  • Rebecca Reed 2 ,
  • Max A. Schofield 2 ,
  • Jeffrey Shao 5 ,
  • Vinodh B. Nanjayya 2 , 3 ,
  • Andrew A. Udy 2 , 3 &
  • Toby Jeffcote 2 , 3  

Critical Care volume  28 , Article number:  184 ( 2024 ) Cite this article

Metrics details

The use of composite outcome measures (COM) in clinical trials is increasing. Whilst their use is associated with benefits, several limitations have been highlighted and there is limited literature exploring their use within critical care. The primary aim of this study was to evaluate the use of COM in high-impact critical care trials, and compare study parameters (including sample size, statistical significance, and consistency of effect estimates) in trials using composite versus non-composite outcomes.

A systematic review of 16 high-impact journals was conducted. Randomised controlled trials published between 2012 and 2022 reporting a patient important outcome and involving critical care patients, were included.

8271 trials were screened, and 194 included. 39.1% of all trials used a COM and this increased over time. Of those using a COM, only 52.6% explicitly described the outcome as composite. The median number of components was 2 (IQR 2–3). Trials using a COM recruited fewer participants (409 (198.8–851.5) vs 584 (300–1566, p  = 0.004), and their use was not associated with increased rates of statistical significance (19.7% vs 17.8%, p  = 0.380). Predicted effect sizes were overestimated in all but 6 trials. For studies using a COM the effect estimates were consistent across all components in 43.4% of trials. 93% of COM included components that were not patient important.

Conclusions

COM are increasingly used in critical care trials; however effect estimates are frequently inconsistent across COM components confounding outcome interpretations. The use of COM was associated with smaller sample sizes, and no increased likelihood of statistically significant results. Many of the limitations inherent to the use of COM are relevant to critical care research.

Randomised controlled trials (RCT) are the gold standard by which clinicians assess current and emerging treatments in critical care medicine [ 1 ]. Unfortunately, RCTs are associated with significant financial and opportunity costs [ 2 ] and frequently do not identify statistically significant differences in outcomes for a given treatment strategy [ 3 ]. These issues are particularly marked when mortality is used as the primary outcome measure [ 4 , 5 , 6 ].

The selection of appropriate outcomes for critical care trials is important if they are to effectively guide clinical decision making. Trial outcomes need to be meaningful to patients, enable efficient trial design and minimise the risk of missing important treatment effects.

Composite outcome measures (COM) combine multiple clinical events (≥ 2 component outcomes) into a single outcome [ 7 ]. They are designed to capture a greater number of outcome events, and thus increase a trial’s ability to demonstrate a statistically significant treatment effect [ 8 ]. Other proposed methodological benefits include reduced sample size requirements, avoidance of the need to choose a single primary outcome, reduced use of multiple statistical comparisons [ 9 , 10 ] and to account for competing risks [ 11 ]. However, significant limitations have been identified regarding the implementation [ 11 , 12 ] and interpretation [ 8 , 10 ] of COM. A major concern is a lack of consistency of the individual component event rates, resulting in the overall event rate being driven by individual components of the COM. This imbalance risks the creation of a “misleading impression of the impact of treatment” [ 13 , 14 ]. Additionally, the outcome component driving the overall event rate may not be patient important [ 15 ].

COM are increasingly used in the design and implementation of clinical trials, and their use has been evaluated in several medical specialities [ 16 , 17 ]. Within critical care, COM, such as ventilator free days (VFD), are widely used. There is also a trend towards the analysis [ 18 ] or reanalysis [ 19 , 20 ] of trial data using more complex COM. However, their use is occurring in the setting of a paucity of literature evaluating the validity of COM use in critical care research.

Given the increasing utilisation of COM, the trend towards increasingly complex COM and the implications for trial validity, we performed a systematic review to summarise the use and reporting of COM in high-impact critical care RCTs. We focused specifically on COM that include patient important outcomes as there is increasing recognition that trial outcomes should be relevant to patients [ 21 ]. Our primary aim was to quantify COM use in this population, and compare study parameters (including sample size, statistical significance, and consistency of effect estimates) between trials using composite and non-composite outcomes.

Methods for inclusion and analysis of studies in this systematic review were pre-specified in a protocol developed in accordance with the most recent Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) [ 22 ] guidelines. This protocol was prospectively registered with PROSPERO (CRD42022380606). No funding for this study was obtained.

Search strategy

The full search strategy is described in the online data supplement and used the following databases: Ovid MEDLINE, PubMed and the Cochrane Library. RCTs published from 1st August 2012 until 31st December 2022 were included. The reference sections of selected trials from preliminary searches were reviewed to guide the refinement of search strategies.

Selection criteria

The titles and abstracts were independently screened by two of three reviewers (HW, AB, IV) and scored against prespecified eligibility criteria. Disagreements were resolved by a third senior reviewer (TJ). All trials deemed eligible underwent full text review prior to data extraction to ensure inclusion criteria were met .

We included RCTs that were conducted in the critical care population, published in the English language in sixteen high impact factor journals. Journal selection was based on a previous review by Harhay et al. [ 5 ]. The primary outcome or at least one component of the COM had to be a patient important outcome as defined by Gaudry et al. [ 23 ]: “mortality at any time and/or quality of life, functional/cognitive/neurological outcomes assessed after ICU discharge”.

Exclusion criteria included non randomised controlled trials, involving those < 18 years old or any trial that did not include a patient important outcome. The full inclusion and exclusion criteria, alongside the journals selected are published in the online data supplement (Table S1 ).

Data collection and risk of bias

All selected trials were independently reviewed by two of six reviewers (HW, AB, IV, MS, JS, RR). Risk of bias analysis was performed using The Cochrane Risk of Bias 2 (RoB2) tool [ 24 ]. Data were extracted and recorded using standardised data forms on a web-based collaboration software platform (Covidence) [ 25 ]. The data forms were designed prior to the commencement of data collection. Discrepancies in risk of bias score, methodological classification or other data points were identified by the Covidence system and were resolved in consultation with a third senior reviewer (TJ, AU, VN). No missing data were imputed.

Definitions of outcomes

Outcome measures were categorised as follows:

Categorical composites in which patients were defined as having the primary outcome if they met pre-defined criteria for one or more of the discrete components.

Continuous composites included any outcome that measured duration of organ support or disease process (e.g. VFD).

Non-COM studies consisted of trials using:

A single binary outcome e.g. mortality

Other included those single outcomes that did not fit into the above categories. They principally measured patient performance within a single domain (such as Modified Rankin Scale or Glasgow Outcome Score as a measure of functional status)

Outcomes and data synthesis

Each trial was primarily categorised as a COM or non-COM trial. COM trials were then sub-categorised according to the classification above. When classifying components of a COM, all potential possibilities of meeting the outcome were included as individual components. For example, the COM “Days Alive and Free of Life Support” included 4 potential outcomes: mortality, duration of mechanical ventilation, circulatory support, and renal replacement therapy respectively. Trials with co-primary outcomes were included. If a co-primary outcome consisted of both a non-COM and a COM e.g. Kawazoe et al. [ 26 ], then it was classified as a COM. Trials were also included if utilising both patient and non-patient important co-primary outcomes e.g. Zarbock et al. [ 27 ].

For comparison of achieved effect sizes, the difference between predicted and observed effect size was calculated. Predicted relative effect estimates were calculated to compare the predicted magnitude of effect used for power calculations. Methods for calculating the event rate gap are described in the online data supplement. Trials with more than two arms, 2 × 2 factorial designs or studies with co-primary outcomes were not included in event rate and effect size analyses. Data for comparison of event rates or effect sizes, when presented as a range of values or without reference to an absolute risk difference were excluded.

Statistical analysis

Continuous data are presented as median (interquartile range) and mean (standard deviation) where appropriate. Categorical data are presented as frequency (%). The analysis was primarily descriptive, albeit where comparisons were made, the Mann–Whitney U Test and Fisher’s Exact test have been used, as indicated. The analysis of COM use over time was conducted using the Mann Kendall trend test. To calculate the predicted relative effect estimate, the predicted effect sizes were converted into relative risk estimates by dividing the predicted absolute risk reduction by the expected event rate in the control group. The same principle was used for continuous outcomes (e.g. if an effect size of 2 VFD was predicted with a control rate of 8, then this was a 25% relative difference). These have been separated due to the differences in data and scale used (e.g. VFD and % reduction in mortality).

We conducted a sensitivity analysis in which we included ordinal outcome scales such as mRS (modified Rankin Scale) as COM, and also removed them from the analysis all together.

Significance has been assumed with a p -value < 0.05 and no adjustment for multiplicity was made. All statistical analyses were conducted using GraphPad Prism 10.0.3 [ 28 ].

12,270 trials were identified following the database search and after removal of duplicates, 8271 records underwent title and abstract screening. 194 trials were included in the final analysis. This is shown in Fig.  1 . A reference list is included in the online data supplement.

figure 1

Showing PRISMA flow diagram of trial selection

Trial characteristics

Of the 194 trials included in the analysis, 76 (39.1%) RCTs were classified as using a COM for the primary outcome. For trials using a non-COM, 101 assessed mortality (or survival) at various time points as a primary outcome. The basic characteristics of the included trials are outlined in Table  1 . Figures S1 and S2 in the online data supplement show all populations and interventions studied in the included trials.

COM versus non-COM

The use of COMs in high-impact critical care RCTs has significantly increased since 2012 ( p  = 0.002 for trend over time, Fig.  2 ). Comparing RCTs using a COM vs non-COM respectively, significantly fewer patients were enrolled [409 (198.8–851.5) vs 584 (300–1566) patients, p  = 0.004] and the median number of sites was lower [12 (2–35) vs 31 (11.5–50.5) sites, p  = 0.009]. The median reported power for COM and non-COM trials were similar at 80 (IQR 80–85) % and 80 (IQR 80–90) % respectively. This is shown in Table  2 (and Fig. S3 in the online data supplement).

figure 2

Showing the trend of COM use as a proportion of all trials included in the systematic review

Of note, a significant proportion of trials using a COM (n = 17, 22.4%) investigated COVID-19. Fewer (n = 7, 5.9%) non-COM trials investigated this disease. 40/76 (52.6%) of those using a COM were explicitly described as a COM in the manuscript. 49 (64.5%) had all components reported in the table of results. There was no difference in Risk of Bias assessment scores between COM and non-COM trials (see Fig. S4 in the online data supplement).

The median number of components was 2 (2–3), and 5 (6.6%) trials utilised a COM composed entirely of patient important components. All of the COM included mortality as a component of the composite outcome. Other component types utilised are shown in Fig.  3 . A comprehensive list of all the components utilised in the included trials is provide in the online data supplement (Table S2 ).

figure 3

Showing a heatmap of all the components used in COMs within this systematic review. a Circulatory support includes extracorporeal membrane oxygenation (ECMO). b Includes outcomes assessing both hospital and ICU length of stay. c Operative specific components include outcomes such as re-operation. d Disease specific components includes development of specific diseases or syndromes. For example, myocardial infarction (MI), acute kidney injury (AKI), Acute Respiratory Distress Syndrome (ARDS) or venous thromboembolic event (VTE)

Categorical versus continuous COM

31 (40.8%) of the COM were categorical, with the remaining 45 (59.2%) being continuous. Of the continuous COM, 36 (80.0%) were organ support free days (OSFD). Of these, 7 (15.6%) scored death as -1. Identification within the trial manuscript that a COM was being utilised occurred less frequently in the continuous COM group (83.9% vs 31.1%, p  < 0.001). This is shown in Table  2 .

Consistency of effect estimates and effect sizes

In trials using a COM, it was only possible to assess whether point effect estimates were consistent in 58 (76.3%) of trials with the information provided. Point effect estimates were consistent (e.g. all in favour or not in favour of intervention) across all components in 33 (56.9%) of trials. In the remaining trials, point estimates were not consistent (n = 16 categorical and n = 9 continuous). There was no apparent difference between trials using a categorical or continuous COM [15/31 (48.4%) vs 18/45 (66.7%), p  = 0.192]. Mortality was the component with the highest event rate in 15/31 (48.4%) of trials using a categorical COM.

The predicted relative effect estimates are shown in Fig.  4 . For trials using a categorical COM this was 35.4 (IQR 24.8–50.0), and for trials using a continuous COM this was 20.2 (IQR 16.7–36.7). For non-COM trials that assessed mortality this was 25.0 (IQR 19.6–32.7) and for other non-COM trials this was 38.9 (IQR 22.5–56.8). Predicted relative effect estimates were significantly greater for categorical COM compared to non-COM trials assessing mortality ( p  < 0.001). Most trials had predicted effect sizes greater than the actual effect size, with only 6 trials having an effect size larger than predicted . This is shown in Fig. S5 in the online data supplement. The event rate gap was similar between COM and Non-COM trials (Table S3 and Fig. S6 in the online data supplement).

figure 4

Showing the predicted relative effect estimates for COM and Non-COM studies. Error bars are median and IQR

Reported statistical significance

There was no difference in the frequency of reported statistical significance between COM and non-COM trials [17/56 (19.7%) vs 21/118 (17.8%), p  = 0.85]. Rates of statistical significance did not vary between categorical COM and continuous COM [8/31 (25.8%) vs 7/45 (15.6%), p  = 0.38].

Sensitivity analyses

19 trials used ordinal outcomes. Results in all sensitivity analyses were unchanged with respect to number of sites, number of participants, trial power and rates of statistical significance (Tables S5 –S8 in the online data supplement).

We performed a systematic review of critical care trials published in high impact journals with a specific focus on COM. We assessed the frequency of use and reporting of COM in the relevant studies. We documented the different forms of COM, the consistency of treatment effect between individual components of the composites and the proportion of patient centred outcomes within these composites. Lastly, we compared the accuracy of effect size calculations and frequency of statistically significant results between COM and non-COM studies.

The recently published CONSORT-Outcomes 2022 extension [ 7 ] calls for all components of a COM to be reported in order to clarify the interpretation of trial findings. Only 65% of trials identified in our systematic review had complete reporting of all individual components in the manuscript. Additionally, only 50% of the trials explicitly acknowledged the use of a COM in the published manuscript. There was a notable lack of acknowledgement of the composite nature of continuous COM such as organ support free days, a finding that is consistent with previous literature [ 29 ]. The median number of components for all trials with a COM was 2 (IQR 2–3) and the maximum number of components was 21 [ 30 ] . This is important as increasing components will complicate the interpretation of which components are driving any treatment effect identified by clinical trials. The number of trials using a COM composed entirely of patient important outcomes was low (7%). Treatment effects were therefore partly determined by non-patient important outcomes (e.g. asymptomatic DVT, acute kidney injury, worsening oxygenation, radiographic progression) in 93% of trials. While we recognise the clinical validity of a range of non-patient important outcomes, we would argue that these should be analysed in a way that accounts for the varying importance of specific components within a COM. One analytical method that allows for the prioritisation of outcomes within a single COM may be the Win Ratio [ 31 ] which has been used recently in the re-analysis of several large critical care trials and may mitigate the challenge of the competing risk of death and other components of composite outcomes [ 19 , 20 ].

Despite the well documented challenges associated with their use [ 29 , 32 ] continuous COM, and particularly OSFDs, were used widely (60% and 50% respectively). One specific issue with these forms of outcome measure is inconsistency of mortality scoring within the continuous COM. Our review revealed that both mortality and ongoing dependence on organ support were scored as zero in approximately 80% of studies, whereas in a further 20%, death was scored as minus one and ongoing dependence on support as zero. This heterogeneity has been previously documented [ 33 ] and is a significant barrier to study comparisons and meta-analyses. A widely accepted standard for the calculation of failure free days has not yet been established but could reduce methodological heterogeneity [ 34 ].

48% of categorical COM demonstrated consistency of treatment effect across COM components. Given the complexity of physiological derangements in critically ill patients this finding is not unexpected. There is, however, an argument for trialists to demonstrate biological plausibility for the components of a COM. Further, the explicit recognition of trial outcomes that are based on conflicting treatment effects or entirely driven by a single component of a COM should be encouraged.

Potential benefits of COM include increased power and/or reductions in sample size requirements [ 8 , 12 ]. These may be particularly valuable in critical care trials where overestimation of effect size has been consistently demonstrated [ 5 , 35 ]. However, our review revealed that only 1 trial utilising a COM had an effect size greater than predicted [ 36 ]. Greater predicted relative effect estimates for categorical COM in comparison to non-COM trials with mortality as the primary outcome was shown, and median study power for both COM and non-COM studies were similar, implying that in critical care trials COM are employed primarily to reduce sample size requirements rather than increase statistical power. In some settings (such as the recent COVID-19 pandemic) smaller sample sizes may facilitate rapid assessment of interventions. Indeed, our findings indicate increased use of continuous COM during the COVID-19 pandemic. We would caution however, that the use of COM to reduce sample size necessitates increased scrutiny of the components of the COM, the heterogeneity of component treatment effects and the degree of patient centredness. We would also argue for pre-trial documentation of COM components, expected treatment effects and the specification of what the investigators consider a minimum clinically important difference (MCID) [ 37 ] to determine if a particular result represents a genuine treatment effect or a type 2 error.

The use of COM has been associated with a trend towards increased rates of statistical significance in other specialties such as cardiology [ 38 ]. In our review there was no difference in rates of statistical significance between COM and non-COM trials. Additionally, the rate of statistical significance in our review (18.5%), is lower than previously reported for critical care trials [ 3 , 5 ]. It is noteworthy though, that previous reviews were not restricted to patient important outcomes. The observation that rates of statistical significance for clinical trials utilising COM differ between different medical specialties may reflect differences in the disease states investigated. A significant sub-set of the diseases investigated in critical care trials e.g. traumatic brain injury, acute respiratory distress syndrome, can be characterised as physiological syndromes rather than diseases with a clear pathophysiology such as acute myocardial ischaemia. The heterogeneity of critical care syndromes is increasingly recognised as a confounding factor in critical care trials [ 39 ]. A COM which reduces sample size requirements for a predicted treatment effect but fails to capture this heterogeneity will not lead to increased rates of statistically significant trials. While careful patient selection and cohort enrichment are key factors in improving trial efficiency in critical care, the flexibility of COM represents an opportunity for significantly improved critical care trial design. A thoughtfully designed COM that can represent the heterogenous nature of a critical care syndrome may be able to successfully demonstrate treatment effects with smaller sample sizes and the use of fewer resources. Recommendations based on the findings from this systematic review regarding the use of COM in critical care trials are made in Table  3 .

Strengths and limitations

To our knowledge this is the first systematic review of COM use in the critical care RCTs. We have utilised a robust and reproducible search strategy and a study selection methodology based on prior literature designed to assess the outcomes and characteristics of ICU based RCTs [ 5 ]. We have sought to distinguish between continuous and categorical COM, document the reporting of COM, assess the validity of sample size calculation and the frequency of statistically significant results. Given their increasing use, the lack of literature exploring their characteristics and the advent of more complex COM we submit that this is a timely and useful review of the literature.

Limitations of the review include the focus on 16 high impact critical care journals. There will undoubtedly be critical care trials using patient focused COM that are not captured by this review. However, we feel that practice change is predominantly based on larger trials published in high impact journals so a focus on this category of evidence is justified. Our review also omits studies utilising COM without any patient centred components. We feel that our use of a recognised definition of patient centred outcomes [ 23 ] and the increasing importance of this concept [ 40 ] adds weight to the validity of this approach. The omission of ordinal scales such as the modified Rankin scale or Glasgow outcome scale from our definition of COM could be considered inconsistent with our inclusion of continuous COM such as ventilator free days. We acknowledge the challenges and nuances associated with classification of these outcomes. Our decision to classify them as non-COM for our primary statistical analyses results from finding no examples of these scales being described as COM within the literature, which contrasts to OSFDs [ 29 ]. Additionally, it could be suggested that all levels of ordinal scales assessing neurological outcome fall within the single domain of function [ 41 ]. In recognition of the ambiguity surrounding their classification we performed multiple sensitivity analyses. The results of these analyses do not materially change our conclusions. Additionally, the sensitivity analyses of these scales reveal heterogeneity in terms of their application and analysis, a finding that has previously been described [ 42 , 43 ].

In high-impact critical care RCTs assessing patient important outcomes, COM are used frequently. The use of COM has increased over time. The inherent limitations of COM identified in trials from related medical specialties, are also relevant to critical care. Many trials using a COM did not demonstrate consistency across all components of the COM and 93% of COM included non patient-important components. The primary methodological benefit of a COM is a reduction in sample size. However, their use does not lead to increased rates of statistically significant results in critical care trials and predicted effect sizes remain grossly over optimistic. Further work to improve the use of COM in critical care should focus on the design and validation of COM that include patient important outcomes and effectively represent the heterogeneity of the pathologies studied in the critical care literature.

Availability of data and materials

The datasets used and analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Randomised controlled trial

Composite outcome measure

Non-composite outcome measure

Ventilator free days

Extracorporeal membrane oxygenation

Myocardial infarction

Acute kidney injury

Acute respiratory distress syndrome

Venous thromboembolic event

Organ support free days

Modified rankin scale

Glasgow outcome score extended

Granholm A, Alhazzani W, Derde LPG, Angus DC, Zampieri FG, Hammond NE, et al. Randomised clinical trials in critical care: past, present and future. Intensive Care Med. 2022;48:164–78.

Article   CAS   PubMed   Google Scholar  

Speich B, von Niederhäusern B, Schur N, Hemkens LG, Fürst T, Bhatnagar N, et al. Systematic review on costs and resource use of randomized clinical trials shows a lack of transparent and comprehensive data. J Clin Epidemiol. 2018;96:1–11.

Article   PubMed   Google Scholar  

Kampman JM, Sperna Weiland NH, Hermanides J, Hollmann MW, Repping S, et al. Randomized controlled trials in ICU in the four highest-impact general medicine journals. Crit Care Med. 2023;51:e179–83.

Article   PubMed   PubMed Central   Google Scholar  

Veldhoen RA, Howes D, Maslove DM. Is mortality a useful primary end point for critical care trials? Chest. 2020;158:206–11.

Harhay M, Wagner J, Ratcliffe SJ, Bronheim RS, Gopal A, Green S, et al. Outcomes and statistical power in adult critical care randomized trials. Am J Respir Crit Care Med. 2014;189:1469–78.

Santacruz CA, Pereira AJ, Celis E, Vincent JL. Which multicenter randomized controlled trials in critical care medicine have shown reduced mortality? A systematic review. Crit Care Med. 2019;47:1680–91.

Butcher NJ, Monsour A, Mew EJ, Chan A-W, Moher D, Mayo-Wilson E, et al. Guidelines for reporting outcomes in trial protocols. JAMA. 2022;328:2345–56.

Palileo-Villanueva LM, Dans AL. Composite endpoints. J Clin Epidemiol. 2020;128:157–8.

Diaz-Quijano FA. Estimating and testing an index of bias attributable to composite outcomes in comparative studies. J Clin Epidemiol. 2021;132:1–9.

Dash K, Goodacre S, Sutton L. Composite outcomes in clinical prediction modeling: are we trying to predict apples and oranges? Ann Emerg Med. 2022;80:12–9.

Manja V, Al Bashir S, Guyatt G. Criteria for use of composite end points for competing risks—a systematic survey of the literature with recommendations. J Clin Epidemiol. 2017;82:4–11.

Choi SW, Cheung CW. The case of the misleading composite: one outcome is better than two. Anaesthesia. 2016;71:1101–3.

Myles PS, Devereaux PJ. Pros and cons of composite endpoints in anesthesia trials. Anesthesiology. 2010;113:776–8.

Irony TZ. The “utility” in composite outcome measures: measuring what is important to patients. JAMA. 2017;318:1820–1.

Ferreira-González I, Busse JW, Heels-Ansdell D, Montori VM, Akl EA, Bryant DM, et al. Problems with use of composite end points in cardiovascular trials: systematic review of randomised controlled trials. BMJ. 2007;334:786.

Herman D, Lor KY, Qadree A, Horn D, D’Souza R. Composite adverse outcomes in obstetric studies: a systematic review. BMC Pregnancy Childb. 2021;21:107.

Article   Google Scholar  

Lai NM, Yap AQY, Ong HC, Wai SX, Yeo JHH, Koo CYE, et al. Use of composite outcomes in neonatal trials: an analysis of the cochrane reviews. Neonatology. 2021;118:259–63.

Freund Y, Viglino D, Cachanado M, Cassard C, Montassier E, Douay B, et al. Effect of Noninvasive airway management of comatose patients with acute poisoning: a randomized clinical trial. JAMA. 2023;330:2267–74.

Zampieri FG, Damiani LP, Biondi RS, Freitas FGR, Veiga VC, Figueiredo RC, et al. Hierarchical endpoint analysis using win ratio in critical care: an exploration using the balanced solutions in intensive care study (BaSICS). J Crit Care. 2022;71:154113.

Zampieri FG, Serpa-Neto A, Wald R, Bellomo R, Bagshaw SM. Hierarchical endpoints in critical care: A post-hoc exploratory analysis of the standard versus accelerated initiation of renal-replacement therapy in acute kidney injury and the intensity of continuous renal-replacement therapy in critically ill patients trials. J Crit Care. 2024;82:154767.

Iwashyna TJ, McPeake J. Choosing outcomes for clinical trials: a pragmatic perspective. Curr Opin Crit Care. 2018;24:428–33.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.

Gaudry S, Messika J, Ricard JD, Guillo S, Pasquet B, Dubief E, et al. Patient-important outcomes in randomized controlled trials in critically ill patients: a systematic review. Ann Intensive Care. 2017;7:28.

Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898.

Covidence Systematic Review Software (2023). Melbourne, Australia: Veritas Health Innovation at https://www.covidence.org

Kawazoe Y, Miyamoto K, Morimoto T, Yamamoto T, Fuke A, Hashimoto A, et al. Effect of dexmedetomidine on mortality and ventilator-free days in patients requiring mechanical ventilation with sepsis: a randomized clinical trial. JAMA. 2017;317:1321–8.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Zarbock A, Küllmar M, Kindgen-Milles D, Wempe C, Gerss J, Brandenburger T, et al. Effect of regional citrate anticoagulation vs systemic heparin anticoagulation during continuous kidney replacement therapy on dialysis filter life span and mortality among critically Ill patients with acute kidney injury: a randomized clinical trial. JAMA. 2020;324:1629–39.

GraphPad Prism version 10.0.3 for Windows (2023). Boston, Massachusetts, USA: Graph Pad Software at www.graphpad.com

Yehya N, Harhay MO, Curley MAQ, Schoenfeld DA, Reeder RW. Reappraisal of ventilator-free days in critical care research. Am J Resp Crit Care. 2019;200:828–36.

Chesnut RM, Temkin N, Carney N, Dikmen S, Rondina C, et al. A trial of intracranial-pressure monitoring in traumatic brain injury. N Engl J Med. 2012;367:2471–81.

Pocock SJ, Ariti CA, Collier TJ, Wang D. The win ratio: a new approach to the analysis of composite endpoints in clinical trials based on clinical priorities. Eur Heart J. 2012;33:176–82.

Verghis RM, McDowell C, Blackwood B, Lee B, McAuley DF, Clarke M. Re-analysis of ventilator-free days (VFD) in acute respiratory distress syndrome (ARDS) studies. Trials. 2023;24:183.

Contentin L, Ehrmann S, Giraudeau B. Heterogeneity in the definition of mechanical ventilation duration and ventilator-free days. Am J Respir Crit Care Med. 2014;189:998–1002.

Triché LR, Futier E, Carvalho MD, Piñol-Domenech N, Bodet-Contentin L, Jabaudon M, et al. Sample size estimation in clinical trials using ventilator-free days as the primary outcome: a systematic review. Crit Care. 2023;27:303.

Abrams D, Montesi SB, Moore SKL, Manson DK, Klipper KM, Case MA, et al. Powering bias and clinically important treatment effects in randomized trials of critical illness. Crit Care Med. 2020;48:1710–9.

Zhou Y, Jin X, Lv Y, Wang P, Yang Y, et al. Early application of airway pressure release ventilation may reduce the duration of mechanical ventilation in acute respiratory distress syndrome. Intens Care Med. 2017;43:1648–59.

Draak THP, de Greef BTA, Faber CG, Merkies ISJ. The minimum clinically important difference: which direction to take. Eur J Neurol. 2019;26:850–5.

Tan NS, Ali SH, Lebovic G, Mamdani M, Laupacis A, Yan AT. Temporal trends in use of composite end points in major cardiovascular randomized clinical trials in prominent medical journals. Circ Cardiovasc Qual Outcomes. 2018;10:e003753.

Laffey JG, Kavanagh BP. Negative trials in critical care: why most research is probably wrong. Lancet Respir Med. 2018;6:659–60.

Dinglas VD, Faraone LN, Needham DM. Understanding patient-important outcomes after critical illness. Curr Opin Crit Care. 2018;24:401–9.

Broderick JP, Adeoye O, Elm J. Evolution of the modified rankin scale and its use in future stroke trials. Stroke. 2017;48:2007–12.

Saver JL, Chaisinanunkul N, Campbell BCV, Grotta JC, Hill MD, Khatri P, et al. Standardized nomenclature for modified rankin scale global disability outcomes: consensus recommendations from stroke therapy academic industry roundtable XI. Stroke. 2021;52:3054–62.

Selman CJ, Lee KJ, Ferguson KN, et al. Statistical analyses of ordinal outcomes in randomised controlled trials: a scoping review. Trials. 2024;25:241.

Download references

Acknowledgements

Not applicable.

No funding was obtained for this study.

Author information

Humphrey G. M. Walker and Alastair J. Brown are joint first authors.

Authors and Affiliations

Department of Critical Care, St Vincent’s Hospital, Melbourne, VIC, Australia

Humphrey G. M. Walker & Alastair J. Brown

Department of Intensive Care and Hyperbaric Medicine, The Alfred Hospital, Melbourne, Australia

Humphrey G. M. Walker, Alastair J. Brown, Ines P. Vaz, Rebecca Reed, Max A. Schofield, Vinodh B. Nanjayya, Andrew A. Udy & Toby Jeffcote

Australian and New Zealand Intensive Care Research Centre, School of Public Health and Preventive Medicine, Monash University, Prahran, VIC, Australia

Alastair J. Brown, Vinodh B. Nanjayya, Andrew A. Udy & Toby Jeffcote

Department of Critical Care, University of Melbourne, Melbourne, VIC, Australia

Alastair J. Brown

Monash University, Melbourne, VIC, Australia

Jeffrey Shao

You can also search for this author in PubMed   Google Scholar

Contributions

H.W., T.J., V.N., A.U. and A.B. designed the study. H.W., I.V., M.S., J.S., R.R., and A.B. screened trials, reviewed trial abstracts, and collected data. H.W., A.B., and T.J. drafted the initial manuscript. H.W. produced all the figures. H.W. and V.N. conducted the statistical analysis. A.B., A.U., V.N., and T.J. provided senior support. All authors reviewed and approved the manuscript.

Corresponding author

Correspondence to Humphrey G. M. Walker .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

No potential conflicts of interest relevant to this article were reported however AU reports grants (paid to Monash University) from NHMRC Australia, MRFF Australia and the Department of Health, Commonwealth of Australia that are unrelated to this work and the receipt of in-kind support (trial consumables) from Integra Lifesciences for a project outside of the current work.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Walker, H.G.M., Brown, A.J., Vaz, I.P. et al. Composite outcome measures in high-impact critical care randomised controlled trials: a systematic review. Crit Care 28 , 184 (2024). https://doi.org/10.1186/s13054-024-04967-3

Download citation

Received : 08 April 2024

Accepted : 22 May 2024

Published : 28 May 2024

DOI : https://doi.org/10.1186/s13054-024-04967-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Critical care outcomes
  • Outcome assessment
  • Critical care
  • Randomized controlled trials as topic

Critical Care

ISSN: 1364-8535

four components of a literature review

Log in using your username and password

  • Search More Search for this keyword Advanced search
  • Latest content
  • For authors
  • Browse by collection
  • BMJ Journals More You are viewing from: Google Indexer

You are here

  • Volume 14, Issue 5
  • Effectiveness of educational and psychological survivorship interventions to improve health-related quality of life outcomes for men with prostate cancer on androgen deprivation therapy: a systematic review
  • Article Text
  • Article info
  • Citation Tools
  • Rapid Responses
  • Article metrics

Download PDF

  • http://orcid.org/0000-0001-9268-5331 Sally AM Sara 1 ,
  • http://orcid.org/0000-0001-8102-1871 Nicole Heneka 2 ,
  • http://orcid.org/0000-0002-7541-3665 Anna Green 2 ,
  • http://orcid.org/0000-0003-2369-6111 Suzanne K Chambers 2 , 3 ,
  • http://orcid.org/0000-0002-1180-3381 Jeff Dunn 2 , 4 ,
  • http://orcid.org/0000-0003-4154-8526 Victoria R Terry 1
  • 1 University of Southern Queensland , Toowoomba , Queensland , Australia
  • 2 University of Southern Queensland , Springfleld , Queensland , Australia
  • 3 Australian Catholic University , Brisbane , Queensland , Australia
  • 4 Prostate Cancer Foundation of Australia , St Leonards , New South Wales , Australia
  • Correspondence to Sally AM Sara; sally.sara{at}unisq.edu.au

Objectives Androgen deprivation therapy (ADT), a common treatment for prostate cancer, has debilitating impacts on physical and psychological quality of life. While some interventions focus on managing the physical side effects of ADT, there is a paucity of interventions that also address psychosocial and educational needs. The objective of this systematic review was to identify psychological and educational survivorship interventions targeting health-related quality of life (HRQoL) outcomes in men on ADT.

Design A systematic review of randomised controlled trials.

Data sources Web of Science, Cochrane, EBSCO Host, PubMed, SCOPUS from inception (1984) to 28 January 2023.

Eligibility criteria for selecting studies Psychological and/or educational survivorship interventions targeting HRQoL outcomes for men on ADT; minimum 80% of participants on ADT; used a validated HRQoL outcome measure; published in English in a peer-reviewed journal.

Data extraction and synthesis Data extraction using pre-specified study criteria was conducted. Heterogeneity of eligible studies precluded a meta-analysis.

Results A total of 3381 publications were identified with eight meeting the criteria. Interventions were either psychological with a cognitive behavioural approach (n=4), or educational with (n=2) or without (n=2) psychoeducational components.

Two studies reported a statistically significant improvement using a specific HRQoL measure. Most studies were not adequately powered and/or included small sample sizes limiting the conclusions that can be drawn on effectiveness. The most effective interventions were (i) individually based, (ii) educational with a psychoeducational component, (iii) supplemented with information packages and/or homework and (iv) included personalised needs assessments.

Conclusion There is a paucity of literature reporting psychological and educational survivorship interventions targeting HRQoL outcomes for men on ADT. What is urgently needed are person-centred survivorship interventions that are flexible enough to identify and address individual needs, taking into account the impact ADT has on both physical and psychological quality of life.

PROSPERO registration number CRD4202230809.

  • Prostatic Neoplasms
  • Clinical Trial
  • Health Education
  • MENTAL HEALTH
  • Nursing Care

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:  http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/bmjopen-2023-080310

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

To our knowledge, this is the first systematic review of educational and psychological-related interventions aiming to improve or maintain health-related quality of life in men on androgen deprivation therapy.

The focus on randomised controlled trials ensures a review of the highest level of evidence in relation to the effectiveness of educational and/or psychological interventions.

A number of studies had small sample sizes, and some had very short follow-up times so findings may not have fully reflected the men’s experience over time.

Not all included studies were adequately powered, and two were powered for a pilot study only, so caution is needed in interpreting results.

Only studies published in English were included; hence, studies conducted in non-English speaking countries may have been missed.

Introduction/background

Prostate cancer is the second most commonly diagnosed cancer globally in men and a significant cause of morbidity and mortality. 1 Androgen deprivation therapy (ADT), also known as hormone therapy, describes a common form of prostate cancer treatment that blocks the production of the male androgen testosterone, a hormone that stimulates the growth of prostate cancer cells. By reducing the amount of testosterone circulating in the body, the growth of prostate cancer is slowed, inhibiting progression of the cancer and increasing survival. ADT is the mainstay treatment for metastatic prostate cancer and routinely used as adjuvant or neo-adjuvant treatment with radiation therapy for intermediate to high-risk localised and locally advanced disease. Although very effective in reducing disease progression, the side effect profile is debilitating with significant impact on physical, psychological, sexual and metabolic health. 2 3

Men undergoing ADT lose muscle mass and bone mineral density, increasing risk of falls and bone fractures, and are at greater risk of death from cardiovascular disease. 4 5 Moreover, men report a profound impact on health-related quality of life (HRQoL) from testosterone loss, in particular changes to mood and cognition, loss of sexual function and libido, hot flushes and physical changes such as genital shrinkage, weight gain and growth of breast tissue. Reports in the literature indicate that men on ADT have significantly lower HRQoL scores than other prostate cancer treatments such as brachytherapy, external beam radiation therapy (without adjuvant ADT) and radical prostatectomy. 6 In addition to treatment side effects, men undergoing androgen deprivation live with the knowledge that they have high-risk localised, locally advanced or metastatic prostate cancer. Rates of depression in men with prostate cancer are higher than the general population, and higher again in men treated with ADT. 7–9 Of further concern, men diagnosed with prostate cancer have a 70% higher risk of suicide when compared with the general population, with men undergoing ADT at increased risk of suicidal ideation. 7

Survivorship care is an essential component of quality cancer care. Prioritising quality of life and well-being across the cancer trajectory, survivorship care incorporates the psychological, physical, social, emotional, financial and spiritual effects of cancer, from the point of diagnosis through the rest of life. 10 Survivorship interventions target short and long-term physical and/or psychosocial effects of the cancer and treatment. 11 Placing men with prostate cancer at the centre of their care, prostate cancer survivorship interventions should be widely accessible and take into account educational, psychosocial and informational needs in addition to physical activity, exercise medicine and nutritional interventions. 12 13 Consistent with current trends in prostate cancer survivorship care, intervention development and delivery should be guided by contemporary best practice frameworks that support responsive and coordinated short and long-term survivorship care. 12 There is a plethora of studies reporting the benefits of exercise medicine on the physical and psychological well-being of men on ADT, including increased muscle strength and weight control, lessening of fatigue and improved emotional well-being and quality of life. These studies have been soundly reviewed and reported in a number of recent systematic reviews focusing on the benefits of exercise in managing ADT-related toxicities and supporting the view that referral to tailored exercise programmes should be considered standard of care when prescribing ADT for the treatment of prostate cancer. 14–16 Further systematic reviews report specifically on the positive impact of exercise on quality of life. 17 18 Similarly, there is evidence relating to the impact of lifestyle modification including exercise and nutrition on maintaining HRQoL in men on ADT. 19 20 Despite the success of these interventions, men on ADT report significant unmet informational and supportive care needs in relation to the impact of treatment for prostate cancer on their lives, including loss of masculinity, reduced sense of control, fear of death and dying, uncertainty around disease progression, insomnia, hot flushes, sexual dysfunction and mood changes. 21–23 Physical changes, growth of breast tissue, loss of hair and genital shrinkage can have a profound psychological effect with men feeling like their bodies have undergone a feminisation process causing embarrassment, grief and decreased self-esteem. 24

With an estimated 30% to 50% of men diagnosed with prostate cancer undergoing ADT at some stage in their treatment trajectory, 25 and with a growing number of prostate cancer survivors predicted over the next few decades, 26 there is a critical need clinically for interventions that aim to improve overall health and HRQoL for men undergoing androgen deprivation, in addition to the benefits delivered by exercise and nutritional programmes. 8 22 27 This requires a systematic review and synthesis of the evidence in relation to key components and modes of delivery of educational and psychological survivorship interventions that are effective in improving HRQoL outcomes with the view to informing future intervention design.

This systematic review of the literature aims to (1) identify educational and/or psychological survivorship interventions (‘interventions’) targeting health-related quality of life outcomes for men with prostate cancer on ADT and evaluate their effectiveness and (2) analyse the key components and modes of delivery of these interventions to inform future intervention design.

This systematic review of randomised controlled trials was reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis 2020 Statement. 28 The review protocol was prospectively registered with PROSPERO (ID # CRD42022308096). No patients or public were involved in the study design.

Eligibility criteria

Inclusion criteria.

Studies were included if they met the predetermined criteria for review: (1) compared an educational and/or psychological survivorship intervention targeting HRQoL outcomes for men on ADT with standard care or another intervention, (2) randomised controlled trial study design with at least 80% of participants on ADT, (3) used a validated HRQoL measure to report primary or secondary outcomes and (4) available in English and published in a peer-reviewed journal.

For the purposes of this review, HRQoL was defined as specific health characteristics (such as health status, fitness, well-being and satisfaction) while also taking into account general quality of life factors such as physical, psychological, social and environmental factors. 29 There are a variety of validated instruments designed to measure HRQoL, including those that measure HRQoL more generally, and those that are more disease-specific.

Exclusion criteria

Studies were excluded if they described (1) interventions for men on ADT that did not report on HRQoL outcomes or did not use validated HRQoL measures; (2) surgical, radiological or pharmaceutical interventions; (3) interventions using nutritional or dietary supplements or other ingestive therapies; (4) exercise and/or nutritional and dietary interventions; (5) complementary and alternative medicine (CAM) interventions; (6) case reports, conference abstracts, editorials and studies not of randomised controlled trial design; and (7) studies not reported in English or published in a peer-reviewed journal.

Information sources and search strategy

Five electronic databases and search platforms were searched using key search terms: Web of Science, Cochrane, EBSCO Host, PubMed and SCOPUS. A search strategy was created and refined with the assistance of a health research librarian at the University of Southern Queensland ( online supplemental material 1 ).

Supplemental material

Terms within each set were combined using the Boolean ‘OR’ operator, and the sets were combined using the ‘AND’ operator. Potential search terms were trialled and mapped to indexed medical subject headings terms including prostatic neoplasms, randomised controlled trial and survivorship. Key search terms included prostate cancer, androgen deprivation therapy, randomised controlled trial, quality of life, side effects and survivorship. Reference lists of included articles were also searched.

All searches were run from database inception to 28 January 2023.

Data collection, extraction and synthesis

Identified articles from each database were imported into EndNote. After removing duplicates, the remaining titles and abstracts were imported into COVIDENCE. Initially, 10% of papers were independently reviewed against the eligibility criteria by three authors to check the inter-rater reliability (SS, NH and AG). The remaining title and abstracts were equally distributed between the same three authors who undertook independent review. Any disagreements were resolved by discussion until consensus was reached.

Full-text versions of potentially eligible studies were reviewed and screened against the eligibility criteria by one author (SS) using a data extraction table. Articles identified as meeting the inclusion criteria were checked by a second reviewer (NH). There were no disputes to resolve.

Data extraction using pre-specified study criteria was conducted by one author (SS) and checked by a second author (NH). Data extraction included study setting; participant demographics; study characteristics; intervention type, aim and outcomes measured; and results. Intervention characteristics extracted included intervention type and mode of delivery; content and components; frequency and duration. Outcomes included patient-reported HRQoL outcomes such as emotional, physical, social and functional well-being in addition to anxiety, depression, self-management and prostate cancer-specific HRQoL outcomes.

Due to the heterogeneity of the eligible studies (ie, diversity in outcome measures, duration, modes of delivery and aims), a meta-analysis was not conducted. This review followed Popay et al .’s guidance on the conduct of narrative synthesis in systematic reviews. 30

Study risk of bias assessment

Risk of bias was assessed by the lead author (SS) and independently checked by a second author (NH) using the Critical Appraisal Skills Programme Randomised Controlled Trials Standard Checklist. 31 Although the overall quality of the included studies was sound, and all studies had a clear protocol and research aims, there were some differences in the way some methods were presented. However, there were no issues around quality that led us to exclude a study. A detailed summary of the quality appraisal results can be found in online supplemental material 2 .

Patient and public involvement

Study selection.

The initial searches identified 3378 unique records with an additional three articles identified through other sources. Following removal of duplicates and title and abstract screening, 251 articles were included for full-text review. Eight publications 32–39 met the pre-established eligibility criteria and were included in the review (refer figure 1 ).

  • Download figure
  • Open in new tab
  • Download powerpoint

Flowchart of studies through the review process.

Study characteristics

All studies were published since 2004. Three studies were from the USA, 35 38 39 one from Denmark 33 and one each from Australia, 32 Scotland, 34 England 36 and Northern Ireland. 37 Two studies were conducted using web-based technology, 38 39 two were telephone-based 32 36 and four involved in-person sessions in outpatient settings. 33–35 37 Six studies reported 100% of participants on ADT 34–39 and the remaining two studies reported greater than 89% on ADT. 32 33 Refer to online supplemental table 1 for a summary of included studies.

A total of 656 men participated in the studies. Mean age of male participants across all studies ranged from 66.0 to 74.9 years, with an overall mean age of 69.2 (SD±2.6) years. Only four studies recorded time since diagnosis 32 36 38 39 which ranged from 2.1 to 6.6 years. Three studies included men with advanced prostate cancer, 32 34 39 three studies included men with either localised or advanced disease 33 35 36 and two studies did not report stage of disease at all. 37 38

All studies were described as randomised controlled trials and involved two arms (an intervention and a control). 32–39 Of the control arms across the eight studies, six were described as ‘Usual Care’, 33–38 one as ‘Minimally Enhanced Usual Care’ 32 and one as ‘Health Promotion Attention Control’. 39

All studies included as a primary or secondary outcome, a specific HRQoL measure comprising general health, cancer-related or disease-specific quality of life plus additional outcome measures such as anxiety, distress, depression, coping styles, fatigue, physical activity, hot flushes, night sweats, cognitive functioning, supportive care needs, disease knowledge and self-efficacy (refer online supplemental table 1 ). Several general and disease-specific instruments were used to measure HRQoL outcomes across the studies. Two studies 33 35 used the general health-related HRQoL Medical Outcome Study Short Form-12 (SF-12) 40 to measure the impact of health on everyday life. 29 Two studies 37 39 used the Functional Assessment of Cancer Therapy - General (FACT-G) 41 to measure cancer-related HRQoL. Two studies 34 36 used the European Organisation for Research and Treatment of Cancer Study Group on Quality of Life Questionnaire (EORTC QLQ-C30) 42 to assess themes relevant to people with cancer, and two studies 32 37 used the Functional Assessment of Cancer Therapy - Prostate (FACT-P) 41 to assess prostate cancer-specific HRQoL. In addition to using SF-12 to assess general HRQoL, one study used the Expanded Prostate Cancer Index Composite (EPIC) 33 to measure prostate cancer symptom-related HRQoL. All studies assessed participants at baseline and between 4 weeks and 9 months post intervention.

Intervention characteristics and outcomes

Interventions delivered tended to be either psychological 32 36 38 39 or educational, 35 37 with two educational interventions also including a psychoeducational component 33 34 (refer online supplemental table 2 and figure 2 ). For the purposes of this review, interventions delivering cognitive behavioural or relaxation therapy, or cognitive training delivered by a health professional were categorised as psychological interventions. Educational interventions included information about treatment and physical symptoms and side effect management (with no cognitive behavioural approaches) and were delivered primarily by nurses or in combination with other members of the healthcare team such as physical therapists, clinicians or trained facilitators

Matrix of outcomes and intervention characteristics.

Psychological interventions

There were four psychological interventions. 32 36 38 39 One involved cognitive behavioural stress management (CBSM) interventions, 39 one cognitive behavioural therapy, 36 one mindfulness-based cognitive therapy 32 and one computerised cognitive training. 38 The content of two interventions 36 39 included information on ADT side effects in addition to the cognitive approaches. Mode of delivery was web-based 38 39 or phone-based. 32 36 Two interventions were group-based 32 39 and two were individual only. 36 38 All four included homework in the form of a practice programme or diary, three had some degree of supervision 32 and one was a fully self-directed online package. 38

None of these psychological interventions demonstrated a statistically significant improvement on HRQoL measures but two studies showed an improvement in HRQoL-associated outcomes on symptom burden and depressive symptoms. 36 39 Notably, however one of these studies, 36 while powered to detect a clinically significant difference in hot flush and night sweat rating, reported a modest sample size in each arm. The authors of the other study 39 reported that the study was underpowered to detect significant intervention effects.

Educational interventions with psychoeducational component

There were two educational interventions that included a psychoeducational component. 33 34 Both interventions were individually based and delivered in person. They included explicit side effect management education and included written information packages. Both involved assessment of individual needs to enable delivery of a personalised, tailored intervention and involved a multidisciplinary approach (delivered by nurses and/or allied health professionals).

One study demonstrated a statistically significant improvement in HRQoL outcome on SF-12 (physical component summary p=0.002). 33 This study was both powered sufficiently and demonstrated a small to moderate effect size on prostate cancer-specific symptom bother and physical HRQoL. The second study 34 was powered for a pilot trial sample size and did not report effect size. While not demonstrating statistically significant HRQoL outcomes over time, this study did demonstrate statistically significant reduction in unmet supportive care needs in the intervention group at 3 months compared with control (p=0.002), with greatest improvements in the following domains of unmet needs: physical symptoms, fear of cancer spreading, fear of death and dying, changes in sexual feelings, informational needs and self-management.

Educational interventions with no psychoeducational component

There were two educational interventions with no psychoeducational content included. 35 37 Both interventions were individually based and involved in-person delivery sessions. The first was nurse-delivered and included an information booklet for participants to supplement the education session. 37 This intervention demonstrated a statistically significant improvement in HRQoL outcome in FACT-G (p<0.001) and FACT-P (p<0.001) between pre-test and post-test, with additional significant changes in emotional and functional well-being following FACT-P subscale analysis (p<0.01). 37 However, this study did not report power analysis or effect size and had a very short follow-up with the post-test questionnaire completed 4 weeks post intervention.

The second study involved multidisciplinary assessment and counselling on symptom management and was delivered by a dietitian, palliative care physician, and trainer. 35 There were no statistically significant differences between treatment arms for all primary and secondary outcomes nor did this study meet the recruitment target or report effect size.

Significant outcomes

Importantly, only two of the eight studies reported a statistically significant improvement using a specific HRQoL measure, namely FACT-G and FACT-P scales and the SF-12. One was a nurse-led educational intervention 37 and the other was a multidisciplinary educational intervention with psychoeducational components. 33 Both interventions were delivered in the individual setting, and included supplementary educational materials and specific information on the management of ADT side effects. 33 37 One additional study, 39 a CBSM web-based programme, reported a positive trend in functional well-being (p=0.06) and emotional well-being (p=0.07) on the FACT-G subscale.

Although only two studies showed statistically significant changes related to specific HRQoL outcomes, there were three studies that showed statistically significant improvement in associated outcomes such as symptom burden, anxiety and depressive symptoms and unmet supportive care needs. 34 36 39 Of these, 34 36 39 two were psychological interventions 36 39 and one was educational/informational in design and included psychoeducational components. 34

Irrespective of whether the statistically significant improvement was in HRQoL outcomes or an associated outcome, all five of these studies included homework or a supplementary information package and included specific information about ADT side effects. Four involved individual participation 33 34 36 37 and one was group-based. 39 Only two included individual needs assessment allowing for individual care planning and personalisation of the intervention for each participant. 33 34

This systematic review aimed to identify educational and psychological survivorship interventions targeting HRQoL outcomes for men with prostate cancer on ADT and analyse their key components, modes of delivery and their effectiveness in order to highlight any gaps in the literature and to inform future intervention design. Of the 3319 studies screened, only eight studies focused on addressing HRQoL issues for men on ADT. This small number of eligible studies indicates a lack of research into this area. Of note, the majority of included studies were not adequately powered and/or included small sample sizes limiting the conclusions that can be drawn on intervention effectiveness. Two reported small to moderate effect sizes in HRQoL outcomes 33 39 and three did not report effect sizes at all. 34 35 37 Consequently, caution needs to be applied when interpreting the findings including the studies that reported statistically significant changes.

The majority of included studies described interventions with cognitive-based psychological or psychoeducational components. Concerningly, only two studies demonstrated statistically significant improvements using a specific HRQoL measure. 33 37 One was a nurse-led educational intervention 37 which supports the evidence in the literature that nurse-led interventions lead to significant improvements in HRQoL. 43 The other was a multidisciplinary educational intervention with psychoeducational components. 33 Both interventions were delivered in the individual setting, and included supplementary educational materials and specific information on the management of ADT side effects.

In addition to the two studies reporting statistically significant HRQoL outcomes, a further three studies demonstrated significant associated outcomes that are likely to impact overall HRQoL, such as improvement in symptom burden, cancer-related depressive symptoms and supportive care needs. 33 34 36 37 39 Interestingly, what these five studies had in common was that they were clinician-led, primarily directed at individuals, included a supplementary information or homework package and included specific information about ADT side effects. All but one of the interventions demonstrating significant improvements were supervised which highlights the importance of participants feeling they are not alone by linking them to a person or team delivering the intervention. 44 45 Of note, only three of the eight studies were designed exclusively for men with metastatic prostate cancer. 32 34 39 We expected more given that men with advanced prostate cancer are recognised as being at risk of poorer psychosocial outcomes. 22 46 In our experience, studies focusing on men with metastatic cancer are harder to recruit for; however, retention rates may be higher if the mode of delivery caters for their needs, highlighting the importance for interventions to be designed so they can be tailored to men’s health and social needs, including modes of delivery that may lessen the impact of travel and appointment attendance. 22 43

Interestingly, no single intervention included cognitive behavioural approaches in addition to educational and psychoeducational aspects. This was unexpected as the literature indicates that multimodal approaches combining cognitive-behavioural and educational approaches addressing disease and treatment management information, side effect advice, stress management, and problem solving, goal setting and cognitive behavioural approaches have been shown to reduce distress and improve HRQoL outcomes in the cancer setting. 43 47–49 Surprisingly, across all eight studies, only two included individualised needs assessments enabling personalised care and tailoring of the intervention to each participant’s identified needs. 33 34 Both of these studies demonstrated a statistically significant outcome although only one showed a statistically significant change in a specific HRQoL outcome. 33 The other study 34 included a supportive care needs assessment which led to an individualised self-management plan, demonstrating an association between supportive care needs and HRQoL, with evidence in other studies that if supportive care needs are not met, HRQoL is impaired. 50 51 With a global trend towards personalised medicine and person-centred care, the design of future interventions addressing HRQoL for men on ADT should move away from a ‘one size fits all’ to an individualised approach. 12 52 Given the interplay between HRQoL and individual care needs it is vital that maintaining HRQoL should be a key goal in the delivery of person-centred survivorship care.

Until recently, models of care supporting the delivery of coordinated, accessible and personalised survivorship care have been missing from the prostate cancer setting. Since 2020, the Prostate Cancer Survivorship Essentials Framework 12 has provided a set of key domains that directly influence HRQoL in men with prostate cancer, yet when we reviewed the literature for examples of survivorship interventions that address these domains in the ADT setting (such as health promotion, vigilance, care coordination and personal agency), the results were sparse. Beyond exercise medicine and nutrition, there are very limited examples of effective survivorship interventions that address the informational, physical and psychological needs of men undergoing androgen deprivation. This void has implications clinically where psychological health issues can have a serious impact on HRQoL in men on ADT. Consequently, interventions that incorporate psychological care are paramount. 52 In addition, despite limited studies in this review demonstrating significant improvement educational interventions may have on HRQoL, education is a crucial component of health promotion and personal agency and access to personalised educational interventions should be considered a critical element of best practice survivorship care. For health professionals looking for examples of effective and accessible interventions they can translate and deliver into clinical practice, the lack of effective educational and psychological interventions is of concern.

This systematic review comparing intervention type, mode of delivery, content, duration and outcome suggests that the most effective characteristics of interventions aiming to improve HRQoL outcomes for men with prostate cancer on ADT are interventions that are (i) individually based, (ii) educational in design with a psychoeducational component, (iii) supplemented with home-based information packages with reading and/or activities and (iv) include personalised individual needs assessments. Cognitive-based psychological components may add to the effectiveness when delivered in conjunction with educational components but appear to be less effective when delivered as a standalone intervention. While we focused on studies that included a validated HRQoL measure, it is critical to remember that addressing factors such as anxiety, depression and fear of recurrence will impact on overall HRQoL. Ideally, interventions should commence early, aligning as close of possible to commencement of ADT, and include a multisession approach with ‘check in’ opportunities between men and their healthcare team when the side effects really start to take hold, recognising that informational and supportive care needs can vary over time. Supervised sessions that are individually tailored appear to have a higher chance of improving HRQoL. The inclusion of low-intensity psychological care with cognitive behavioural approaches should be considered in relation to future design, specifically in relation to stress and coping, problem solving and goal setting. 53

When designing interventions for men undergoing ADT, it is important that health professionals and researchers take into account the influence masculinity can have on health outcomes, and work to incorporate male preferences in terms of design and acceptability. 46 54 55 For example, a problem-solving approach can lead men to identify individual problems, explore solutions, set goals, test strategies and determine the best solution for them, with the ultimate goal to reduce or limit some of the sources of stress in their lives. 53 Competing demands between work, family and social commitments, and masculine ideals, such as stoicism and self-reliance can mean that men are reluctant to access services, and actively seek out support. 55 Interventions need to be accessible, men-centred and provide opportunities for targeted support tailored to the needs of the individual, using problem-solving approaches. In the clinical setting, consideration should also be given to access and equity with an increasing focus on technology. 56 At a minimum, interventions should include educational materials and information about the impact of ADT including side effect management, screening for distress and identification of problems and needs, leading to an individualised person-centred care plan. 57

Limitations

This review included studies published in English only due to financial costs and time factors relating to professional translation; hence, studies conducted in non-English speaking countries may have been missed. Web-based machine translation such as Google Translate was not employed due to concerns around evaluation of context, and degree of accuracy in the absence of word for word translation. 58 59

There were a number of studies with small sample sizes; some had very short follow-up times so findings may not have fully reflected the men’s experience over time. Moreover, not all the studies were adequately powered, and two were powered for a pilot study only, so caution is needed in interpreting results. Despite these limitations, to our knowledge, this is the first systematic review of educational and psychological-related interventions aiming to improve or maintain HRQoL in men on ADT.

It is well established that men on ADT often face severe decrements in quality of life. While there is a large body of literature describing the impact ADT has physically and psychologically, men still report significant unmet informational, educational and supportive care needs. There is limited evidence of interventions that effectively address these concerns. While there are many studies to mitigate ADT side effects using exercise, there is a scarcity of evidence evaluating the effectiveness of educational and psychological survivorship interventions on health-related quality of life, and what can be found appears to be hindered by small sample sizes and inadequate powering of studies. What is urgently needed are person-centred interventions that are flexible enough to identify and address individual needs, taking into account the impact ADT has on both physical and psychological quality of life.

When designing interventions for men undergoing ADT, it is imperative that health professionals and researchers incorporate men’s health behaviours, consider male preferences in terms of design and acceptability and incorporate cognitive behavioural approaches with educational and psychoeducational components. Interventions need to be accessible, use problem-solving approaches and provide opportunities for targeted support tailored to the needs of the individual. A one size fits all approach with no psychoeducational component or individual assessment is least likely to address HRQoL outcomes in a meaningful way.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

Acknowledgments.

The authors acknowledge the contribution of Ms Rowena McGregor, Health Librarian University of Southern Queensland, in the development of the literature search strategy.

  • Siegel RL , et al
  • Bellmunt J ,
  • Briers E , et al
  • Chipperfield K ,
  • Fletcher J ,
  • Millar J , et al
  • Gunter JH ,
  • Heathcote P , et al
  • Edmunds K ,
  • Tuffaha H ,
  • Galvão DA , et al
  • Drummond FJ ,
  • Kinnear H ,
  • O’Leary E , et al
  • Chambers S ,
  • Green A , et al
  • Donovan KA ,
  • Walker LM ,
  • Wassersug RJ , et al
  • Berruti A ,
  • Cracco C , et al
  • National Cancer Institute
  • Denlinger CS ,
  • Carlson RW ,
  • Are M , et al
  • Ralph N , et al
  • Lazenby M , et al
  • Scuffham P , et al
  • Brooker J ,
  • Fletcher J , et al
  • Chan A , et al
  • Hasenoehrl T ,
  • Keilani M ,
  • Sedghi Komanadj T , et al
  • Carmack Taylor CL ,
  • de Moor C , et al
  • Gonzalez BD ,
  • Nelson AM , et al
  • Paterson C ,
  • Nandwani G , et al
  • Primeau C ,
  • Robinson JW
  • de Reijke TM ,
  • Van Tienhoven G , et al
  • Soerjomataram I , et al
  • McKenzie JE ,
  • Bossuyt PM , et al
  • Theofilou P
  • Roberts H ,
  • Sowden A , et al
  • Critical Appraisal Skills Programme
  • Chambers SK ,
  • Occhipinti S ,
  • Foley E , et al
  • Dieperink KB ,
  • Johansen C ,
  • Hansen S , et al
  • Pollock Y ,
  • Kenfield SA , et al
  • Stefanopoulou E ,
  • Grunfeld EA , et al
  • Templeton H ,
  • Tanenbaum ML , et al
  • McGinty HL ,
  • Mohr DC , et al
  • Kosinski M ,
  • Tulsky DS ,
  • Gray G , et al
  • Bottomley A
  • Donovan M ,
  • Willener R ,
  • Spichiger E
  • Laurie K , et al
  • Smith DP , et al
  • Penedo FJ ,
  • Molton I , et al
  • Matcham F ,
  • Hutton J , et al
  • Chen J , et al
  • Schnabel A ,
  • Lordick F ,
  • Oberth P , et al
  • Boorjian SA ,
  • Briganti A , et al
  • Oliffe JL ,
  • Rossnagel E ,
  • Bottorff JL , et al
  • Dickinson R ,
  • Sinclair JE , et al
  • Newton RU ,
  • Chen ML , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

  • Data supplement 1
  • Data supplement 2
  • Data supplement 3
  • Data supplement 4

Contributors SS is the author responsible for the overall content as the guarantor. Planning for this paper was undertaken by SS, VT, SC, NH and JD. Data collection and management were undertaken by SS and reviewed by NH and AG. VT, JD, SC, NH and AG provided critical review of the article. All authors reviewed and gave final approval of the version to be published.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

Disclaimer » Advertising

  • HealthyChildren.org

Key Findings

Strengths and limitations, conclusions, the clinical definition of children with medical complexity: a modified delphi study.

K Millar and A Garland contributed equally as co-senior authors.

FUNDING: All phases of this study were supported by Michael Nesbitt Family Complex Care Fellowship.

CONFLICT OF INTEREST DISCLOSURES: Dr Cohen reported being a member of the Committee to Evaluate Drugs, which provides advice to Ontario’s Ministry of Health on public drug policy. The other authors have indicated they have no potential conflicts of interest relevant to this article to disclose.

  • Split-Screen
  • Article contents
  • Figures & tables
  • Supplementary Data
  • Peer Review
  • CME Quiz Close Quiz
  • Open the PDF for in another window
  • Get Permissions
  • Cite Icon Cite
  • Search Site

Kyle Millar , Celia Rodd , Gina Rempel , Eyal Cohen , Kathryn M. Sibley , Allan Garland; The Clinical Definition of Children With Medical Complexity: A Modified Delphi Study. Pediatrics 2024; e2023064556. 10.1542/peds.2023-064556

Download citation file:

  • Ris (Zotero)
  • Reference Manager

Video Abstract

Children with medical complexity (CMC) comprise a subgroup of children with severe chronic diseases. A conceptual definition for CMC has been formulated, but there is no agreement on criteria to fulfill each of the 4 proposed domains: diagnostic conditions, functional limitations, health care use, and family-identified needs. Our objective with this study was to identify a standardized definition of CMC.

Through a scoping review of the CMC literature, we identified potential criteria to fulfill each domain. These were incorporated into an electronic survey that was completed by a geographic and professionally varied panel of 81 American and Canadian respondents with expertise in managing CMC (response rate 70%) as part of a 4-iteration Delphi procedure. Respondents were asked to vote for the inclusion of each criterion in the definition, and for those with quantitative components (eg, hospitalization rates), to generate a consensus threshold value for meeting that criterion. The final criteria were analyzed by a committee and collapsed when situations of redundancy arose.

Of 1411 studies considered, 132 informed 55 criteria for the initial survey, which was presented to 81 respondents. Consensus for inclusion was reached on 48 criteria and for exclusion on 1 criterion. The committee collapsed those 48 criteria into 39 final criteria, 1 for diagnostic conditions, 2 for functional limitations, 13 for health care use, and 23 for family needs.

These results represent the first consensus-based, standardized definition of CMC. Standardized identification is needed to advance understanding of their epidemiology and outcomes, as well as to rigorously study treatment strategies and care models.

Children with medical complexity are growing in number, which impacts the health care system and caregivers. Their complexity requires new methods of delivering health care; however, research into new methods is limited by the lack of a standardized definition.

The Cohen 2011 definitional framework is one of the most widely used in the field. It requires the fulfillment of 4 domains; however, there are no domain fulfillment criteria. This study proposed fulfillment criteria and, hence, a standardized definition.

Children with medical complexity (CMC) have been a particular target for health system improvement among individuals with chronic diseases because of their pronounced health care utilization (HU), functional limitations (FL), and family needs (FN). 1 Due in part to a lack of consistent methods for identifying them, population prevalence estimates range almost 50-fold, between 0.20% and 11.4% of children. 2–6 The importance of this group relates to their unique and intense needs that health systems and caregivers are often unable to meet. In population-level analyses, estimates suggest that their prevalence is low but their impact is large. CMC may account for <1% of children, yet they account for 33% of all pediatric health care expenditures, 37% of hospitalizations, 54% of inpatient days, and 57% of hospital costs. 2 , 5 Additionally, parents of CMC experience substantial risk of psychosocial stress, health deterioration, and financial pressures. 7–13 Their prevalence is likely rising with increasing survival among premature infants, children with congenital anomalies, and oncology survivors. 14–26  

The most widely accepted framework defining CMC was created in 2011 by Cohen et al. 27–31 It requires fulfilling all of the following 4 domains: diagnostic conditions (DC), FL, FN, and increased HU. 29 However, the framework does not specify criteria to fulfill the domains; thus, each domain can have broad, variable means of fulfillment. 32 Without fulfillment criteria, this framework cannot yet be considered a standardized definition, which is an absence that impedes the generalizability of future research and system building. 12 , 27–29 , 32–46 Both the American Academy of Pediatrics and the Canadian Pediatric Society have called for such standardization to advance investigations into optimal care models and management strategies. 27 , 47–49  

A criticism of the 2011 framework is that it does not integrate social determinants of health (SDH) into the definition. 28 SDH are critical contributors to health outcomes, and specific to CMC, they may obstruct the delivery of care in the community, impact the affordability of health treatments, and impact the mental health of the CMC and their caregivers. 50–52 Detrimental SDH may be preexisting before the child with medical complexity arrives or develop subsequently as a consequence of their needs. Although the conceptual definition lacks direct reference to SDH, the FN domain has significant overlap with SDH. 28  

Our objective was to identify specific criteria for each of the 4 domains of the 2011 conceptual framework that additionally incorporate SDH. For this purpose, we used a modified Delphi survey method with a panel of American and Canadian CMC clinicians and researchers.

We performed a 4-iteration Delphi survey study in 2020. A Survey Development Committee (SDC) included an epidemiologist, 2 complex care clinicians, 2 pediatric researchers, and 2 experts in Delphi methods. The SDC (1) reviewed the literature for variables to inform potential criteria, (2) built those criteria into questions for the first iteration survey, and (3) after each iteration, reviewed respondents’ comments.

Potential criteria to be included in the initial survey were identified by reviewing the CMC literature from January 1998 to September 2019. Articles were identified by using scoping review methodology. 53 Broad search criteria were used to search the Ovid Medline, Ovid Embase, and Scopus databases ( Supplemental Table 4 ). Abstracts were chosen for full article review if they were published in a peer-reviewed journal and the authors assessed the definition, epidemiology, etiology, natural history, or management of CMC. Abstracts were excluded for further consideration if they were limited to children with specific chronic conditions, case studies with <10 children, studies of the education of medical learners, or evaluations of the impact of CMC in adulthood, including transition planning.

During full review, potential domain criteria were extracted and originated as independent variables, dependent variables, covariates, or qualitative themes. Abstract review, full review, and criterion extraction was completed by a single reviewer (KM). The potential criteria were converted to criteria statements by the same researcher, and then, through an iterative process, the SDC debated and revised the statements. Through unanimous agreement, the SDC chose the final list of criteria for inclusion in the first iteration survey.

Special attention was given to 2 aspects of criteria extraction and generation. Firstly, SDH were included as criteria in the FN domain. 28 Secondly, the criteria in the DC domain were presented in a way that aligned with a decategorization definition philosophy. Decategorization definitions group patients by their closely shared health and social consequences rather than by their underlying disease. 54–56 Using 4 conceptual criteria, we asked the respondents whether the DC domain was needed in the framework or whether any condition, chronic condition, or complex chronic condition was needed for domain fulfillment.

Through survey questions, we presented potential domain criteria to the respondents. They were informed to indicate their agreement on whether a particular criterion, if present in a patient, would solely fulfill its corresponding domain (Level 1 criteria), with the understanding that an individual must fulfill at least 1 criterion from each domain.

Agreement was rated on a 5-point Likert scale. Quantitative items had follow-up questions to determine the threshold value required to fulfill a criterion (Level 2 criteria). For example, increased hospitalizations may represent a potential Level 1 criteria, and 3 hospitalizations per year may be the minimum fulfillment threshold representing Level 2 criteria. For the first survey iteration only, respondents had the opportunity to suggest a threshold, and in subsequent iterations, these responses were converted to Level 2 criteria scales. For each question respondents were encouraged to record the rationale for their votes and, in iteration 1, to suggest changes to existing criteria and propose new criteria. The first iteration survey was pilot tested for face validity by 2 content experts and 1 survey methodology expert.

Eligibility criteria for respondents were ≥2 years of experience clinically managing CMC or having been first or senior author on at least 1 peer-reviewed article in the past 3 years on the topic of CMC. Criterion sampling was used to construct a sample of potential Delphi respondents. Respondents were recruited from 4 sources, including the Complex Care Google-group members of the Complex Care Special Interest group of the Pediatric Academic Society, the Complex Care Committee of the American Academy of Cerebral Palsy and Developmental Medicine, the Complex Care Section of the Canadian Pediatric Society, and authors of publications included in the scoping review. For the first 3 of these, e-mail invitations to participate were sent anonymously to members by the organizations. As such, we cannot calculate population response rates.

A maximum of 4 survey iterations was predetermined. When considering Level 1 criteria, the consensus threshold was 80% answering strongly agree or agree (agreement) or strongly disagree or disagree (disagreement). For Level 2 criteria, the consensus threshold was 70% of respondents choosing the same threshold count. Criteria reaching consensus for agreement in any iteration were used in the final definition and removed from subsequent iterations. Criteria reaching consensus for disagreement were removed. Criteria that did not reach consensus were presented to respondents in the next survey iteration and not included in the final definition if non-consensus persisted through the final iteration. For Level 2 criteria in which consensus criteria were not reached but whose corresponding Level 1 criteria had reached inclusion consensus, we selected the threshold using an a priori designed algorithm ( Supplemental Fig 3 ).

For all questions, included in iterations 2 to 4, anonymous, structured feedback from the previous iteration was provided, including (1) the individual’s previous answer, (2) a histogram of all answers, and (3) a distillation of themes from the comments. With each iteration, the respondents’ comments were reviewed by a single reviewer (KM), themes were generated by using content analysis, and then the comments were reviewed by a second reviewer (AG). Responses were anonymous to other respondents and the researchers.

On completion of all survey iterations, the SDC agreed that several sets of criteria that reached inclusion consensus were complementary or redundant and were amenable to collapse into a single criterion. The collapsed criteria were generated by a reviewer (KM), and the SDC refined and approved the collapsed criteria through iterative discussion.

Descriptive statistics were reported as count and percentages of overall survey completion rate, respondents’ demographics, commenting rates per criterion, and agreement and disagreement rates with level 1 and level 2 criteria.

All respondents gave their informed consent for participation in the study. The study progressed in accordance with the principles embodied in the Declaration of Helsinki, and the protocol was approved by the University of Manitoba Research Ethics Board (REB# H2019:262 [HS22970]).

Among 1411 abstracts reviewed, 325 manuscripts underwent a full-text review, of which 132 contributed 197 variables that had the potential to populate criteria in the first iteration survey ( Fig 1 , Supplemental Table 5 ). The SDC refined these into 28 criteria for the FN domain, 14 for HU, 9 for FL, and 4 for DC.

Flow diagram demonstrating the article selection process and criteria creation, stratified by domain.

Flow diagram demonstrating the article selection process and criteria creation, stratified by domain.

A total of 82 professionals replied to the invitation to participate, of whom 1 formally withdrew after iteration 1. Of those who replied, survey completion rates varied 69% to 82% across the 4 iterations, with 2.5% to 8.5% partly completing surveys ( Table 1 ). Of participants who were enrolled, 7 (8.5%) never completed a survey iteration, 12 (15%) completed 1, 5 (6.1%) completed 2, 15 (18%) completed 3, and 43 (52%) completed all 4. Table 2 reveals respondent characteristics. Most respondents (77%) were from the United States, and although most were physicians, there was representation by a wide variety of professionals.

Completion Characteristics of Delphi Survey Iterations 1 to 4

A survey was “fully complete” if all criteria were voted on (does not include completing demographics). A survey was “partially complete” if any criterion was voted on. “Incomplete” surveys were those that were left completely blank.

“New” respondents did not participate in the previous iteration. “Returning” participated in the previous iteration. “Withdrawn” did not participate in the current iteration but did participate in the previous iteration. “No participation” did not participate in the previous or current iteration.

Demographic Description of the Delphi Respondents Across Iterations 1 to 4

“Total responses” indicate the number and percentage of respondents who fully or partially completed a survey that chose to indicate their demographic variable.

In survey iteration 1, respondents suggested wording changes to 24 of 55 level 1 criteria, and 18 of 21 level 2 criteria ( Supplemental Table 6 ). After the completion of 4 iterations, 48 level 1 criteria reached consensus for inclusion, including 25 in iteration 1, 10 in iteration 2, 10 in iteration 3, and 3 in iteration 4 ( Fig 2 ). Of this total, 24 reached consensus in the FN domain (86% inclusion consensus rate), 13 in HU (93%), 9 in FL (100%), and 2 in DC (50%). One level 1 criteria in the DC domain reached consensus for exclusion. There were 21 level 2 criteria for consideration, with 6 in FN, 11 in HU, 1 in FL and 3 in DC ( Supplemental Table 7 ). Of these, 5 reached consensus by the end of iteration 4, and obeying the algorithm to deal with unresolved criteria, another 11 reached consensus after completion of the final iteration.

The number of definition criteria that reached inclusion and exclusion consensus across all 4 iterations stratified by definitional domain.

The number of definition criteria that reached inclusion and exclusion consensus across all 4 iterations stratified by definitional domain.

After the final survey iteration, the SDC collapsed 8 FL criteria into 1 criterion, 2 DC criteria into 1 criterion, and 2 FN criteria into 1 criterion ( Supplemental Table 8 ). After collapsing, 39 criteria made up the final definition, 23 in FN, 13 in HU, 2 in FL, and 1 in DC ( Table 3 ).

Final Definitional Criteria of CMC, Stratified by Domain

ADL, activities of daily living; VBSF, vital body structures or function.

Without a standardized definition of CMC, it is not possible to make comparable estimates of basic epidemiology, compare health outcomes, or compare the efficacy of interventions. This work is the first to standardize the fulfillment criteria for the 4 domains of Cohen’s definitional framework, with at least 1 criterion required from each domain. 29 The content of the 39 final criteria should be obtainable from most medical records and caregiver interviews; however, residual criteria subjectivity and limitations in some medical records may make this difficult in some settings. Additionally, with this study, we addressed a criticism of the framework by integrating SDH criteria into the FN domain. SDHs were represented with criteria directly related to 5 of the 6 concepts of the Artiga and Henton 2018 model of SDH, including (1) economic stability (represented by items 17 to 19 in Table 3 ), (2) neighborhood and physical environment (items 37 to 38), (3) education (item 28), (4) community and social context, 22–26 , 34 , 35 and (5) health care system (items 20, 22, and 30 to 31). 50 Consideration of SDH are important as detrimental SDH can amplify medical complexity of a child through their fundamental impact on health outcomes and care delivery. 28 , 50–52  

The evolution of consensus provides insights into the perception of our respondents. First, succinctly incorporating the breadth of criteria that map onto a domain like FN is challenging. For example, this domain has many criteria chosen for consensus, and there is greater subjectivity in wording compared with the others. Despite this, the rate of consensus of the FN criteria was similar to that of the other domains, with no new items suggested by respondents in iteration 1. Second, regarding FL, some authors argue that children with medial complexity should require assistance from medical technology as a representation of the severity of their limitations. 29 , 48 , 57–59 Our respondents’ consensus was that only the presence of a limitation, without a qualifier of severity, was necessary to fulfill this domain. Lastly, regarding DCs, respondents agreed that a chronic diagnostic condition must be present to fulfill that domain. Notably, the respondents rejected the presented option to exclude this domain altogether or to include all DCs, independent of chronicity or complexity. The consensus agreed with the philosophy of Cohen et al that a chronic condition must be present to fulfill the definition of CMC but excluded the need for a complex chronic condition as defined by Fuedtner et al in 2014. 29 , 60 , 61  

The complex interaction of factors in the life of any child with medical complexity led to difficulty categorizing criteria into 4 completely distinct domains. This is recognized in the World Health Organization’s International Classification of Functioning, Disability, and Health (ICF). The ICF framework represents a biopsychosocial model in which disability is viewed as a complex interaction between measures of health conditions, functioning, participation, environmental factors, and personal factors. 59 Although the respondents agreed that the final criteria fulfilled their assigned domain, we acknowledge that criteria do not always exclusively fit in 1 domain. First, criteria residing in 1 domain may have features applicable to other domains. An example is item 2 ( Table 3 ) in the FL domain, which refers to the need for prescribed specialized therapy. Such therapies can also be considered health services and may also fit in the HU domain. Second, some criteria may directly influence another domain. For example, in the FN domain, item 20 refers to the lack of medical insurance, item 29 to the child missing an impactful number of medically necessary appointments, and item 30 to the failure to receive management deemed medically necessary. The presence of these items could also reduce the amount of HU due to lack of access.

Notable strengths of this study include the extensive and broad literature review with which we identified nearly 200 variables with the potential to inform criteria in the 4 domains. Additionally, these potential criteria were considered by a diverse group of CMC professionals in the United States and Canada by using robust Delphi methodology. Lastly, by incorporating SDHs into the FN domain, with this study, we addressed a criticism of the Cohen et al 2011 framework.

Our study has several limitations. First, there were instances in which a single coder or reviewer was used rather than double reviewing. However, the authors of a study comparing the performance of 1 versus 2 reviewers concluded that a single reviewer is acceptable when that reviewer is experienced in the field, as was the case in this study. 62 Second, volunteer bias is possible; however, this is an intrinsic issue for all Delphi studies because we are actively seeking out respondents motivated on the issue. Third, overrepresentation of physicians over other clinicians, clinicians over academics, and Canadians over Americans may suggest that our respondents may not fully represent the views of the greater population. Fourth, the modification of the final definitional criteria was performed. The modification of consensus items is a relatively common occurrence in the Delphi literature. Some authors will alter or remove items because of redundancy, which was the motivation in this study. 63 Finally, this study focused on the opinions of clinicians without including input from CMC or their caregivers. The decision to focus first on clinicians reflects the recognition that, although caregivers have vital lived experiences that clinicians lack, we required respondents that had experience with the highly diverse children that can be defined as CMC. An important next step in this process will involve engaging CMC caregivers for their input on these criteria.

Although not a methodologic limitation, we recognize that this definition is broad, heterogenous, and contains subjectivity in some criteria. Subjectivity may be an unavoidable feature of the definition and not necessarily a limitation. For example, criterion 17 states that “In order to care for the medical needs of their child, the caregiver(s) are experiencing an impactful amount of planned or unplanned work disruption.” An objective measure of “impactful” may not be generalizable to all children, depending on their circumstances. With this study, we laid the groundwork to move toward a broad, core definition of these children, with future work needed to resolve criteria weighting and, if possible, subjectivity. It is acknowledged the definition may not function fully when using some medical records as the sole data source. It is inevitable that there will be adaptations of this core definition as clinicians and researchers adjust the definition to their goals, needs, and data availability, particularly when using administrative datasets.

By utilizing the consensus of a diverse group of CMC clinicians, we have, for the first time, specifically defined CMC. By reducing subjectivity in fulfilling the 4 domains, with these 39 criteria, we move to standardize the clinical and research definition of CMC. In addition, with this study, we successfully incorporated SDH into the definition and discovered that respondents (1) had difficulty removing subjectivity from the FN domain, (2) did not believe an FL required dependence on medical technology, and (3) believed that a chronic DC is necessary for the definition of CMC. In the future, these results must be amended using CMC and caregiver input and trialed in a clinical population, and criteria subjectivity and weighting must be resolved. It could then be used as a template to generate more specific administrative data definitions and define more generalizable clinical and research populations. Additionally, this definition is broad and heterogenous because the CMC concept refers to a shared endpoint reached through the accumulation of many combinations of criteria. Therefore, future research is needed to identify subpopulations of CMC that could inform better patterns of resource allocation and programming design.

Dr Millar conceptualized and designed the study, designed the data collection instrument, performed and supervised data collection, drafted the initial manuscript, and conducted the analysis; Drs Rodd, Rempel, Cohen, and Sibley designed the study and critically reviewed and revised the data collection instrument; Dr Garland conceptualized and designed the study, critically reviewed and revised the data collection instrument, and supervised data collection; and all authors critically reviewed and revised the manuscript, approved the final manuscript as submitted, and agree to be accountable for all aspects of the work.

COMPANION PAPER: A companion to this article can be found online at  https://www.pediatrics.org/cgi/doi/10.1542/peds.2023-065014 .

children with medical complexity

diagnostic conditions

functional limitations

family needs

health care utilization

World Health Organization’s International Classification of Functioning, Disability, and Health

Survey Development Committee

social determinants of health

Competing Interests

Supplementary data.

Advertising Disclaimer »

Citing articles via

Email alerts.

four components of a literature review

Affiliations

  • Editorial Board
  • Editorial Policies
  • Journal Blogs
  • Pediatrics On Call
  • Online ISSN 1098-4275
  • Print ISSN 0031-4005
  • Pediatrics Open Science
  • Hospital Pediatrics
  • Pediatrics in Review
  • AAP Grand Rounds
  • Latest News
  • Pediatric Care Online
  • Red Book Online
  • Pediatric Patient Education
  • AAP Toolkits
  • AAP Pediatric Coding Newsletter

First 1,000 Days Knowledge Center

Institutions/librarians, group practices, licensing/permissions, integrations, advertising.

  • Privacy Statement | Accessibility Statement | Terms of Use | Support Center | Contact Us
  • © Copyright American Academy of Pediatrics

This Feature Is Available To Subscribers Only

Sign In or Create an Account

IMAGES

  1. The process of writing a literature review

    four components of a literature review

  2. main components of a literature review

    four components of a literature review

  3. Literature review introduction

    four components of a literature review

  4. what are the different components of a literature review

    four components of a literature review

  5. Literature Review Outline: Writing Approaches With Examples

    four components of a literature review

  6. Process of literature review

    four components of a literature review

VIDEO

  1. Components of literature of and methods of get relevant literature

  2. Essential Components of the Literature Review

  3. Components of Literature: b. Setting

  4. Components of Literature: Point of View

  5. Indian Aesthetics Important Components #ugcnet #englishliterature #indianaesthetics #theory #shorts

  6. Literature Review Definition,Importance,types,steps,issues in Urdu and Hindi

COMMENTS

  1. How to Write a Literature Review

    Examples of literature reviews. Step 1 - Search for relevant literature. Step 2 - Evaluate and select sources. Step 3 - Identify themes, debates, and gaps. Step 4 - Outline your literature review's structure. Step 5 - Write your literature review.

  2. Components of the Literature Review

    Literature Review. This is the most time-consuming aspect in the preparation of your research proposal and it is a key component of the research proposal. As described in Chapter 5, the literature review provides the background to your study and demonstrates the significance of the proposed research. Specifically, it is a review and synthesis ...

  3. Writing a Literature Review

    A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays).

  4. PDF How to Write a Literature Review

    academic work. Nonetheless, both the strategies and components of literature reviews vary based on the genre, length, and prospective audience of a text. This resource gives advice on how to effectively assess, synthesize, summarize, and make connections between a variety of sources. THE PURPOSES OF A LITERATURE REVIEW

  5. Literature Reviews

    A literature review can be just a simple summary of the sources, but it usually has an organizational pattern and combines both summary and synthesis. A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information. It might give a new interpretation of old material or ...

  6. How To Write A Literature Review (+ Free Template)

    Okay - with the why out the way, let's move on to the how. As mentioned above, writing your literature review is a process, which I'll break down into three steps: Finding the most suitable literature. Understanding, distilling and organising the literature. Planning and writing up your literature review chapter.

  7. What is a Literature Review?

    A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research. There are five key steps to writing a literature review: Search for relevant literature. Evaluate sources. Identify themes, debates and gaps.

  8. Write a Literature Review

    A literature review may constitute an essential chapter of a thesis or dissertation, or may be a self-contained review of writings on a subject. In either case, its purpose is to: Place each work in the context of its contribution to the understanding of the subject under review. Describe the relationship of each work to the others under ...

  9. PDF Conducting a Literature Review

    Literature Review A literature review is a survey of scholarly sources that provides an overview of a particular topic. ... The basic components of a literature review include: a description of the publication a summary of the publication's main points a discussion of gaps in research an evaluation of the publication's contribution to the ...

  10. Comprehensive Literature Review: A Guide

    A literature review is a collection of selected articles, books and other sources about a specific subject. The purpose is to summarize the existing research that has been done on the subject in order to put your research in context and to highlight what your research will add to the existing body of knowledge. ... Components of a Literature ...

  11. Literature Review: The What, Why and How-to Guide

    Example: Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework: 10.1177/08948453211037398 ; Systematic review: "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139).

  12. How to write a literature review

    Steps in the literature review process. Preparation of a literature review may be divided into four steps: Define your subject and the scope of the review. Search the library catalogue, subject specific databases and other search tools to find sources that are relevant to your topic. Read and evaluate the sources and to determine their ...

  13. How To Structure A Literature Review (Free Template)

    How To Structure Your Literature Review. Like any other chapter in your thesis or dissertation, your literature review needs to have a clear, logical structure. At a minimum, it should have three essential components - an introduction, a body and a conclusion. Let's take a closer look at each of these. 1: The Introduction Section

  14. 5. The Literature Review

    A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories.A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that ...

  15. 3 Essential Components Of A Literature Review

    Literature Review: 3 Essential Ingredients. The theoretical framework, empirical research and research gap. Writing a comprehensive but concise literature review is no simple task. There's a lot of ground to cover and it can be challenging to figure out what's important and what's not. In this post, we'll unpack three essential ...

  16. What is a literature review?

    A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important ...

  17. What are the parts of a Literature Review?

    In a stand-alone literature review, this statement will sum up and evaluate the current state of this field of research; In a review that is an introduction or preparatory to a thesis or research report, it will suggest how the review findings will lead to the research the writer proposes to undertake. Body Purpose:

  18. Literature Review

    In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your ...

  19. Components of literature review

    It is also very important at this point to justify your reasons for writing the review. A body- this will be the bulk of the review and here you will discuss each piece of literature in turn. Research studies should be presented in a logical order e.g. chronological, thematically etc. Previous studies should be summarised and critically ...

  20. PDF Your essential guide to literature reviews

    The basic components of a literature review include: a description of the publication a summary of the publication's main points an evaluation of the publication's contribution to the topic identification of critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches indicates potential directions for ...

  21. PDF 7 Your Literature Review Developing and Presenting distribute

    Once you have completed your literature review, you may want to refer to Appendix B: Rubric for Evaluating a Literature Review. This chapter is divided into two sections. Section I, "Instruction," discusses the purpose and function of the literature review; the role the literature review plays in a dissertation, pointing

  22. The Four-Part Literature Review Process: Breaking It Down for Students

    Breaking down the literature review into a four-part process helps students decrease frustration and increase quality. This article provides usable advice for anyone teaching or writing literature reviews. Tips and illustrations illuminate each part of the process, including 1) Developing a Topic; 2) Searching the Literature; 3) Narrowing the ...

  23. Chapter 9 Methods for Literature Reviews

    9.3. Types of Review Articles and Brief Illustrations. EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic.

  24. Chaos to Clarity: Structuring Your Literature Review Format

    Also Read: Essential Components of a Literature Review. The Introduction Section. The introduction of your literature review is where you set the stage for the entire document. It's your first opportunity to engage your readers and provide a clear blueprint of what your review will cover and why it matters. This section does more than merely ...

  25. Rapid literature review: definition and methodology

    Introduction: A rapid literature review (RLR) is an alternative to systematic literature review (SLR) that can speed up the analysis of newly published data. The objective was to identify and summarize available information regarding different approaches to defining RLR and the methodology applied to the conduct of such reviews.

  26. Composite outcome measures in high-impact critical care randomised

    The use of composite outcome measures (COM) in clinical trials is increasing. Whilst their use is associated with benefits, several limitations have been highlighted and there is limited literature exploring their use within critical care. The primary aim of this study was to evaluate the use of COM in high-impact critical care trials, and compare study parameters (including sample size ...

  27. Effectiveness of educational and psychological survivorship

    Objectives Androgen deprivation therapy (ADT), a common treatment for prostate cancer, has debilitating impacts on physical and psychological quality of life. While some interventions focus on managing the physical side effects of ADT, there is a paucity of interventions that also address psychosocial and educational needs. The objective of this systematic review was to identify psychological ...

  28. The Clinical Definition of Children With Medical Complexity: A Modified

    Through a scoping review of the CMC literature, we identified potential criteria to fulfill each domain. These were incorporated into an electronic survey that was completed by a geographic and professionally varied panel of 81 American and Canadian respondents with expertise in managing CMC (response rate 70%) as part of a 4-iteration Delphi procedure.

  29. The Role of NRF2 in Trinucleotide Repeat Expansion Disorders

    Trinucleotide repeat expansion disorders, a diverse group of neurodegenerative diseases, are caused by abnormal expansions within specific genes. These expansions trigger a cascade of cellular damage, including protein aggregation and abnormal RNA binding. A key contributor to this damage is oxidative stress, an imbalance of reactive oxygen species that harms cellular components. This review ...