how to formulate a research question systematic review

Are animal-assisted therapies as effective as traditional cognitive behavioral therapies in treating people with depressive disorders?

Intervention: the intervention/practice/treatment that you are evaluating (e.g. initial examination of patients by allied health staff)

experiences

no support or "cold turkey"

Children

etting, erspective, ntervention or Phenomenon of nterest, omparison, valuation ( )

ample, henomenon of nterest, esign, valuation, esearch type ( )

spective, etting, henomena of interest/Problem, nvironment, omparison (optional), me/timing, indings ( )

Items # 1, 3, 5, 6, 10, 13, 14, 16: rated or

Items # 2, 4, 7, 8, 9 : rated or

Items # 11 , 12, 15: rated or

24 assessment items: rated

5 items regarding level of concern: rated

 Research Registry-

 Registry of Systematic Reviews/Meta-Analyses

Aggregate data

Individual

participant data

Pairwise comparisons of effect estimates, CI

Overall effect estimate, CI, value

Evaluation of heterogeneity

Box-and-whisker plot, bubble plot

Forest plot (without summary effect estimate)

eMERGe [ ]

ENTREQ [ ]

For RCTs: Cochrane RoB2 [ ]

For NRSI:

ROBINS-I [ ]

Other primary research

Factor review QUIPS [ ]

Model review PROBAST [ ]

CASP qualitative checklist [ ]

JBI Critical Appraisal Checklist [ ]

For NRSI: ROBINS-I [ ]

Other primary research

CERQual [ ]

ConQual [ ]

GRADE (for intervention reviews)

Risk factors

Evidence synthesis with meta-analysis

Systematic review with meta-analysis

Systematic review of systematic reviews

Review of reviews

Meta-review

Single-subject research

N-of-1 design

Quality of evidence

Grade of evidence

Level of evidence

Strength of evidence

Narrative synthesis , narrative summary

Qualitative synthesis

Descriptive synthesis, descriptive summary

Expectation

Client group

Location

Impact

Professionals

Service

Setting

Population/Perspective

Intervention

Comparison

Evaluation

Sample

Phenomenon of Interest

Design

Evaluation

Research type

Chronic disease

Patient-led care models

Peters MDJ, Godfrey C, McInerney P, Munn Z, Tricco AC, Khalil, H. Chapter 11: Scoping Reviews (2020 version). In: Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis, JBI, 2020. Available from   https://synthesismanual.jbi.global  .    https://doi.org/10.46658/JBIMES-20-12

A model useful for qualitative and mixed method type research questions.

Example question: What are young parents’ experiences of attending antenatal education? (Cooke et al., 2012)

Cooke, A., Smith, D. and Booth, A. (2012) 'Beyond PICO: the SPIDER tool for qualitative evidence synthesis.' Qualitative Health Research , 22(10) pp. 1435-1443

A model useful for qualitative and mixed method type research questions. 

Example question: How effective is mindfulness used as a cognitive therapy in a counseling service in improving the attitudes of patients diagnosed with cancer?

Example question taken from: Tate, KJ., Newbury-Birch, D., and McGeechan, GJ. (2018) ‘A systematic review of qualitative evidence of  cancer patients’ attitudes to mindfulness.’ European Journal of Cancer Care , 27(2) pp. 1 – 10.

A model useful for qualitative and mixed method type research questions, especially for question examining particular services or professions.

Example question: Cross service communication in supporting adults with learning difficulties

You might find that your topic does not always fall into one of the models listed on this page. You can always modify a model to make it work for your topic, and either remove or incorporate additional elements.

The important thing is to ensure that you have a high quality question that can be separated into its component parts.

• << Previous: Timescales and processes
• Next: Inclusion and exclusion criteria >>
• Last Updated: Jan 23, 2024 10:52 AM
• URL: https://plymouth.libguides.com/systematicreviews

Systematic Review

• Library Help
• What is a Systematic Review (SR)?
• Steps of a Systematic Review
• Framing a Research Question
• Developing a Search Strategy
• Searching the Literature
• Managing the Process
• Meta-analysis
• Publishing your Systematic Review

Developing a Research Question

_______________________________________________________________________

Watch the 4 min. video on how to frame a research question with PICO.

___ ______ ______________________________________________________________

Frameworks for research questions

Further reading:

Methley, A. M., Campbell, S., Chew-Graham, C., McNally, R., & Cheraghi-Sohi, S. (2014). PICO, PICOS and SPIDER: A comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews.   BMC Health Services Research, 14 (1), 579.

• << Previous: Steps of a Systematic Review
• Next: Developing a Search Strategy >>
• Last Updated: Jul 11, 2024 6:38 AM
• URL: https://lib.guides.umd.edu/SR

Doing a Systematic Review: A Student’s Guide

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

What is Systematic Review?

A systematic review is a comprehensive, structured analysis of existing research on a specific topic. It uses predefined criteria to identify, evaluate, and synthesize relevant studies, aiming to provide an unbiased summary of the current evidence.

The explicit and systematic approach of a systematic review distinguishes it from traditional reviews and commentaries.

Here are some key ways that systematic reviews differ from narrative reviews:

• Goals: Narrative reviews provide a summary or overview of a topic, while systematic reviews answer a focused review question.
• Sources of Literature: Narrative reviews often use a non-exhaustive and unstated body of literature, which can lead to publication bias. Systematic reviews consider a list of databases, grey literature, and other sources.
• Selection Criteria: Narrative reviews usually use subjective or no selection criteria, which can lead to selection bias. Systematic reviews have a clear and explicit selection process.
• Appraisal of Study Quality: Narrative reviews vary in their evaluation of study quality. Systematic reviews use standard checklists for a rigorous appraisal of study quality.

Systematic reviews are time-intensive and need a research team with multiple skills and contributions. There are some cases where systematic reviews are unable to meet the necessary objectives of the review question.

In these cases, scoping reviews (which are sometimes called scoping exercises/scoping studies) may be more useful to consider.

Scoping reviews are different from systematic reviews because they may not include a mandatory critical appraisal of the included studies or synthesize the findings from individual studies.

Assessing The Need For A Systematic Review

When assessing the need for a systematic review, one must first check if any existing or ongoing reviews already exist and determine if a new review is justified.

This process should begin by searching relevant databases.

Resources to consider searching include:

• NICE : National Institute for Health and Clinical Excellence
• Campbell Library of Systematic Reviews for reviews in education, crime and justice, and social welfare
• EPPI : Evidence for Policy and Practice Information Centre, particularly their database of systematic and non-systematic reviews of public health interventions (DoPHER)
• MEDLINE : Primarily covers the medical domain, making it a primary resource for systematic reviews concerning healthcare interventions
• PsycINFO : For research in psychology, psychiatry, behavioral sciences, and social sciences
• Cochrane Library (specifically CDSR) : Focuses on systematic reviews of health care interventions, providing regularly updated and critically appraised reviews

If an existing review addressing the question of interest is found, its quality should be assessed to determine its suitability for guiding policy and practice.

If a high-quality, relevant review is located, but its completion date is some time ago, updating the review might be warranted.

Assessing current relevance is vital, especially in rapidly evolving research fields. Collaboration with the original research team might be beneficial during the update process, as they could provide access to their data.

If the review is deemed to be of adequate quality and remains relevant, undertaking another systematic review may not be necessary.

When a new systematic review or an update is deemed necessary, the subsequent step involves establishing a review team and potentially an advisory group, who will then develop the review protocol.

How To Conduct A Systematic Review

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) is a reporting guideline designed to improve the transparency and completeness of systematic review reporting.

PRISMA was created to tackle the issue of inadequate reporting often found in systematic reviews:

• Checklist : PRISMA features a 27-item checklist covering all aspects of a systematic review, from the rationale and objectives to the synthesis of findings and discussion of limitations. Each checklist item is accompanied by detailed reporting recommendations in an Explanation and Elaboration document .
• Flow Diagram : PRISMA also includes a flow diagram to visually represent the study selection process, offering a clear, standardized way to illustrate how researchers arrived at the final set of included studies.

Step 1: write a research protocol

A protocol in the context of systematic reviews is a detailed plan that outlines the methodology to be employed throughout the review process.

The protocol serves as a roadmap, guiding researchers through each stage of the review in a transparent and replicable manner.

This document should provide specific details about every stage of the research process, including the methodology for identifying, selecting, and analyzing relevant studies.

For example, the protocol should specify search strategies for relevant studies, including whether the search will encompass unpublished works.

The protocol should be created before beginning the research process to ensure transparency and reproducibility.

This pre-determined plan ensures that decisions made during the review are objective and free from bias, as they are based on pre-established criteria.

Protocol modifications are sometimes necessary during systematic reviews. While adhering to the protocol is crucial for minimizing bias, there are instances where modifications are justified. For instance, a deeper understanding of the research question that emerges from examining primary research might necessitate changes to the protocol.

Systematic reviews should be registered at inception (at the protocol stage) for these reasons:

• To help avoid unplanned duplication
• To enable the comparison of reported review methods with what was planned in the protocol

This registration prevents duplication (research waste) and makes the process easy when the full systematic review is sent for publication.

PROSPERO is an international database of prospectively registered systematic reviews in health and social care. Non-Cochrane protocols should be registered on PROSPERO.

Research Protocol

Rasika Jayasekara, Nicholas Procter. The effects of cognitive behaviour therapy for major depression in older adults: a systematic review. PROSPERO 2012 CRD42012003151 Available from:  https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42012003151

Review question

How effective is CBT compared with other interventions, placebo or standard treatment in achieving relapse prevention and improving mental status for older adults with major depression?

The search strategy aims to find both published and unpublished studies and publications. The search will be limited to English language papers published from 2002 to 2012.

A three-step search strategy will be developed using MeSH terminology and keywords to ensure that all materials relevant to the review are captured.

An initial limited search of MEDLINE and CINAHL will be undertaken followed by an analysis of the text words contained in the title and abstract, and of the index terms used to describe the article. A second search using all identified keywords and index terms will then be undertaken.

Thirdly, the reference list of all identified reports and articles will be searched for additional studies.

The databases to be searched included:

• Cochrane Central Register of Controlled Trials
• Controlled Trials
• Current Contents

The search for unpublished studies will include:

• Digital Dissertations (Proquest)
• Conference Proceedings

Experts in the field will be contacted for ongoing and unpublished trials. Experts will be identified through journal publications.

Types of study to be included

All randomised controlled trials (RCTs) assessing the effectiveness of CBT as a treatment for older adults with major depression when compared to standard care, specific medication, other therapies and no intervention will be considered.

In the absence of RCTs, other research designs such as quasi-experimental studies, case-controlled studies and cohort studies will be examined. However, descriptive studies and expert opinion will be excluded.

Condition or domain being studied

Major depression is diagnosed according to DSM IV or ICD 10 criteria.

Where trials fail to employ diagnostic criteria, the severity of depression will be described by the use of standardised rating scales, including the Hamilton Depression Rating Scale, Montgomery and Asberg Rating Scale and the Geriatric Depression Rating Scale.

The trials including participants with an explicit diagnosis of dementia or Parkinson’s disease and other mental illnesses will be excluded.

The review will include trials conducted in primary, secondary, community, nursing homes and in-patient settings.

Participants/population

The review will include trials in which patients are described as elderly, geriatric, or older adults, or in which all patients will be aged 55 or over (many North American trials of older adult populations use a cut-off of 55 years).

The review will include trials with subjects of either sex. Where possible, participants will be categorised as community or long term care residents.

Intervention(s), exposure(s)

The review will focus on interventions designed to assess the effects of CBT for older adults with major depression.

The label cognitive behavioural therapy has been applied to a variety of interventions and, accordingly, it is difficult to provide a single, unambiguous definition.

In order to be classified as CBT the intervention must clearly demonstrate the following components:

• the intervention involves the recipient establishing links between their thoughts, feelings and actions with respect to the target symptom;
• the intervention involves the correction of the person’s misperceptions, irrational beliefs and reasoning biases related to the target symptom.
• – the recipient monitoring his or her own thoughts, feelings and behaviours with respect to the target symptom; and
• – the promotion of alternative ways of coping with the target symptom.

In addition, all therapies that do not meet these criteria (or that provide insufficient information) but are labelled as ‘CBT’ or ‘Cognitive Therapy’ will be included as ‘less well defined’ CBT.

Comparator(s)/control

other interventions, placebo or standard treatment

Main outcome(s)

Primary outcomes

• Depression level as assessed by Hamilton Depression Rating Scale, Montgomery or Asberg Rating Scale or the Geriatric Depression Rating Scale.
• Relapse (as defined in the individual studies)
• Death (sudden, unexpected death or suicide).
• Psychological well being (as defined in the individual studies)

Measures of effect

The review will categorise outcomes into those measured in the shorter term (within 12 weeks of the onset of therapy), medium term (within 13 to 26 weeks of the onset of therapy) and longer term (over 26 weeks since the onset of therapy).

Additional outcome(s)

Secondary outcomes

• Mental state
• Quality of life
• Social functioning
• Hospital readmission
• Unexpected or unwanted effect (adverse effects), such as anxiety, depression and dependence on the relationship with the therapist

Data extraction (selection and coding)

Data will be extracted from papers included in the review using JBI-MAStARI. In this stage, any relevant studies will be extracted in relation to their population, interventions, study methods and outcomes.

Where data are missing or unclear, authors will be contacted to obtain information.

Risk of bias (quality) assessment

All papers selected for retrieval will be assessed by two independent reviewers for methodological validity prior to inclusion in the review.

Since the review will evaluate the experimental studies only, The Joanna Briggs Institute Meta Analysis of Statistics Assessment and Review Instrument (JBI-MAStARI) will be used to evaluate each study’s methodological validity.

If there is a disagreement between the two reviewers, there will be a discussion with the third reviewer to solve the dissimilarity.

Strategy for data synthesis

Where possible quantitative research study results will be pooled in statistical meta-analysis using Review Manager Software from the Cochrane Collaboration.

Odds ratio (for categorical outcome data) or standardised mean differences (for continuous data) and their 95% confidence intervals will be calculated for each study.

Heterogeneity will be assessed using the standard Chi-square. Where statistical pooling is not possible the findings will be presented in narrative form.

Step 2: formulate a research question 

Developing a focused research question is crucial for a systematic review, as it underpins every stage of the review process.

The question defines the review’s nature and scope, guides the identification of relevant studies, and shapes the data extraction and synthesis processes.

It’s essential that the research question is answerable and clearly stated in the review protocol, ensuring that the review’s boundaries are well-defined.

A narrow question may limit the number of relevant studies and generalizability, while a broad question can make it challenging to reach specific conclusions.

PICO Framework

The PICO framework is a model for creating focused clinical research questions. The acronym PICO stands for:

• P opulation/Patient/Problem: This element defines the specific group of people the research question pertains to.
• I ntervention: This is the treatment, test, or exposure being considered for the population.
• C omparison: This is the alternative intervention or control group against which the intervention is being compared.
• O utcome: This element specifies the results or effects of the interventions being investigated

Using the PICO format when designing research helps to minimize bias because the questions and methods of the review are formulated before reviewing any literature.

The PICO elements are also helpful in defining the inclusion criteria used to select sources for the systematic review.

The PICO framework is commonly employed in systematic reviews that primarily analyze data from randomized controlled trials .

Not every element of PICO is required for every research question. For instance, it is not always necessary to have a comparison

Types of questions that can be answered using PICO:

“In patients with a recent acute stroke (less than 6 weeks) with reduced mobility ( P ), is any specific physiotherapy approach ( I ) more beneficial than no physiotherapy ( C ) at improving independence in activities of daily living and gait speed ( O )?

“For women who have experienced domestic violence ( P ), how effective are advocacy programmes ( I ) compared to other treatments ( C ) on improving the quality of life ( O )?”

Etiology/Harm

Are women with a history of pelvic inflammatory disease (PID) ( P ) at higher risk for gynecological cancers ( O ) than women with no history of PID ( C )?

Among asymptomatic adults at low risk of colon cancer ( P ), is fecal immunochemical testing (FIT) ( I ) as sensitive and specific for diagnosing colon cancer ( O ) as colonoscopy ( C )?

Among adults with pneumonia ( P ), do those with chronic kidney disease (CKD) ( I ) have a higher mortality rate ( O ) than those without CKD ( C )?

Alternative Frameworks

• PICOCS : This framework, used in public health research, adds a “ C ontext” element to the PICO framework. This is useful for examining how the environment or setting in which an intervention is delivered might influence its effectiveness.
• PICOC : This framework expands on PICO by incorporating “ C osts” as an element of the research question. It is particularly relevant to research questions involving economic evaluations of interventions.
• ECLIPSE : E xpectations, C lient group, L ocation, I mpact, P rofessionals involved, S ervice, and E valuation. It is a mnemonic device designed to aid in searching for health policy and management information.
• PEO : This acronym, standing for P atient, E xposure, and O utcome, is a variation of PICO used when the research question focuses on the relationship between exposure to a risk factor and a specific outcome.
• PIRD : This acronym stands for P opulation, I ndex Test, R eference Test, and D iagnosis of Interest, guiding research questions that focus on evaluating the diagnostic accuracy of a particular test.
• PFO : This acronym, representing P opulation, P rognostic F actors, and O utcome, is tailored for research questions that aim to investigate the relationship between specific prognostic factors and a particular health outcome.
• SDMO : This framework, which stands for S tudies, D ata, M ethods, and O utcomes, assists in structuring research questions focused on methodological aspects of research, examining the impact of different research methods or designs on the quality of research findings.

Step 3: Search Strategy

PRISMA  (Preferred Reporting Items for Systematic reviews and Meta-Analyses) provide appropriate guidance for reporting quantitative literature searches.

Present the full search strategies for all databases, registers and websites, including any filters and limits used. PRISMA 2020 Checklist

A search strategy is a comprehensive and reproducible plan for identifying all relevant research studies that address a specific research question.

This systematic approach to searching helps minimize bias and distinguishes systematic reviews from other types of literature reviews.

It’s important to be transparent about the search strategy and document all decisions for auditability. The goal is to identify all potentially relevant studies for consideration.

Here’s a breakdown of a search strategy:

Search String Construction

It is recommended to consult topic experts on the review team and advisory board in order to create as complete a list of search terms as possible for each concept.

To retrieve the most relevant results, a search string is used. This string is made up of:

• Keywords:  Search terms should be relevant to the subject areas of the research question and should be identified for all components of the research question (e.g., Population, Intervention, Comparator, and Outcomes – PICO). Using relevant keywords helps minimize irrelevant search returns. Sources such as dictionaries, textbooks, and published articles can help identify appropriate keywords.
• Synonyms: These are words or phrases with similar meanings to the keywords, as authors may use different terms to describe the same concepts. Including synonyms helps cover variations in terminology and increases the chances of finding all relevant studies. For example, a drug intervention may be referred to by its generic name or by one of its several proprietary names.
• Truncation symbols : These broaden the search by capturing variations of a keyword. They function by locating every word that begins with a specific root. For example, if a user was researching interventions for smoking, they might use a truncation symbol to search for “smok*” to retrieve records with the words “smoke,” “smoker,” “smoking,” or “smokes.” This can save time and effort by eliminating the need to input every variation of a word into a database.
• Boolean operators: The use of Boolean operators (AND/OR/NEAR/NOT) helps to combine these terms effectively, ensuring that the search strategy is both sensitive and specific. For instance, using “AND” narrows the search to include only results containing both terms, while “OR” expands it to include results containing either term.

Information Sources

The primary goal is to find all published and unpublished studies that meet the predefined criteria of the research question. This includes considering various sources beyond typical databases

Information sources for systematic reviews can include a wide range of resources like scholarly databases, unpublished literature, conference papers, books, and even expert consultations.

Specify all databases, registers, websites, organisations, reference lists and other sources searched or consulted to identify studies. Specify the date when each source was last searched or consulted. PRISMA 2020 Checklist

An exhaustive, systematic search strategy is developed with the assistance of an expert librarian.

• Electronic Databases:  Searches should include seven key databases: CINAHL, Medline, APA PsycArticles, Psychology and Behavioral Sciences Collection, APA PsycInfo, SocINDEX with Full Text, and Web of Science: Core Collections.
• Grey Literature:  In addition to databases, forensic or ‘expansive’ searches can be conducted. This includes: conference proceedings, unpublished reports, theses, ongoing clinical trial databases, searches by names of authors of relevant publications. Independent research bodies may also be good sources of material, e.g. Centre for Research in Ethnic Relations , Joseph Rowntree Foundation , Carers UK .
• Citation Searching : Reference lists often lead to highly cited and influential papers in the field, providing valuable context and background information for the review.
• Handsearching: Manually searching through specific journals or conference proceedings page-by-page is another way to ensure all relevant studies are captured, particularly those not yet indexed in databases.
• Contacting Experts: Reaching out to researchers or experts in the field can provide access to unpublished data or ongoing research not yet publicly available.

It is important to note that this may not be an exhaustive list of all potential databases.

A systematic computerized search was performed for publications that appeared between 1974 and 2018 in English language journals. Four databases were searched including PsychINFO, Embase, OVOID MEDLINE, and AMED. The databases were searched with combinations of search terms relating to attachment (“attachment” OR “working model” OR “safe haven” OR “secure base” OR “felt security”) AND romantic couples (“dyad” OR “couple” OR “spous” OR “partner” OR “romantic” OR “wife” OR “husband” OR “close relationship” OR “interpersonal” OR “intimate” OR “mari”) AND social support (“support prov” OR “caregiving” OR “support giv” OR “social support” OR “enacted support” OR “support received” OR “receiv* support” OR “prov support” OR “dyadic coping” OR “interpersonal coping” OR “collaborative coping” OR “help‐seeking” OR “emotional support” OR “tangible support” OR “instrumental support” OR “perceived support” OR “responsive” OR “buffer” OR “partner support” OR “Support avail*” OR “available support”). The reference lists of the retrieved studies were checked to find other relevant publications, which were not identified in the computerized database searches.

Inclusion Criteria

Specify the inclusion and exclusion criteria for the review. PRISMA 2020 Checklist

Before beginning the literature search, researchers should establish clear eligibility criteria for study inclusion.

Inclusion criteria are used to select studies for a systematic review and should be based on the study’s research method and PICO elements.

To maintain transparency and minimize bias, eligibility criteria for study inclusion should be established a priori. Ideally, researchers should aim to include only high-quality randomized controlled trials that adhere to the intention-to-treat principle.

The selection of studies should not be arbitrary, and the rationale behind inclusion and exclusion criteria should be clearly articulated in the research protocol.

When specifying the inclusion and exclusion criteria, consider the following aspects:

• Intervention Characteristics: Researchers might decide that, in order to be included in the review, an intervention must have specific characteristics. They might require the intervention to last for a certain length of time, or they might determine that only interventions with a specific theoretical basis are appropriate for their review.
• Population Characteristics: A systematic review might focus on the effects of an intervention for a specific population. For instance, researchers might choose to focus on studies that included only nurses or physicians.
• Outcome Measures: Researchers might choose to include only studies that used outcome measures that met a specific standard.
• Age of Participants: If a systematic review is examining the effects of a treatment or intervention for children, the authors of the review will likely choose to exclude any studies that did not include children in the target age range.
• Diagnostic Status of Participants: Researchers conducting a systematic review of treatments for anxiety will likely exclude any studies where the participants were not diagnosed with an anxiety disorder.
• Study Design: Researchers might determine that only studies that used a particular research design, such as a randomized controlled trial, will be included in the review.
• Control Group: In a systematic review of an intervention, researchers might choose to include only studies that included certain types of control groups, such as a waiting list control or another type of intervention.
• Publication status : Decide whether only published studies will be included or if unpublished works, such as dissertations or conference proceedings, will also be considered.

Studies that met the following criteria were included: (a) empirical studies of couples (of any gender) who are in a committed romantic relationship, whether married or not; (b) measurement of the association between adult attachment and support in the context of this relationship; (c) the article was a full report published in English; and (d) the articles were reports of empirical studies published in peer‐reviewed journals, dissertations, review papers, and conference presentations.

Iterative Process

The iterative nature of developing a search strategy for systematic reviews stems from the need to refine and adapt the search process based on the information encountered at each stage.

A single attempt rarely yields the perfect final strategy. Instead, it is an evolving process involving a series of test searches, analysis of results, and discussions among the review team.

Here’s how the iterative process unfolds:

• Initial Strategy Formulation: Based on the research question, the team develops a preliminary search strategy, including identifying relevant keywords, synonyms, databases, and search limits.
• Test Searches and Refinement: The initial search strategy is then tested on chosen databases. The results are reviewed for relevance, and the search strategy is refined accordingly. This might involve adding or modifying keywords, adjusting Boolean operators, or reconsidering the databases used.
• Discussions and Iteration: The search results and proposed refinements are discussed within the review team. The team collaboratively decides on the best modifications to improve the search’s comprehensiveness and relevance.
• Repeating the Cycle: This cycle of test searches, analysis, discussions, and refinements is repeated until the team is satisfied with the strategy’s ability to capture all relevant studies while minimizing irrelevant results.

The iterative nature of developing a search strategy is crucial for ensuring that the systematic review is comprehensive and unbiased.

By constantly refining the search strategy based on the results and feedback, researchers can be more confident that they have identified all relevant studies.

This iterative process ensures that the applied search strategy is sensitive enough to capture all relevant studies while maintaining a manageable scope.

Throughout this process, meticulous documentation of the search strategy, including any modifications, is crucial for transparency and future replication of the systematic review.

Step 4: Search the Literature

Conduct a systematic search of the literature using clearly defined search terms and databases.

Applying the search strategy involves entering the constructed search strings into the respective databases’ search interfaces. These search strings, crafted using Boolean operators, truncation symbols, wildcards, and database-specific syntax, aim to retrieve all potentially relevant studies addressing the research question.

The researcher, during this stage, interacts with the database’s features to refine the search and manage the retrieved results.

This might involve employing search filters provided by the database to focus on specific study designs, publication types, or other relevant parameters.

Applying the search strategy is not merely a mechanical process of inputting terms; it demands a thorough understanding of database functionalities and a discerning eye to adjust the search based on the nature of retrieved results.

Step 5: screening and selecting research articles

Once the search strategy is finalized, it is applied to the selected databases, yielding a set of search results.

These search results are then screened against pre-defined inclusion criteria to determine their eligibility for inclusion in the review.

The goal is to identify studies that are both relevant to the research question and of sufficient quality to contribute to a meaningful synthesis.

Studies meeting the inclusion criteria are usually saved into electronic databases, such as Endnote or Mendeley , and include title, authors, date and publication journal along with an abstract (if available).

Study Selection

Specify the methods used to decide whether a study met the inclusion criteria of the review, including how many reviewers screened each record and each report retrieved, whether they worked independently, and if applicable, details of automation tools used in the process. PRISMA 2020 Checklist

The selection process in a systematic review involves multiple reviewers to ensure rigor and reliability.

To minimize bias and enhance the reliability of the study selection process, it is recommended that at least two reviewers independently assess the eligibility of each study. This independent assessment helps reduce the impact of individual biases or errors in judgment.

• Initial screening of titles and abstracts: After applying a strategy to search the literature, the next step involves screening the titles and abstracts of the identified articles against the predefined inclusion and exclusion criteria. During this initial screening, reviewers aim to identify potentially relevant studies while excluding those clearly outside the scope of the review. It is crucial to prioritize over-inclusion at this stage, meaning that reviewers should err on the side of keeping studies even if there is uncertainty about their relevance. This cautious approach helps minimize the risk of inadvertently excluding potentially valuable studies.
• Retrieving and assessing full texts: For studies which a definitive decision cannot be made based on the title and abstract alone, reviewers need to obtain the full text of the articles for a comprehensive assessment against the predefined inclusion and exclusion criteria. This stage involves meticulously reviewing the full text of each potentially relevant study to determine its eligibility definitively.
• Resolution of disagreements : In cases of disagreement between reviewers regarding a study’s eligibility, a predefined strategy involving consensus-building discussions or arbitration by a third reviewer should be in place to reach a final decision. This collaborative approach ensures a fair and impartial selection process, further strengthening the review’s reliability.

First, the search results from separate databases were combined, and any duplicates were removed. The lead author (S. M.) and a postgraduate researcher (F. N.) applied the described inclusion criteria in a standardized manner. First, both the titles and abstracts of the articles were evaluated for relevance. If, on the basis of the title and/or abstract, the study looked likely to meet inclusion criteria hard copies of the manuscripts were obtained. If there was doubt about the suitability of an article, then the manuscript was included in the next step. The remaining articles were obtained for full‐text review, and the method and results sections were read to examine whether the article fitted the inclusion criteria. If there was doubt about the suitability of the manuscripts during this phase, then this article was discussed with another author (C. H.). Finally, the reference lists of the eligible articles were checked for additional relevant articles not identified during the computerized search. For the selected articles (n = 43), the results regarding the relationship between attachment and support were included in this review (see Figure 1, for PRISMA flowchart).

PRISMA Flowchart

The PRISMA flowchart is a visual representation of the study selection process within a systematic review.

The flowchart illustrates the step-by-step process of screening, filtering, and selecting studies based on predefined inclusion and exclusion criteria.

The flowchart visually depicts the following stages:

• Identification: The initial number of titles and abstracts identified through database searches.
• Screening: The screening process, based on titles and abstracts.
• Eligibility: Full-text copies of the remaining records are retrieved and assessed for eligibility.
• Inclusion: Applying the predefined inclusion criteria resulted in the inclusion of publications that met all the criteria for the review.
• Exclusion: The flowchart details the reasons for excluding the remaining records.

This systematic and transparent approach, as visualized in the PRISMA flowchart, ensures a robust and unbiased selection process, enhancing the reliability of the systematic review’s findings.

The flowchart serves as a visual record of the decisions made during the study selection process, allowing readers to assess the rigor and comprehensiveness of the review.

Step 6: Criticallay Appraising the Quality of Included Studies

Quality assessment provides a measure of the strength of the evidence presented in a review.

High-quality studies with rigorous methodologies contribute to a more robust and reliable evidence base, increasing confidence in the review’s conclusions.

Conversely, including low-quality studies with methodological weaknesses can undermine the review’s findings and potentially lead to inaccurate recommendations.

To judge the quality of studies included in a systematic review, standardized instruments, such as checklists and scales, are commonly used. These tools help to ensure a transparent and reproducible assessment process.

The choice of tool should be justified and aligned with the study design and the level of detail required. Using quality scores alone is discouraged; instead, individual aspects of methodological quality should be considered.

Here are some specific tools mentioned in the sources:

• Jadad score
• Cochrane Risk of Bias tool
• Cochrane Effective Practice and Organisation of Care (EPOC) Group Risk of Bias Tool
• Quality Assessment of Diagnostic Accuracy Studies (QUADAS)
• Newcastle – Ottawa Quality Assessment Scale for case-control and cohort studies
• EPHPP Assessment Tool
• Critical Appraisal Skills Programme (CASP) Appraisal Checklist
• Cochrane Public Health Group (CPHG)

The quality of the study was not an inclusion criterion; however, a study quality check was carried out. Two independent reviewers (S. M. and C. H.) rated studies that met the inclusion criteria to determine the strength of the evidence. The Effective Public Health Practice Project Quality Assessment Tool for Quantitative Studies was adapted to assess the methodological quality of each study (Thomas, Ciliska, Dobbins, & Micucci, 2004). The tool was adjusted to include domains relevant to the method of each study. For example, blinding was removed for nonexperimental studies. Following recommendations by Thomas et al. (2004) each domain was rated as either weak (3 points), moderate (2 points), or strong (1 point). The mean score across questions was used as an indicator of overall quality, and studies were assigned an overall quality rating of strong (1.00–1.50), moderate (1.51–2.50),

Evidence Tables

Aspects of the appraisal of studies included in the review should be recorded as evidence tables (NICE 2009): simple text tables where the design and scope of studies are summarised.

The reader of the review can use the evidence tables to check the details, and assess the credibility and generalisability of findings, of particular studies.

Critical appraisal of the quality of included studies may be combined with data extraction tables.

Step 7: extracting data from studies

To effectively extract data from studies that meet your systematic review’s inclusion criteria, you should follow a structured process that ensures accuracy, consistency, and minimizes bias.

1. Develop a data extraction form:

• Design a standardized form (paper or electronic) to guide the data extraction process : This form should be tailored to your specific review question and the types of studies included.
• Pilot test the form : Test the form on a small sample of included studies (e.g., 3-5). Assess for clarity, completeness, and usability. Refine the form based on feedback and initial experiences.
• Reliability : Ensure all team members understand how to use the form consistently.

2. Extract the data:

• General Information: This includes basic bibliographic details (journal, title, author, volume, page numbers), study objective as stated by the authors, study design, and funding source.
• Study Characteristics: Capture details about the study population (demographics, inclusion/exclusion criteria, recruitment procedures), interventions (description, delivery methods), and comparators (description if applicable).
• Outcome Data: Record the results of the intervention and how they were measured, including specific statistics used. Clearly define all outcomes for which data are being extracted.
• Risk of Bias Assessment: Document the methods used to assess the quality of the included studies and any potential sources of bias. This might involve using standardized checklists or scales.
• Additional Information: Depending on your review, you may need to extract data on other variables like adverse effects, economic evaluations, or specific methodological details.

3. Dual independent review:

• Ensure that at least two reviewers independently extract data from each study using the standardized form. Cross-check extracted data for accuracy to minimize bias and helps identify any discrepancies.
• Have a predefined strategy for resolving disagreements: This might involve discussion, consensus, or arbitration by a third reviewer.
• Record the reasons for excluding any studies during the data extraction phase. :This enhances the transparency and reproducibility of your review.
• If necessary, contact study authors to obtain missing or clarify unclear information : This is particularly important for data critical to your review’s outcomes.
• Clearly document your entire data extraction process, including any challenges encountered and decisions made. This enhances the transparency and rigor of your systematic review.

By following these steps, you can effectively extract data from studies that meet your inclusion criteria, forming a solid foundation for the analysis and synthesis phases of your systematic review.

Step 8: synthesize the extracted data

The key element of a systematic review is the synthesis: that is the process that brings together the findings from the set of included studies in order to draw conclusions based on the body of evidence.

Data synthesis in a systematic review involves collating, combining, and summarizing findings from the included studies.

This process aims to provide a reliable and comprehensive answer to the review question by considering the strength of the evidence, examining the consistency of observed effects, and investigating any inconsistencies.

The data synthesis will be presented in the results section of the systematic review.

• Develop a clear text narrative that explains the key findings
• Use a logical heading structure to guide readers through your results synthesis
• Ensure your text narrative addresses the review’s research questions
• Use tables to summarise findings (can be same table as data extraction)

Identifying patterns, trends, and differences across studies

Narrative synthesis uses a textual approach to analyze relationships within and between studies to provide an overall assessment of the evidence’s robustness. All systematic reviews should incorporate elements of narrative synthesis, such as tables and text.

Remember, the goal of a narrative synthesis is to go beyond simply summarizing individual studies. You’re aiming to create a new understanding by integrating and interpreting the available evidence in a systematic and transparent way.

Organize your data:

• Group studies by themes, interventions, or outcomes
• Create summary tables to display key information across studies
• Use visual aids like concept maps to show relationships between studies

Describe the studies:

• Summarize the characteristics of included studies (e.g., designs, sample sizes, settings)
• Highlight similarities and differences across studies
• Discuss the overall quality of the evidence

Develop a preliminary synthesis:

• Start by describing the results of individual studies
• Group similar findings together
• Identify overarching themes or trends

Explore relationships:

• Look for patterns in the data
• Identify factors that might explain differences in results across studies
• Consider how study characteristics relate to outcomes

Address contradictions:

• Consider differences in study populations, interventions, or contexts
• Look at methodological differences that might explain discrepancies
• Consider the implications of inconsistent results
• Don’t ignore conflicting findings
• Discuss possible reasons for contradictions

Avoid vote counting:

• Don’t simply tally positive versus negative results
• Instead, consider the strength and quality of evidence for each finding

Assess the robustness of the synthesis:

• Reflect on the strength of evidence for each finding
• Consider how gaps or limitations in the primary studies affect your conclusions
• Discuss any potential biases in the synthesis process

Step 9: discussion section and conclusion

Summarize key findings:.

• Summarize key findings in relation to your research questions
• Highlight main themes or patterns across studies
• Explain the nuances and complexities in the evidence
• Discuss the overall strength and consistency of the evidence
• This provides a clear takeaway message for readers

Consider study quality and context:

• Assess whether higher quality studies tend to show different results
• Examine if findings differ based on study setting or participant characteristics
• This helps readers weigh the relative importance of conflicting findings

Discuss implications:

• For practice: How might professionals apply these findings?
• For policy: What policy changes might be supported by the evidence?
• Consider both positive and negative implications
• This helps translate your findings into real-world applications

Identify gaps and future research:

• Point out areas where evidence is lacking or inconsistent
• Suggest specific research questions or study designs to address these gaps
• This helps guide future research efforts in the field

State strengths and limitations:

• Discuss the strengths of your review (e.g., comprehensive search, rigorous methodology)
• Acknowledge limitations (e.g., language restrictions, potential for publication bias)
• This balanced approach demonstrates critical thinking and helps readers interpret your findings

Minimizing Bias

To reduce bias in a systematic review, it is crucial to establish a systematic and transparent review process that minimizes bias at every stage. Sources provide insights into strategies and methods to achieve this goal.

• Protocol development and publication: Developing a comprehensive protocol before starting the review is essential. Publishing the protocol in repositories like PROSPERO or Cochrane Library promotes transparency and helps avoid deviations from the planned approach, thereby minimizing the risk of bias.
• Transparent reporting: Adhering to reporting guidelines, such as PRISMA, ensures that all essential aspects of the review are adequately documented, increasing the reader’s confidence in the transparency and completeness of systematic review reporting.
• Dual independent review: Employing two or more reviewers independently at multiple stages of the review process (study selection, data extraction, quality assessment) minimizes bias. Any disagreements between reviewers should be resolved through discussion or by consulting a third reviewer. This approach reduces the impact of individual reviewers’ subjective interpretations or errors.
• Rigorous quality assessment: Assessing the methodological quality of included studies is crucial for minimizing bias in the review findings. Using standardized critical appraisal tools and checklists helps identify potential biases within individual studies, such as selection bias, performance bias, attrition bias, and detection bias.
• Searching beyond published literature: Explore sources of “grey literature” such as conference proceedings, unpublished reports, theses, and ongoing clinical trial databases.
• Contacting experts in the field : Researchers can reach out to authors and investigators to inquire about unpublished or ongoing studies
• Considering language bias : Expanding the search to include studies published in languages other than English can help reduce language bias, although this may increase the complexity and cost of the review.

Reading List

• Galante, J., Galante, I., Bekkers, M. J., & Gallacher, J. (2014). Effect of kindness-based meditation on health and well-being: a systematic review and meta-analysis .  Journal of consulting and clinical psychology ,  82 (6), 1101.
• Schneider, M., & Preckel, F. (2017). Variables associated with achievement in higher education: A systematic review of meta-analyses .  Psychological bulletin ,  143 (6), 565.
• Murray, J., Farrington, D. P., & Sekol, I. (2012). Children’s antisocial behavior, mental health, drug use, and educational performance after parental incarceration: a systematic review and meta-analysis .  Psychological bulletin ,  138 (2), 175.
• Roberts, B. W., Luo, J., Briley, D. A., Chow, P. I., Su, R., & Hill, P. L. (2017). A systematic review of personality trait change through intervention .  Psychological bulletin ,  143 (2), 117.
• Chu, C., Buchman-Schmitt, J. M., Stanley, I. H., Hom, M. A., Tucker, R. P., Hagan, C. R., … & Joiner Jr, T. E. (2017). The interpersonal theory of suicide: A systematic review and meta-analysis of a decade of cross-national research.   Psychological bulletin ,  143 (12), 1313.
• McLeod, S., Berry, K., Hodgson, C., & Wearden, A. (2020). Attachment and social support in romantic dyads: A systematic review .  Journal of clinical psychology ,  76 (1), 59-101.

Teaching and Research guides

Systematic reviews.

• Starting the review
• About systematic reviews

Develop your research question

Types of questions, pico framework, spice, spider and eclipse.

• Plan your search
• Sources to search
• Search example
• Screen and analyse
• Further help

A systematic review is an in-depth attempt to answer a specific, focused question in a methodical way.

Start with a clearly defined, researchable  question , that should accurately and succinctly sum up the review's line of inquiry.

A well formulated review question will help determine your inclusion and exclusion criteria, the creation of your search strategy, the collection of data and the presentation of your findings.

It is important to ensure the question:

• relates to what you really need to know about your topic
• is answerable, specific and focused
• should strike a suitable balance between being too broad or too narrow in scope
• has been formulated with care so as to avoid missing relevant studies or collecting a potentially biased result set

Is the research question justified?

• Are healthcare providers, consumers, researchers, and policy makers requiring this evidence for their healthcare decisions?
• Is there a gap in the current literature? The question should be worthy of an answer.
• Has a similar review been done before?

Question types

To help in focusing the question and determining the most appropriate type of evidence consider the type of question. Is there is a study design (eg. Randomized Controlled Trials, Meta-Analysis) that would provide the best answer.

Is your research question to focus on:

• Diagnosis : How to select and interpret diagnostic tests
• Intervention/Therapy : How to select treatments to offer patients that do more good than harm and that are worth the efforts and costs of using them
• Prediction/Prognosis : How to estimate the patient’s likely clinical course over time and anticipate likely complications of disease
• Exploration/Etiology : How to identify causes for disease, including genetics

If appropriate, use a  framework  to help in the development of your research question. A framework will assist in identifying the important concepts in your question.

A good question will combine several concepts. Identifying the relevant concepts is crucial to successful development and execution of your systematic search. Your research question should provide you with a checklist for the main concepts to be included in your search strategy.

Using a framework to aid in the development of a research question can be useful. The more you understand your question the more likely you are to obtain relevant results for your review. There are a number of different frameworks available.

A technique often used in research for formulating a  clinical research question  is the PICO   model. PICO is explored in more detail in this guide. Slightly different versions of this concept are used to search for quantitative and qualitative reviews.

For quantitative reviews-

PICO  = Population, Intervention, Comparison, Outcome

For qualitative reviews-

• Booth, A. (2006). Clear and present questions: Formulating questions for evidence based practice. Library hi tech, 24(3), 355-368.

• Cooke, A., Smith, D., & Booth, A. (2012). Beyond PICO: The SPIDER tool for qualitative evidence synthesis. Qualitative Health Research, 22(10), 1435-1443.

• Wildridge, V., & Bell, L. (2002). How CLIP became ECLIPSE: A mnemonic to assist in searching for health policy/management information. Health Information & Libraries Journal, 19(2), 113-115.
• << Previous: About systematic reviews
• Next: Protocol >>

• Last Updated: Jul 17, 2024 10:40 AM
• URL: https://rmit.libguides.com/systematicreviews

Covidence website will be inaccessible as we upgrading our platform on Monday 23rd August at 10am AEST, / 2am CEST/1am BST (Sunday, 15th August 8pm EDT/5pm PDT) 

How to formulate the review question using PICO. 5 steps to get you started.

Home | Blog | How To | How to formulate the review question using PICO. 5 steps to get you started.

Covidence covers five key steps to formulate your review question using PICO

You’ve decided to go ahead. You have identified a gap in the evidence and you know that conducting a systematic review, with its explicit methods and replicable search, is the best way to fill it – great choice 🙌. 

The review will produce useful information to enable informed decision-making and to improve patient care. Your review team’s first job is to capture exactly what you need to know in a well-formulated review question.

At this stage there is a lot to plan. You might be recruiting people to your review team, thinking about the time-frame for completion and considering what software to use. It’s tempting to get straight on to the search for studies 🏃. 

Take it slowly: it’s vital to get the review question right. A clear and precise question will ensure that you gather the appropriate data to answer your question. Time invested up-front to consider every aspect of the question will pay off once the review is underway. The review question will shape all the subsequent stages in the review, particularly setting the criteria for including and excluding studies, the search strategy, and the way you choose to present the results. So it’s worth taking the time to get this right!

Let’s take a look at five key steps in formulating the question for a standard systematic review of interventions. It’s a process that requires careful thought from a range of stakeholders and meticulous planning. But what if, once you have started the review, you find that you need to tweak the question anyway? Don’t worry, we’ll cover that too ✅.

📌 Consider the audience of the review

Who will use this review? What do they want to know? How do they measure effectiveness? Good review teams partner with the people who will use the evidence and make sure that their research plan (or protocol) asks a question that is relevant and important for patients.

📌 Think about what you already know

How much do you need to know about the topic area at this stage? Ideally, enough to come up with a relevant, useful question but not so much that your knowledge influences the way in which you phrase it. Why? Because setting a review question when you are already familiar with the data can introduce bias by allowing you to direct the question in favour of achieving a particular result. In practice, the review team is very likely to have some knowledge of relevant studies and some preconceived ideas about how the treatments work. That’s fine – and it’s useful – but it’s also good practice to recognise the influence this knowledge and these ideas might have on the choice of question. Issues of bias will come up again as we work through the rest of these steps.

If not enough is known about the subject area to ask a useful question, you might undertake a scoping review . This is a separate exercise from a systematic review and is sometimes used by researchers to map the literature and highlight gaps in the evidence before they start work on a systematic review. 

📌 Use a framework

Faced with a heady mixture of concepts, ideas, aims and outcomes, researchers in every field have come up with question frameworks (and some great backronyms ) to help them. Question frameworks impose order on a complex thought process by breaking down a question into its component parts. A commonly used framework in clinical medicine is PICO:

👦 P opulation (or patients) refers to the characteristics of the people that you want to study. For example, the review might look at children with nocturnal enuresis.

💊 I ntervention is the treatment you are investigating. For example, the review might look at the effectiveness of enuresis alarms.

💊 C omparison, if you decide to use one, is the treatment you want to compare the intervention with. For example, the review might look at the effectiveness of enuresis alarms versus the effectiveness of drug therapy. 

📏 O utcomes are the measures used to assess the effectiveness of the treatment. It’s particularly important to select outcomes that matter to the end users of the review. In this example, a useful outcome might be bedwetting. (Helpfully, some clinical areas use standardised sets of outcomes in their clinical trials to facilitate the comparison of data between studies 👏.) 

But back to bedwetting. In our example, a PICO review question would look something like this:

“In children with nocturnal enuresis (population), how effective are alarms (intervention) versus drug treatments (comparison) for the prevention of bedwetting (outcome)?”

PICO is suitable for reviews of interventions. If you plan to review prognostic or qualitative data, or diagnostic test accuracy, PICO is unlikely to be a suitable framework for your question. In Covidence you can save your PICO for easy reference throughout the screening, extraction and quality assessment phases of your review.

📌 Set the scope

The scope of a review question requires careful thought. To answer the example PICO question above, the review would compare one treatment (alarms) with another (drug therapy). A broader question might consider all the available treatments for nocturnal enuresis in children. The broad scope of this question would still allow the review team to drill down and separate the data into groups of specific treatments later in the review process. And to minimise bias, the intended grouping of data would be pre-specified and justified in the protocol or research plan.

Broader systematic reviews are great because they summarise all the evidence on a given topic in one place. A potential disadvantage is that they can produce a large volume of data that is difficult to manage. 

If the size of the review has started to escalate beyond your comfort zone, you might consider narrowing the scope. This can make the size of the review more manageable, both for the review team and for the reader. But it’s worth examining the motivations for narrowing the scope more closely. Suppose we wanted to define a smaller population in the example PICO question. Is there a good reason (other than to reduce the review team’s workload) to restrict the population to boys with nocturnal enuresis? Or to children under 10 years old? On the basis of what is already known, could the treatment effect be expected to differ by sex or age of the study participants? 🤔 Be prepared to explain your choices and to demonstrate that they are legitimate. 

Some reviews with a narrow scope retrieve only a small number of studies. If this happens, there is a risk that the data collected from these studies might not be enough to produce a useful synthesis or to guide decision-making. It can be frustrating for review teams who have spent time defining the question, planning the methods, and conducting an extensive search to find that their question is unanswerable. This is another reason why it is useful for the review team to have prior knowledge of the subject area and some familiarity with the existing evidence. The Cochrane Handbook contains some useful contingencies for dealing with sparse data .

Covidence can help review teams to save time whatever the scope and size of the review. In Covidence, data can be grouped to the review team’s exact specification for seamless export into data analysis software. The intuitive workflow makes collaboration simple so if one reviewer spots a problem, they can alert the rest of the team quickly and easily.

📌 Adjust if necessary

Systematic reviews follow explicit, pre-specified methods. So it’s no surprise to learn that the review question needs to be considered carefully and explained in detail before the review gets underway. But what about the unknown unknowns – those issues that the review teams will have to deal with later in the process but that they cannot foresee at the outset, no matter how much time they spend on due diligence? 

Clearly, reviews need the agility to control for issues that the project plan did not anticipate – strict adherence to the pre-specified process when a good reason to deviate has come to light would carry its own risks for the quality of the review. So if an initial scan of, for example, the search results indicates that it would be sensible to modify the question, this can be done. The research plan might make explicit the process for dealing with these types of changes. It might also contain plans for sensitivity analysis , to examine whether these choices have any effect on the findings of the review. As mentioned above with regard to scope, it might be difficult to defend a data-driven change to the question. And as before, the issue is the risk of bias and the danger of producing a spurious result.

(Figure 4. Image from Eshun‐Wilson  I, Siegfried  N, Akena  DH, Stein  DJ, Obuku  EA, Joska  JA. Antidepressants for depression in adults with HIV infection. Cochrane Database of Systematic Reviews 2018, Issue 1. Art. No.: CD008525. DOI: 10.1002/14651858.CD008525.pub3. Accessed 27 May 2021.)

This blog post is part of the Covidence series on how to write a systematic review. 

Sign up for a free trial of Covidence today!

Laura Mellor. Portsmouth, UK

Perhaps you'd also like....

Top 5 Tips for High-Quality Systematic Review Data Extraction

Data extraction can be a complex step in the systematic review process. Here are 5 top tips from our experts to help prepare and achieve high quality data extraction.

How to get through study quality assessment Systematic Review

Find out 5 tops tips to conducting quality assessment and why it’s an important step in the systematic review process.

How to extract study data for your systematic review

Learn the basic process and some tips to build data extraction forms for your systematic review with Covidence.

Better systematic review management

Head office, working for an institution or organisation.

Find out why over 350 of the world’s leading institutions are seeing a surge in publications since using Covidence!

Request a consultation with one of our team members and start empowering your researchers: 

By using our site you consent to our use of cookies to measure and improve our site’s performance. Please see our Privacy Policy for more information. 

Jump to navigation

Cochrane Training

Chapter 2: determining the scope of the review and the questions it will address.

James Thomas, Dylan Kneale, Joanne E McKenzie, Sue E Brennan, Soumyadeep Bhaumik

Key Points:

• Systematic reviews should address answerable questions and fill important gaps in knowledge.
• Developing good review questions takes time, expertise and engagement with intended users of the review.
• Cochrane Reviews can focus on broad questions, or be more narrowly defined. There are advantages and disadvantages of each.
• Logic models are a way of documenting how interventions, particularly complex interventions, are intended to ‘work’, and can be used to refine review questions and the broader scope of the review.
• Using priority-setting exercises, involving relevant stakeholders, and ensuring that the review takes account of issues relating to equity can be strategies for ensuring that the scope and focus of reviews address the right questions.

Cite this chapter as: Thomas J, Kneale D, McKenzie JE, Brennan SE, Bhaumik S. Chapter 2: Determining the scope of the review and the questions it will address. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated August 2023). Cochrane, 2023. Available from www.training.cochrane.org/handbook .

2.1 Rationale for well-formulated questions

As with any research, the first and most important decision in preparing a systematic review is to determine its focus. This is best done by clearly framing the questions the review seeks to answer. The focus of any Cochrane Review should be on questions that are important to people making decisions about health or health care. These decisions will usually need to take into account both the benefits and harms of interventions (see MECIR Box 2.1.a ). Good review questions often take time to develop, requiring engagement with not only the subject area, but with a wide group of stakeholders (Section 2.4.2 ).

Well-formulated questions will guide many aspects of the review process, including determining eligibility criteria, searching for studies, collecting data from included studies, structuring the syntheses and presenting findings (Cooper 1984, Hedges 1994, Oliver et al 2017) . In Cochrane Reviews, questions are stated broadly as review ‘Objectives’, and operationalized in terms of the studies that will be eligible to answer those questions as ‘Criteria for considering studies for this review’. As well as focusing review conduct, the contents of these sections are used by readers in their initial assessments of whether the review is likely to be directly relevant to the issues they face.

The FINER criteria have been proposed as encapsulating the issues that should be addressed when developing research questions. These state that questions should be F easible, I nteresting, N ovel, E thical, and R elevant (Cummings et al 2007). All of these criteria raise important issues for consideration at the outset of a review and should be borne in mind when questions are formulated.

A feasible review is one that asks a question that the author team is capable of addressing using the evidence available. Issues concerning the breadth of a review are discussed in Section 2.3.1 , but in terms of feasibility it is important not to ask a question that will result in retrieving unmanageable quantities of information; up-front scoping work will help authors to define sensible boundaries for their reviews. Likewise, while it can be useful to identify gaps in the evidence base, review authors and stakeholders should be aware of the possibility of asking a question that may not be answerable using the existing evidence (i.e. that will result in an ‘empty’ review, see also Section 2.5.3 ).

Embarking on a review that authors are interested in is important because reviews are a significant undertaking and review authors need sufficient commitment to see the work through to its conclusion.

A novel review will address a genuine gap in knowledge, so review authors should be aware of any related or overlapping reviews. This reduces duplication of effort, and also ensures that authors understand the wider research context to which their review will contribute. Authors should check for pre-existing syntheses in the published research literature and also for ongoing reviews in the PROSPERO register of systematic reviews before beginning their own review.

Given the opportunity cost involved in undertaking an activity as demanding as a systematic review, authors should ensure that their work is relevant by: (i) involving relevant stakeholders in defining its focus and the questions it will address; and (ii) writing up the review in such a way as to facilitate the translation of its findings to inform decisions. The GRADE framework aims to achieve this, and should be considered throughout the review process, not only when it is being written up (see Chapter 14 and Chapter 15 ).

Consideration of opportunity costs is also relevant in terms of the ethics of conducting a review, though ethical issues should also be considered primarily in terms of the questions that are prioritized for answering and the way that they are framed. Research questions are often not value-neutral, and the way that a given problem is approached can have political implications which can result in, for example, the widening of health inequalities (whether intentional or not). These issues are explored in Section 2.4.3 and Chapter 16 .

MECIR Box 2.1.a Relevant expectations for conduct of intervention reviews

2.2 Aims of reviews of interventions

Systematic reviews can address any question that can be answered by a primary research study. This Handbook focuses on a subset of all possible review questions: the impact of intervention(s) implemented within a specified human population. Even within these limits, systematic reviews examining the effects of intervention(s) can vary quite markedly in their aims. Some will focus specifically on evidence of an effect of an intervention compared with a specific alternative, whereas others may examine a range of different interventions. Reviews that examine multiple interventions and aim to identify which might be the most effective can be broader and more challenging than those looking at single interventions. These can also be the most useful for end users, where decision making involves selecting from a number of intervention options. The incorporation of network meta-analysis as a core method in this edition of the Handbook (see Chapter 11 ) reflects the growing importance of these types of reviews.

As well as looking at the balance of benefit and harm that can be attributed to a given intervention, reviews within the ambit of this Handbook might also aim to investigate the relationship between the size of an intervention effect and other characteristics, such as aspects of the population, the intervention itself, how the outcome is measured, or the methodology of the primary research studies included. Such approaches might be used to investigate which components of multi-component interventions are more or less important or essential (and when). While it is not always necessary to know how an intervention achieves its effect for it to be useful, many reviews will aim to articulate an intervention’s mechanisms of action (see Section 2.5.1 ), either by making this an explicit aim of the review itself (see Chapter 17 and Chapter 21 ), or when describing the scope of the review. Understanding how an intervention works (or is intended to work) can be an important aid to decision makers in assessing the applicability of the review to their situation. These investigations can be assisted by the incorporation of results from process evaluations conducted alongside trials (see Chapter 21 ). Further, many decisions in policy and practice are at least partially constrained by the resource available, so review authors often need to consider the economic context of interventions (see Chapter 20 ).

2.3 Defining the scope of a review question

Studies comparing healthcare interventions, notably randomized trials, use the outcomes of participants to compare the effects of different interventions. Statistical syntheses (e.g. meta-analysis) focus on comparisons of interventions, such as a new intervention versus a control intervention (which may represent conditions of usual practice or care), or the comparison of two competing interventions. Throughout the Handbook we use the terminology experimental intervention versus comparator intervention. This implies a need to identify one of the interventions as experimental, and is used only for convenience since all methods apply to both controlled and head-to-head comparisons. The contrast between the outcomes of two groups treated differently is known as the ‘effect’, the ‘treatment effect’ or the ‘intervention effect’; we generally use the last of these throughout the Handbook .

A statement of the review’s objectives should begin with a precise statement of the primary objective, ideally in a single sentence ( MECIR Box 2.3.a ). Where possible the style should be of the form ‘To assess the effects of [ intervention or comparison ] for [ health problem ] in [ types of people, disease or problem and setting if specified ]’. This might be followed by one or more secondary objectives, for example relating to different participant groups, different comparisons of interventions or different outcome measures. The detailed specification of the review question(s) requires consideration of several key components (Richardson et al 1995, Counsell 1997) which can often be encapsulated by the ‘PICO’ mnemonic, an acronym for P opulation, I ntervention, C omparison(s) and O utcome. Equal emphasis in addressing, and equal precision in defining, each PICO component is not necessary. For example, a review might concentrate on competing interventions for a particular stage of breast cancer, with stage and severity of the disease being defined very precisely; or alternately focus on a particular drug for any stage of breast cancer, with the treatment formulation being defined very precisely.

Throughout the Handbook we make a distinction between three different stages in the review at which the PICO construct might be used. This division is helpful for understanding the decisions that need to be made:

• The review PICO (planned at the protocol stage) is the PICO on which eligibility of studies is based (what will be included and what excluded from the review).
• The PICO for each synthesis (also planned at the protocol stage) defines the question that each specific synthesis aims to answer, determining how the synthesis will be structured, specifying planned comparisons (including intervention and comparator groups, any grouping of outcome and population subgroups).
• The PICO of the included studies (determined at the review stage) is what was actually investigated in the included studies.

Reaching the point where it is possible to articulate the review’s objectives in the above form – the review PICO – requires time and detailed discussion between potential authors and users of the review. It is important that those involved in developing the review’s scope and questions have a good knowledge of the practical issues that the review will address as well as the research field to be synthesized. Developing the questions is a critical part of the research process. As such, there are methodological issues to bear in mind, including: how to determine which questions are most important to answer; how to engage stakeholders in question formulation; how to account for changes in focus as the review progresses; and considerations about how broad (or narrow) a review should be.

MECIR Box 2.3 . a Relevant expectations for conduct of intervention reviews

2.3.1 Broad versus narrow reviews

The questions addressed by a review may be broad or narrow in scope. For example, a review might address a broad question regarding whether antiplatelet agents in general are effective in preventing all thrombotic events in humans. Alternatively, a review might address whether a particular antiplatelet agent, such as aspirin, is effective in decreasing the risks of a particular thrombotic event, stroke, in elderly persons with a previous history of stroke. Increasingly, reviews are becoming broader, aiming, for example, to identify which intervention – out of a range of treatment options – is most effective, or to investigate how an intervention varies depending on implementation and participant characteristics.

Overviews of reviews (see  Chapter V ), in which multiple reviews are summarized, can be one way of addressing the need for breadth when synthesizing the evidence base, since they can summarize multiple reviews of different interventions for the same condition, or multiple reviews of the same intervention for different types of participants. It may be considered desirable to plan a series of reviews with a relatively narrow scope, alongside an Overview to summarize their findings. Alternatively, it may be more useful – particularly given the growth in support for network meta-analysis – to combine comparisons of different treatment options within the same review (see Chapter 11 ). When deciding whether or not an overview might be the most appropriate approach, review authors should take account of the breadth of the question being asked and the resources available. Some questions are simply too broad for a review of all relevant primary research to be practicable, and if a field has sufficient high-quality reviews, then the production of another review of primary research that duplicates the others might not be a sensible use of resources.

Some of the advantages and disadvantages of broad and narrow reviews are summarized in Table 2.3.a . While having a broad scope in terms of the range of participants has the potential to increase generalizability, the extent to which findings are ultimately applicable to broader (or different) populations will depend on the participants who have actually been recruited into research studies. Likewise, heterogeneity can be a disadvantage when the expectation is for homogeneity of effects between studies, but an advantage when the review question seeks to understand differential effects (see Chapter 10 ).A distinction should be drawn between the scope of a review and the precise questions within, since it is possible to have a broad review that addresses quite narrow questions. In the antiplatelet agents for preventing thrombotic events example, a systematic review with a broad scope might include all available treatments. Rather than combining all the studies into one comparison though, specific treatments would be compared with one another in separate comparisons, thus breaking a heterogeneous set of treatments into narrower, more homogenous groups. This relates to the three levels of PICO, outlined in Section 2.3 . The review PICO defines the broad scope of the review, and the PICO for comparison defines the specific treatments that will be compared with one another; Chapter 3 elaborates on the use of PICOs.

In practice, a Cochrane Review may start (or have started) with a broad scope, and be divided up into narrower reviews as evidence accumulates and the original review becomes unwieldy. This may be done for practical and logistical reasons, for example to make updating easier as well as to make it easier for readers to see which parts of the evidence base are changing. Individual review authors must decide if there are instances where splitting a broader focused review into a series of more narrowly focused reviews is appropriate and implement appropriate methods to achieve this. If a major change is to be undertaken, such as splitting a broad review into a series of more narrowly focused reviews, a new protocol must be written for each of the component reviews that documents the eligibility criteria for each one.

Ultimately, the selected breadth of a review depends upon multiple factors including perspectives regarding a question’s relevance and potential impact; supporting theoretical, biologic and epidemiological information; the potential generalizability and validity of answers to the questions; and available resources. As outlined in Section 2.4.2 , authors should consider carefully the needs of users of the review and the context(s) in which they expect the review to be used when determining the most optimal scope for their review.

Table 2.3.a Some advantages and disadvantages of broad versus narrow reviews

2.3.2 ‘Lumping’ versus ‘splitting’

It is important not to confuse the issue of the breadth of the review (determined by the review PICO) with concerns about between-study heterogeneity and the legitimacy of combining results from diverse studies in the same analysis (determined by the PICOs for comparison).

Broad reviews have been criticized as ‘mixing apples and oranges’, and one of the inventors of meta-analysis, Gene Glass, has responded “Of course it mixes apples and oranges… comparing apples and oranges is the only endeavour worthy of true scientists; comparing apples to apples is trivial” (Glass 2015). In fact, the two concepts (‘broad reviews’ and ‘mixing apples and oranges’) are different issues. Glass argues that broad reviews, with diverse studies, provide the opportunity to ask interesting questions about the reasons for differential intervention effects.

The ‘apples and oranges’ critique refers to the inappropriate mixing of studies within a single comparison, where the purpose is to estimate an average effect. In situations where good biologic or sociological evidence suggests that various formulations of an intervention behave very differently or that various definitions of the condition of interest are associated with markedly different effects of the intervention, the uncritical aggregation of results from quite different interventions or populations/settings may well be questionable.

Unfortunately, determining the situations where studies are similar enough to combine with one another is not always straightforward, and it can depend, to some extent, on the question being asked. While the decision is sometimes characterized as ‘lumping’ (where studies are combined in the same analysis) or ‘splitting’ (where they are not) (Squires et al 2013), it is better to consider these issues on a continuum, with reviews that have greater variation in the types of included interventions, settings and populations, and study designs being towards the ‘lumped’ end, and those that include little variation in these elements being towards the ‘split’ end (Petticrew and Roberts 2006).

While specification of the review PICO sets the boundary for the inclusion and exclusion of studies, decisions also need to be made when planning the PICO for the comparisons to be made in the analysis as to whether they aim to address broader (‘lumped’) or narrower (‘split’) questions (Caldwell and Welton 2016). The degree of ‘lumping’ in the comparisons will be primarily driven by the review’s objectives, but will sometimes be dictated by the availability of studies (and data) for a particular comparison (see Chapter 9 for discussion of the latter). The former is illustrated by a Cochrane Review that examined the effects of newer-generation antidepressants for depressive disorders in children and adolescents (Hetrick et al 2012).

Newer-generation antidepressants include multiple different compounds (e.g. paroxetine, fluoxetine). The objectives of this review were to (i) estimate the overall effect of newer-generation antidepressants on depression, (ii) estimate the effect of each compound, and (iii) examine whether the compound type and age of the participants (children versus adolescents) is associated with the intervention effect. Objective (i) addresses a broad, ‘in principle’ (Caldwell and Welton 2016), question of whether newer-generation antidepressants improve depression, where the different compounds are ‘lumped’ into a single comparison. Objective (ii) seeks to address narrower, ‘split’, questions that investigate the effect of each compound on depression separately. Answers to both questions can be identified by setting up separate comparisons for each compound, or by subgrouping the ‘lumped’ comparison by compound ( Chapter 10, Section 10.11.2 ). Objective (iii) seeks to explore factors that explain heterogeneity among the intervention effects, or equivalently, whether the intervention effect varies by the factor. This can be examined using subgroup analysis or meta-regression ( Chapter 10, Section 10.11 ) but, in the case of intervention types, is best achieved using network meta-analysis (see Chapter 11 ).

There are various advantages and disadvantages to bear in mind when defining the PICO for the comparison and considering whether ‘lumping’ or ‘splitting’ is appropriate. Lumping allows for the investigation of factors that may explain heterogeneity. Results from these investigations may provide important leads as to whether an intervention operates differently in, for example, different populations (such as in children and adolescents in the example above). Ultimately, this type of knowledge is useful for clinical decision making. However, lumping is likely to introduce heterogeneity, which will not always be explained by a priori specified factors, and this may lead to a combined effect that is clinically difficult to interpret and implement. For example, when multiple intervention types are ‘lumped’ in one comparison (as in objective (i) above), and there is unexplained heterogeneity, the combined intervention effect would not enable a clinical decision as to which intervention should be selected. Splitting comparisons carries its own risk of there being too few studies to yield a useful synthesis. Inevitably, some degree of aggregation across the PICO elements is required for a meta-analysis to be undertaken (Caldwell and Welton 2016).

2.4 Ensuring the review addresses the right questions

Since systematic reviews are intended for use in healthcare decision making, review teams should ensure not only the application of robust methodology, but also that the review question is meaningful for healthcare decision making. Two approaches are discussed below:

• Using results from existing research priority-setting exercises to define the review question.
• In the absence of, or in addition to, existing research priority-setting exercises, engaging with stakeholders to define review questions and establish their relevance to policy and practice.

2.4.1 Using priority-setting exercises to define review questions

A research priority-setting exercise is a “collective activity for deciding which uncertainties are most worth trying to resolve through research; uncertainties considered may be problems to be understood or solutions to be developed or tested; across broad or narrow areas” (Sandy Oliver, referenced in Nasser 2018). Using research priority-setting exercises to define the scope of a review helps to prevent the waste of scarce resources for research by making the review more relevant to stakeholders (Chalmers et al 2014).

Research priority setting is always conducted in a specific context, setting and population with specific principles, values and preferences (which should be articulated). Different stakeholders’ interpretation of the scope and purpose of a ‘research question’ might vary, resulting in priorities that might be difficult to interpret. Researchers or review teams might find it necessary to translate the research priorities into an answerable PICO research question format, and may find it useful to recheck the question with the stakeholder groups to determine whether they have accurately reflected their intentions.

While Cochrane Review teams are in most cases reviewing the effects of an intervention with a global scope, they may find that the priorities identified by important stakeholders (such as the World Health Organization or other organizations or individuals in a representative health system) are informative in planning the review. Review authors may find that differences between different stakeholder groups’ views on priorities and the reasons for these differences can help them to define the scope of the review. This is particularly important for making decisions about excluding specific populations or settings, or being inclusive and potentially conducting subgroup analyses.

Whenever feasible, systematic reviews should be based on priorities identified by key stakeholders such as decision makers, patients/public, and practitioners. Cochrane has developed a list of priorities for reviews in consultation with key stakeholders, which is available on the Cochrane website. Issues relating to equity (see Chapter 16 and Section 2.4.3 ) need to be taken into account when conducting and interpreting the results from priority-setting exercises. Examples of materials to support these processes are available (Viergever et al 2010, Nasser et al 2013, Tong et al 2017).

The results of research priority-setting exercises can be searched for in electronic databases and via websites of relevant organizations. Examples are: James Lind Alliance , World Health Organization, organizations of health professionals including research disciplines, and ministries of health in different countries (Viergever 2010). Examples of search strategies for identifying research priority-setting exercises are available (Bryant et al 2014, Tong et al 2015).

Other sources of questions are often found in ‘implications for future research’ sections of articles in journals and clinical practice guidelines. Some guideline developers have prioritized questions identified through the guideline development process (Sharma et al 2018), although these priorities will be influenced by the needs of health systems in which different guideline development teams are working.

2.4.2 Engaging stakeholders to help define the review questions

In the absence of a relevant research priority-setting exercise, or when a systematic review is being conducted for a very specific purpose (for example, commissioned to inform the development of a guideline), researchers should work with relevant stakeholders to define the review question. This practice is especially important when developing review questions for studying the effectiveness of health systems and policies, because of the variability between countries and regions; the significance of these differences may only become apparent through discussion with the stakeholders.

The stakeholders for a review could include consumers or patients, carers, health professionals of different kinds, policy decision makers and others ( Chapter 1, Section 1.3.1 ). Identifying the stakeholders who are critical to a particular question will depend on the question, who the answer is likely to affect, and who will be expected to implement the intervention if it is found to be effective (or to discontinue it if not).

Stakeholder engagement should, optimally, be an ongoing process throughout the life of the systematic review, from defining the question to dissemination of results (Keown et al 2008). Engaging stakeholders increases relevance, promotes mutual learning, improves uptake and decreases research waste (see Chapter 1, Section 1.3.1 and Section 1.3.2 ). However, because such engagement can be challenging and resource intensive, a one-off engagement process to define the review question might only be possible. Review questions that are conceptualized and refined by multiple stakeholders can capture much of the complexity that should be addressed in a systematic review.

2.4.3 Considering issues relating to equity when defining review questions

Deciding what should be investigated, who the participants should be, and how the analysis will be carried out can be considered political activities, with the potential for increasing or decreasing inequalities in health. For example, we now know that well-intended interventions can actually widen inequalities in health outcomes since researchers have chosen to investigate this issue (Lorenc et al 2013). Decision makers can now take account of this knowledge when planning service provision. Authors should therefore consider the potential impact on disadvantaged groups of the intervention(s) that they are investigating on disadvantaged groups, and whether socio-economic inequalities in health might be affected depending on whether or how they are implemented.

Health equity is the absence of avoidable and unfair differences in health (Whitehead 1992). Health inequity may be experienced across characteristics defined by PROGRESS-Plus (Place of residence, Race/ethnicity/culture/language, Occupation, Gender/sex, Religion, Education, Socio-economic status, Social capital, and other characteristics (‘Plus’) such as sexual orientation, age, and disability) (O’Neill et al 2014). Issues relating to health equity should be considered when review questions are developed ( MECIR Box 2.4.a ). Chapter 16 presents detailed guidance on this issue for review authors.

MECIR Box 2.4 . a Relevant expectations for conduct of intervention reviews

2.5 Methods and tools for structuring the review

It is important for authors to develop the scope of their review with care: without a clear understanding of where the review will contribute to existing knowledge – and how it will be used – it may be at risk of conceptual incoherence. It may mis-specify critical elements of how the intervention(s) interact with the context(s) within which they operate to produce specific outcomes, and become either irrelevant or possibly misleading. For example, in a systematic review about smoking cessation interventions in pregnancy, it was essential for authors to take account of the way that health service provision has changed over time. The type and intensity of ‘usual care’ in more recent evaluations was equivalent to the interventions being evaluated in older studies, and the analysis needed to take this into account. This review also found that the same intervention can have different effects in different settings depending on whether its materials are culturally appropriate in each context (Chamberlain et al 2017).

In order to protect the review against conceptual incoherence and irrelevance, review authors need to spend time at the outset developing definitions for key concepts and ensuring that they are clear about the prior assumptions on which the review depends. These prior assumptions include, for example, why particular populations should be considered inside or outside the review’s scope; how the intervention is thought to achieve its effect; and why specific outcomes are selected for evaluation. Being clear about these prior assumptions also requires review authors to consider the evidential basis for these assumptions and decide for themselves which they can place more or less reliance on. When considered as a whole, this initial conceptual and definitional work states the review’s conceptual framework . Each element of the review’s PICO raises its own definitional challenges, which are discussed in detail in the Chapter 3 .

In this section we consider tools that may help to define the scope of the review and the relationships between its key concepts; in particular, articulating how the intervention gives rise to the outcomes selected. In some situations, long sequences of events are expected to occur between an intervention being implemented and an outcome being observed. For example, a systematic review examining the effects of asthma education interventions in schools on children’s health and well-being needed to consider: the interplay between core intervention components and their introduction into differing school environments; different child-level effect modifiers; how the intervention then had an impact on the knowledge of the child (and their family); the child’s self-efficacy and adherence to their treatment regime; the severity of their asthma; the number of days of restricted activity; how this affected their attendance at school; and finally, the distal outcomes of education attainment and indicators of child health and well-being (Kneale et al 2015).

Several specific tools can help authors to consider issues raised when defining review questions and planning their review; these are also helpful when developing eligibility criteria and classifying included studies. These include the following.

• Taxonomies: hierarchical structures that can be used to categorize (or group) related interventions, outcomes or populations.
• Generic frameworks for examining and structuring the description of intervention characteristics (e.g. TIDieR for the description of interventions (Hoffmann et al 2014), iCAT_SR for describing multiple aspects of complexity in systematic reviews (Lewin et al 2017)).
• Core outcome sets for identifying and defining agreed outcomes that should be measured for specific health conditions (described in more detail in Chapter 3 ).

Unlike these tools, which focus on particular aspects of a review, logic models provide a framework for planning and guiding synthesis at the review level (see Section 2.5.1 ).

2.5.1 Logic models

Logic models (sometimes referred to as conceptual frameworks or theories of change) are graphical representations of theories about how interventions work. They depict intervention components, mechanisms (pathways of action), outputs, and outcomes as sequential (although not necessarily linear) chains of events. Among systematic review authors, they were originally proposed as a useful tool when working with evaluations of complex social and population health programmes and interventions, to conceptualize the pathways through which interventions are intended to change outcomes (Anderson et al 2011).

In reviews where intervention complexity is a key consideration (see Chapter 17 ), logic models can be particularly helpful. For example, in a review of psychosocial group interventions for those with HIV, a logic model was used to show how the intervention might work (van der Heijden et al 2017). The review authors depicted proximal outcomes, such as self-esteem, but chose only to include psychological health outcomes in their review. In contrast, Bailey and colleagues included proximal outcomes in their review of computer-based interventions for sexual health promotion using a logic model to show how outcomes were grouped (Bailey et al 2010). Finally, in a review of slum upgrading, a logic model showed the broad range of interventions and their interlinkages with health and socio-economic outcomes (Turley et al 2013), and enabled the review authors to select a specific intervention category (physical upgrading) on which to focus the review. Further resources provide further examples of logic models, and can help review authors develop and use logic models (Anderson et al 2011, Baxter et al 2014, Kneale et al 2015, Pfadenhauer et al 2017, Rohwer et al 2017).

Logic models can vary in their emphasis, with a distinction sometimes made between system-based and process-oriented logic models (Rehfuess et al 2018). System-based logic models have particular value in examining the complexity of the system (e.g. the geographical, epidemiological, political, socio-cultural and socio-economic features of a system), and the interactions between contextual features, participants and the intervention (see Chapter 17 ). Process-oriented logic models aim to capture the complexity of causal pathways by which the intervention leads to outcomes, and any factors that may modify intervention effects. However, this is not a crisp distinction; the two types are interrelated; with some logic models depicting elements of both systems and process models simultaneously.

The way that logic models can be represented diagrammatically (see Chapter 17 for an example) provides a valuable visual summary for readers and can be a communication tool for decision makers and practitioners. They can aid initially in the development of a shared understanding between different stakeholders of the scope of the review and its PICO, helping to support decisions taken throughout the review process, from developing the research question and setting the review parameters, to structuring and interpreting the results. They can be used in planning the PICO elements of a review as well as for determining how the synthesis will be structured (i.e. planned comparisons, including intervention and comparator groups, and any grouping of outcome and population subgroups). These models may help review authors specify the link between the intervention, proximal and distal outcomes, and mediating factors. In other words, they depict the intervention theory underpinning the synthesis plan.

Anderson and colleagues note the main value of logic models in systematic review as (Anderson et al 2011):

• refining review questions;
• deciding on ‘lumping’ or ‘splitting’ a review topic;
• identifying intervention components;
• defining and conducting the review;
• identifying relevant study eligibility criteria;
• guiding the literature search strategy;
• explaining the rationale behind surrogate outcomes used in the review;
• justifying the need for subgroup analyses (e.g. age, sex/gender, socio-economic status);
• making the review relevant to policy and practice;
• structuring the reporting of results;
• illustrating how harms and feasibility are connected with interventions; and
• interpreting results based on intervention theory and systems thinking (see Chapter 17 ).

Logic models can be useful in systematic reviews when considering whether failure to find a beneficial effect of an intervention is due to a theory failure, an implementation failure, or both (see Chapter 17 and Cargo et al 2018). Making a distinction between implementation and intervention theory can help to determine whether and how the intervention interacts with (and potentially changes) its context (see Chapter 3 and Chapter 17 for further discussion of context). This helps to elucidate situations in which variations in how the intervention is implemented have the potential to affect the integrity of the intervention and intended outcomes.

Given their potential value in conceptualizing and structuring a review, logic models are increasingly published in review protocols. Logic models may be specified a priori and remain unchanged throughout the review; it might be expected, however, that the findings of reviews produce evidence and new understandings that could be used to update the logic model in some way (Kneale et al 2015). Some reviews take a more staged approach, pre-specifying points in the review process where the model may be revised on the basis of (new) evidence (Rehfuess et al 2018) and a staged logic model can provide an efficient way to report revisions to the synthesis plan. For example, in a review of portion, package and tableware size for changing selection or consumption of food and other products, the authors presented a logic model that clearly showed changes to their original synthesis plan (Hollands et al 2015).

It is preferable to seek out existing logic models for the intervention and revise or adapt these models in line with the review focus, although this may not always be possible. More commonly, new models are developed starting with the identification of outcomes and theorizing the necessary pre-conditions to reach those outcomes. This process of theorizing and identifying the steps and necessary pre-conditions continues, working backwards from the intended outcomes, until the intervention itself is represented. As many mechanisms of action are invisible and can only be ‘known’ through theory, this process is invaluable in exposing assumptions as to how interventions are thought to work; assumptions that might then be tested in the review. Logic models can be developed with stakeholders (see Section 2.5.2 ) and it is considered good practice to obtain stakeholder input in their development.

Logic models are representations of how interventions are intended to ‘work’, but they can also provide a useful basis for thinking through the unintended consequences of interventions and identifying potential adverse effects that may need to be captured in the review (Bonell et al 2015). While logic models provide a guiding theory of how interventions are intended to work, critiques exist around their use, including their potential to oversimplify complex intervention processes (Rohwer et al 2017). Here, contributions from different stakeholders to the development of a logic model may be able to articulate where complex processes may occur; theorizing unintended intervention impacts; and the explicit representation of ambiguity within certain parts of the causal chain where new theory/explanation is most valuable.

2.5.2 Changing review questions

While questions should be posed in the protocol before initiating the full review, these questions should not prevent exploration of unexpected issues. Reviews are analyses of existing data that are constrained by previously chosen study populations, settings, intervention formulations, outcome measures and study designs. It is generally not possible to formulate an answerable question for a review without knowing some of the studies relevant to the question, and it may become clear that the questions a review addresses need to be modified in light of evidence accumulated in the process of conducting the review.

Although a certain fluidity and refinement of questions is to be expected in reviews as a fuller understanding of the evidence is gained, it is important to guard against bias in modifying questions. Data-driven questions can generate false conclusions based on spurious results. Any changes to the protocol that result from revising the question for the review should be documented at the beginning of the Methods section. Sensitivity analyses may be used to assess the impact of changes on the review findings (see Chapter 10, Section 10.14 ). When refining questions it is useful to ask the following questions.

• What is the motivation for the refinement?
• Could the refinement have been influenced by results from any of the included studies?
• Does the refined question require a modification to the search strategy and/or reassessment of any decisions regarding study eligibility?
• Are data collection methods appropriate to the refined question?
• Does the refined question still meet the FINER criteria discussed in Section 2.1 ?

2.5.3 Building in contingencies to deal with sparse data

The ability to address the review questions will depend on the maturity and validity of the evidence base. When few studies are identified, there will be limited opportunity to address the question through an informative synthesis. In anticipation of this scenario, review authors may build contingencies into their protocol analysis plan that specify grouping (any or multiple) PICO elements at a broader level; thus potentially enabling synthesis of a larger number of studies. Broader groupings will generally address a less specific question, for example:

• ‘the effect of any antioxidant supplement on …’ instead of ‘the effect of vitamin C on …’;
• ‘the effect of sexual health promotion on biological outcomes ’ instead of ‘the effect of sexual health promotion on sexually transmitted infections ’; or
• ‘the effect of cognitive behavioural therapy in children and adolescents on …’ instead of ‘the effect of cognitive behavioural therapy in children on …’.

However, such broader questions may be useful for identifying important leads in areas that lack effective interventions and for guiding future research. Changes in the grouping may affect the assessment of the certainty of the evidence (see Chapter 14 ).

2.5.4 Economic data

Decision makers need to consider the economic aspects of an intervention, such as whether its adoption will lead to a more efficient use of resources. Economic data such as resource use, costs or cost-effectiveness (or a combination of these) may therefore be included as outcomes in a review. It is useful to break down measures of resource use and costs to the level of specific items or categories. It is helpful to consider an international perspective in the discussion of costs. Economics issues are discussed in detail in Chapter 20 .

2.6 Chapter information

Authors: James Thomas, Dylan Kneale, Joanne E McKenzie, Sue E Brennan, Soumyadeep Bhaumik

Acknowledgements: This chapter builds on earlier versions of the Handbook . Mona Nasser, Dan Fox and Sally Crowe contributed to Section 2.4 ; Hilary J Thomson contributed to Section 2.5.1 .

Funding: JT and DK are supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care North Thames at Barts Health NHS Trust. JEM is supported by an Australian National Health and Medical Research Council (NHMRC) Career Development Fellowship (1143429). SEB’s position is supported by the NHMRC Cochrane Collaboration Funding Program. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, the Department of Health or the NHMRC.

2.7 References

Anderson L, Petticrew M, Rehfuess E, Armstrong R, Ueffing E, Baker P, Francis D, Tugwell P. Using logic models to capture complexity in systematic reviews. Research Synthesis Methods 2011; 2 : 33–42.

Bailey JV, Murray E, Rait G, Mercer CH, Morris RW, Peacock R, Cassell J, Nazareth I. Interactive computer-based interventions for sexual health promotion. Cochrane Database of Systematic Reviews 2010; 9 : CD006483.

Baxter SK, Blank L, Woods HB, Payne N, Rimmer M, Goyder E. Using logic model methods in systematic review synthesis: describing complex pathways in referral management interventions. BMC Medical Research Methodology 2014; 14 : 62.

Bonell C, Jamal F, Melendez-Torres GJ, Cummins S. ‘Dark logic’: theorising the harmful consequences of public health interventions. Journal of Epidemiology and Community Health 2015; 69 : 95–98.

Bryant J, Sanson-Fisher R, Walsh J, Stewart J. Health research priority setting in selected high income countries: a narrative review of methods used and recommendations for future practice. Cost Effectiveness and Resource Allocation 2014; 12 : 23.

Caldwell DM, Welton NJ. Approaches for synthesising complex mental health interventions in meta-analysis. Evidence-Based Mental Health 2016; 19 : 16–21.

Cargo M, Harris J, Pantoja T, Booth A, Harden A, Hannes K, Thomas J, Flemming K, Garside R, Noyes J. Cochrane Qualitative and Implementation Methods Group guidance series-paper 4: methods for assessing evidence on intervention implementation. Journal of Clinical Epidemiology 2018; 97 : 59–69.

Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, Howells DW, Ioannidis JPA, Oliver S. How to increase value and reduce waste when research priorities are set. Lancet 2014; 383 : 156–165.

Chamberlain C, O’Mara-Eves A, Porter J, Coleman T, Perlen S, Thomas J, McKenzie J. Psychosocial interventions for supporting women to stop smoking in pregnancy. Cochrane Database of Systematic Reviews 2017; 2 : CD001055.

Cooper H. The problem formulation stage. In: Cooper H, editor. Integrating Research: A Guide for Literature Reviews . Newbury Park (CA) USA: Sage Publications; 1984.

Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Annals of Internal Medicine 1997; 127 : 380–387.

Cummings SR, Browner WS, Hulley SB. Conceiving the research question and developing the study plan. In: Hulley SB, Cummings SR, Browner WS, editors. Designing Clinical Research: An Epidemiological Approach . 4th ed. Philadelphia (PA): Lippincott Williams & Wilkins; 2007. p. 14–22.

Glass GV. Meta-analysis at middle age: a personal history. Research Synthesis Methods 2015; 6 : 221–231.

Hedges LV. Statistical considerations. In: Cooper H, Hedges LV, editors. The Handbook of Research Synthesis . New York (NY): USA: Russell Sage Foundation; 1994.

Hetrick SE, McKenzie JE, Cox GR, Simmons MB, Merry SN. Newer generation antidepressants for depressive disorders in children and adolescents. Cochrane Database of Systematic Reviews 2012; 11 : CD004851.

Hoffmann T, Glasziou P, Boutron I. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ 2014; 348: g1687.

Hollands GJ, Shemilt I, Marteau TM, Jebb SA, Lewis HB, Wei Y, Higgins JPT, Ogilvie D. Portion, package or tableware size for changing selection and consumption of food, alcohol and tobacco. Cochrane Database of Systematic Reviews 2015; 9 : CD011045.

Keown K, Van Eerd D, Irvin E. Stakeholder engagement opportunities in systematic reviews: Knowledge transfer for policy and practice. Journal of Continuing Education in the Health Professions 2008; 28 : 67–72.

Kneale D, Thomas J, Harris K. Developing and optimising the use of logic models in systematic reviews: exploring practice and good practice in the use of programme theory in reviews. PloS One 2015; 10 : e0142187.

Lewin S, Hendry M, Chandler J, Oxman AD, Michie S, Shepperd S, Reeves BC, Tugwell P, Hannes K, Rehfuess EA, Welch V, McKenzie JE, Burford B, Petkovic J, Anderson LM, Harris J, Noyes J. Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR). BMC Medical Research Methodology 2017; 17 : 76.

Lorenc T, Petticrew M, Welch V, Tugwell P. What types of interventions generate inequalities? Evidence from systematic reviews. Journal of Epidemiology and Community Health 2013; 67 : 190–193.

Nasser M, Ueffing E, Welch V, Tugwell P. An equity lens can ensure an equity-oriented approach to agenda setting and priority setting of Cochrane Reviews. Journal of Clinical Epidemiology 2013; 66 : 511–521.

Nasser M. Setting priorities for conducting and updating systematic reviews [PhD Thesis]: University of Plymouth; 2018.

O’Neill J, Tabish H, Welch V, Petticrew M, Pottie K, Clarke M, Evans T, Pardo Pardo J, Waters E, White H, Tugwell P. Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. Journal of Clinical Epidemiology 2014; 67 : 56–64.

Oliver S, Dickson K, Bangpan M, Newman M. Getting started with a review. In: Gough D, Oliver S, Thomas J, editors. An Introduction to Systematic Reviews . London (UK): Sage Publications Ltd.; 2017.

Petticrew M, Roberts H. Systematic Reviews in the Social Sciences: A Practical Guide . Oxford (UK): Blackwell; 2006.

Pfadenhauer L, Gerhardus A, Mozygemba K, Lysdahl KB, Booth A, Hofmann B, Wahlster P, Polus S, Burns J, Brereton L, Rehfuess E. Making sense of complexity in context and implementation: the Context and Implementation of Complex Interventions (CICI) framework. Implementation Science 2017; 12 : 21.

Rehfuess EA, Booth A, Brereton L, Burns J, Gerhardus A, Mozygemba K, Oortwijn W, Pfadenhauer LM, Tummers M, van der Wilt GJ, Rohwer A. Towards a taxonomy of logic models in systematic reviews and health technology assessments: a priori, staged, and iterative approaches. Research Synthesis Methods 2018; 9 : 13–24.

Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. ACP Journal Club 1995; 123 : A12–13.

Rohwer A, Pfadenhauer L, Burns J, Brereton L, Gerhardus A, Booth A, Oortwijn W, Rehfuess E. Series: Clinical epidemiology in South Africa. Paper 3: Logic models help make sense of complexity in systematic reviews and health technology assessments. Journal of Clinical Epidemiology 2017; 83 : 37–47.

Sharma T, Choudhury M, Rejón-Parrilla JC, Jonsson P, Garner S. Using HTA and guideline development as a tool for research priority setting the NICE way: reducing research waste by identifying the right research to fund. BMJ Open 2018; 8 : e019777.

Squires J, Valentine J, Grimshaw J. Systematic reviews of complex interventions: framing the review question. Journal of Clinical Epidemiology 2013; 66 : 1215–1222.

Tong A, Chando S, Crowe S, Manns B, Winkelmayer WC, Hemmelgarn B, Craig JC. Research priority setting in kidney disease: a systematic review. American Journal of Kidney Diseases 2015; 65 : 674–683.

Tong A, Sautenet B, Chapman JR, Harper C, MacDonald P, Shackel N, Crowe S, Hanson C, Hill S, Synnot A, Craig JC. Research priority setting in organ transplantation: a systematic review. Transplant International 2017; 30 : 327–343.

Turley R, Saith R, Bhan N, Rehfuess E, Carter B. Slum upgrading strategies involving physical environment and infrastructure interventions and their effects on health and socio-economic outcomes. Cochrane Database of Systematic Reviews 2013; 1 : CD010067.

van der Heijden I, Abrahams N, Sinclair D. Psychosocial group interventions to improve psychological well-being in adults living with HIV. Cochrane Database of Systematic Reviews 2017; 3 : CD010806.

Viergever RF. Health Research Prioritization at WHO: An Overview of Methodology and High Level Analysis of WHO Led Health Research Priority Setting Exercises . Geneva (Switzerland): World Health Organization; 2010.

Viergever RF, Olifson S, Ghaffar A, Terry RF. A checklist for health research priority setting: nine common themes of good practice. Health Research Policy and Systems 2010; 8 : 36.

Whitehead M. The concepts and principles of equity and health. International Journal of Health Services 1992; 22 : 429–25.

For permission to re-use material from the Handbook (either academic or commercial), please see here for full details.

Systematic review

• How we can help
• What is a systematic review?
• Should I conduct a systematic review or a scoping review?
• How do I develop a research question for systematic review?
• Checking for existing systematic reviews
• Do I need to register a protocol?
• What sources should I search?
• How do I develop a search strategy?
• How can I evaluate the quality of my search strategy?
• How can I record my search strategy?
• How can I manage the research?
• How can I find the full text of journal articles?
• Recommended reading

How do I develop a research question for systematic reveiw?

Once you have identified a topic to investigate you need to define a clear and answerable question for the systematic review. Some initial scoping searches will be needed to check the question is:

• Manageable - not too broad or too narrow 
• Answerable - relevant literature is available 
• Has not already been done recently  

You may need to explore a few different versions of the question depending on what you find during these initial searches. The question must be clearly defined and it may be useful to use a research question framework such as PICO (population, intervention, comparison, outcome) or SPICE (setting, perspective, intervention, comparison, evaluation) to help structure both the question and the search terms. For more information and examples of research question frameworks visit our guide to research question frameworks. 

• Research question frameworks
• << Previous: Should I conduct a systematic review or a scoping review?
• Next: Checking for existing systematic reviews >>
• Last Updated: Jul 23, 2024 12:19 PM
• URL: https://gcu.libguides.com/systematicreview

Systematic Reviews & Meta-Analysis

Identifying your research question.

• Developing Your Protocol
• Conducting Your Search
• Screening & Selection
• Data Extraction & Appraisal
• Meta-Analyses
• Writing the Systematic Review
• Suggested Readings

The first step in performing a Systematic Review is to develop your research question. The guidance provided on how to develop your research question for literature reviews will still apply here. The difference with a systematic review research question is that you must have a clearly defined question and consider what problem are you trying to address by conducting the review. The most important point is that you focus your question and design the question so that it is answerable by the research that you will be systematically examining.

Once you have developed your research question, it should not be changed once the review process has begun, as the review protocol needs to be formed around the question. 

To help develop and focus your research question you may use one of the question frameworks below.

Methods for Refining a Research Topic

PICO questions can be useful in the health or social sciences. PICO stands for:

• Patient, Population, or Problem : What are the characteristics of the patient(s) or population, i.e. their ages, genders, or other demographics? What is the situation, disease, etc., that you are interested in?
• Intervention or Exposure : What do you want to do with the patient, person, or population (i.e. observe, diagnose, treat)?
• Comparison : What is the alternative to the intervention (i.e. a different drug, a different assignment in a classroom)?
• Outcome : What are the relevant outcomes (i.e. complications, morbidity, grades)?

Additionally, the following are variations to the PICO framework:

• PICO(T) : The 'T' stands for Timing, where you would define the duration of treatment and the follow-up schedule that matter to patients. Consider both long- and short-term outcomes.
• PICO(S) : The 'S' stands for Study type (eg. randomized controlled trial), sometimes S can be used to stand for Setting or Sample Size

PPAARE is a useful question framework for patient care:

Problem -  Description of the problem related to the disease or condition

Patient - Description of the patient related to their demographics and risk factors

Action  - Description of the action related to the patient’s diagnosis, diagnostic test, etiology, prognosis, treatment or therapy, harm, prevention, patient ed.

Alternative - Description of the alternative to the action when there is one. (Not required)

Results   -   Identify the patient’s result of the action to produce, improve, or reduce the outcome for the patient

Evidence   -   Identify the level of evidence available after searching

SPIDER is a useful question framework for qualitative evidence synthesis:

Sample  - The group of participants, population, or patients being investigated. Qualitative research is not easy to generalize, which is why sample is preferred over patient.

Phenomenon of Interest - The reasons for behavior and decisions, rather than an intervention.

Design  - The research method and study design used for the research, such as interview or survey.

Evaluation  - The end result of the research or outcome measures.

Research type -   The research type; Qualitative, quantitative and/or mixed methods.

SPICE is a particularly useful method in the social sciences. It stands for

• Setting (e.g. United States)
• Perspective (e.g. adolescents)
• Intervention (e.g. text message reminders)
• Comparisons (e.g. telephone message reminders)
• Evaluation (e.g. number of homework assignments turned in after text message reminder compared to the number of assignments turned in after a telephone reminder)

CIMO is useful method in the social sciences or organisational context. It stands for

• Context - Which individuals, relationships, institutional settings, or wider systems are being studied?
• Intervention - The effects of what event, action, or activity are being studied?
• Mechanism - What are the mechanisms that explain the relationship between interventions and outcomes? Under what circumstances are these mechanisms activated or not activated?
• Outcomes - What are the effects of the intervention? How will the outcomes be measured? What are the intended and unintended effects?

Has Your Systematic Review Already Been Done?

Once you have a reasonably well defined research question, it is important to check if your question has already been asked, or if there are other systematic reviews that are similar to that which you're preparing to do.

In the context of conducting a review, even if you do find one on your topic, it may be sufficiently out of date or you may find other defendable reasons to undertake a new or updated one. In addition, locating an existing systematic reviews may also provide a starting point for selecting a review topic, it may help you refocus your question, or redirect your research toward other gaps in the literature.

You may locate existing systematic reviews or protocols on the following resources:

• Cochrane Library This link opens in a new window The Cochrane Library is a database collection containing high-quality, independent evidence, including systematic reviews and controlled trials, to inform healthcare decision-making. Terms of Use .
• MEDLINE (EBSCO) This link opens in a new window Medline (EBSCO) produced by the U.S. National Library of Medicine is the premier database of biomedicine and health sciences, covering life sciences including biology, environmental science, marine biology, plant and animal science, biophysics and chemistry. Terms of Use . Coverage: 1950-present.

• PsycINFO This link opens in a new window Contains over 5 million citations and summaries of peer-reviewed journal articles, book chapters, and dissertations from the behavioral and social sciences in 29 languages from 50 countries. Terms of Use . Coverage: 1872-present.
• << Previous: Systematic Reviews
• Next: Developing Your Protocol >>
• Last Updated: Jun 28, 2024 10:04 AM
• URL: https://libguides.chapman.edu/systematic_reviews

• Keele University

What is best practice for conducting narrative synthesis for systematic literature review?

Top contributors to discussions in this field.

• Tikrit University

• Al-Furat Al-Awsat Technical University

• Indian Veterinary Research Institute

• Institute for Industrial Research & Toxicology

Get help with your research

Join ResearchGate to ask questions, get input, and advance your work.

All Answers (3)

Similar questions and discussions

• Asked 10 April 2024

• Asked 23 February 2024

• Shall we update the time intervals?
• Should we update and rewrite both the findings and the article?
• Asked 27 November 2023

• Asked 2 May 2024

• Asked 3 November 2016

• Asked 1 August 2017

• Asked 17 August 2017

• Asked 14 October 2017

• Asked 18 October 2017

Related Publications

• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

• Open access
• Published: 22 July 2024

Artificial intelligence in total and unicompartmental knee arthroplasty

• Umile Giuseppe Longo   ORCID: orcid.org/0000-0003-4063-9821 1 , 2 ,
• Sergio De Salvatore 3 , 4 ,
• Federica Valente 2 ,
• Mariajose Villa Corta 2 ,
• Bruno Violante 5 &
• Kristian Samuelsson 2  

BMC Musculoskeletal Disorders volume  25 , Article number:  571 ( 2024 ) Cite this article

Metrics details

The application of Artificial intelligence (AI) and machine learning (ML) tools in total (TKA) and unicompartmental knee arthroplasty (UKA) emerges with the potential to improve patient-centered decision-making and outcome prediction in orthopedics, as ML algorithms can generate patient-specific risk models. This review aims to evaluate the potential of the application of AI/ML models in the prediction of TKA outcomes and the identification of populations at risk.

An extensive search in the following databases: MEDLINE, Scopus, Cinahl, Google Scholar, and EMBASE was conducted using the PIOS approach to formulate the research question. The PRISMA guideline was used for reporting the evidence of the data extracted. A modified eight-item MINORS checklist was employed for the quality assessment. The databases were screened from the inception to June 2022.

Forty-four out of the 542 initially selected articles were eligible for the data analysis; 5 further articles were identified and added to the review from the PUBMED database, for a total of 49 articles included. A total of 2,595,780 patients were identified, with an overall average age of the patients of 70.2 years ± 7.9 years old. The five most common AI/ML models identified in the selected articles were: RF, in 38.77% of studies; GBM, in 36.73% of studies; ANN in 34.7% of articles; LR, in 32.65%; SVM in 26.53% of articles.

This systematic review evaluated the possible uses of AI/ML models in TKA, highlighting their potential to lead to more accurate predictions, less time-consuming data processing, and improved decision-making, all while minimizing user input bias to provide risk-based patient-specific care.

Peer Review reports

Introduction

Artificial intelligence (AI) and Machine learning (ML) tools in knee arthroplasty (KA) have the potential to improve patient-centered decision-making and outcome prediction in orthopedics. The application of ML in KA has been useful for predicting implant size, reconstructing data, and assisting with component positioning and alignment. ML implementation enhances surgical precision and can help predict parameters such as length of hospitalization, healthcare costs, and discharge disposition [ 1 , 2 , 3 ]. 

Additionally, ML algorithms have been proven, in more recent studies, to be useful when selecting the right drugs to treat prosthetic joint infection (PJI) to have a more patient specific approach to medicine; this was possible due to the development of a Random Forest (RF) model able to take notice of several risk variables, such as patients’ characteristics and comorbidities and using the, for the selection [ 4 ]. In data science theory, the quantity and quality of input parameters are crucial; therefore, the previously mentioned variables, if not selected by relevance to the topic of each study, although beneficial in theory, may hinder the full potential of ML algorithms for KA. This is because, analyzing all underlying relations between variables, with a large number of inputs the models may highlight irrelevant patterns, leading to a greater risk of overfitting: the algorithms perform significantly better with the training data in respect to the newly presented one [ 4 , 5 ].

Moreover, patient satisfaction following primary KA is one of many outcome measures currently used to assess the efficacy of this procedure. Patients’ satisfaction is dependent on many factors such as age, gender, and the presence of comorbidities. Therefore, it is essential to understand the relationship between the variables underlying satisfaction to provide the best care and optimized postoperative care for KA patients. ML algorithms, capable of generating patient-specific risk models, appear to be very effective means to achieve this goal [ 6 ].

Overall, the application and use of ML and AI in orthopaedics are beneficial not only for the previously mentioned situations, but also for the identification of possible patients that are at high risk for severe walking limitations post-total knee arthroplasty [ 7 ], and the selection of high-risk patients who will require a blood transfusion after KA [ 8 ].

This review will focus on investigating which predictions are achievable by using AI and ML models in knee arthroplasty, identifying prerequisites for the effective use of this new approach. Moreover, the second aim is to highlight the latest findings of these technologies in predicting outcomes after KA.

Materials and methods

Study selection.

The research question was defined by using a PIO approach: Population (P); Intervention (I); Comparison (C); Outcome (O). The objective of this systematic review was to investigate which outcomes can be assessed by using AI or ML models (I) in patients with knee osteoarthritis who underwent total (TKA) or unicompartmental (UKA) knee replacement (P). The following outcomes were considered: complications, costs, functional outcomes, revision rate, and postoperative satisfaction (O).

Inclusion criteria

Only articles that evaluated AI/ML-based applications in clinical decision-making in knee arthroplasty were considered. Only original clinical studies written in English, Spanish, or Italian were screened.

Exclusion criteria

Studies that did not evaluate AI/ML applications in KA. Studies with nonhuman subjects. Medical imaging analysis studies without explicit reference or application to KA. Inaccessible articles, conference abstracts, reviews, and editorials. No limits were placed on the level of evidence or publication date of the study.

Following the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) guidelines, a thorough literature search was conducted using the following string: ((((total) OR (unicompartmental or unicondylar)) AND (knee replacement)) AND (((artificial intelligence) OR (machine learning)) OR (algorithm))) AND ((((((((complications) OR ((blood) AND ((transfusion) OR (loss)))) OR (functional outcomes)) OR (revision)) OR (satisfaction)) OR (surgical technique)) OR ((length of stay) OR (hospitalization))) OR ((costs) OR (economic analysis))). The use of keywords was both combined and isolated. The following databases were used: MEDLINE (Medical Literature Analysis and Retrieval System Online), Scopus, Cinahl, Google Scholar, PUBMED, and EMBASE (Excerpta Medica Database). The reference lists of selected systematic reviews [ 2 , 5 ] were searched for the selection of further studies. The authors (F.V. and M.V.C.) searched from June of 2022 to January 2024. The databases were screened from the inception to January 2024.

Data collection process

Two independent reviewers (F.V. and M.V.C.) collected the data, and mutual approval resolved differences. A third reviewer (S.D.S) was consulted in case of any disagreement. Title and abstract screening were the first steps, followed by the full-text evaluation of the selected articles. The inclusion and exclusion of the reviewed studies were displayed in the PRISMA flowchart, seen in Fig.  1 .

Prisma flowchart

A database was developed by collecting and categorizing the general study characteristics from the selected articles, which comprised: primary author, year of publication, study design, level of evidence, study duration, AI/ML methods, data source, input variables, output variables, sample size, average patient age, percentage of female patients, Area Under the Receiving Operating Characteristic Curve (AUC-ROC), accuracy, sensitivity, specificity.

Risk of bias assessment

For the quality assessment, a modified eight-item Methodological Index for Non-Randomized Studies (MINORS) checklist was employed to evaluate the selected articles. The eight-item checklist included: disclosure, study aim, input feature, output feature, validation method, dataset distribution, performance metric, and AI model. Each item was scored using the following binary scale: 0 (not reported or unclear) and 1 (reported and adequate). The following criteria were used as a guide when assessing the quality of each publication: 

Disclosure: Scored 1 if clearly reported possible conflicts of interest, funding, or ethical considerations, scored 0 if not reported or unclear. Study aim: scored 1 if the research question and/or objective were clearly reported, scored 0 if unclear or not reported. Input feature: scored 1 if variables were clearly reported, scored 0 if unclear or not reported. Output feature: scored 1 if clearly reported, scored 0 if unclear or not reported. The validation method involves the evaluation of the AI/ML model’s performance by specific methods: scored 1 if the tools external validation, cross-validation, and/or bootstrapping were used and clearly reported, scored 0 if not reported nor used. Dataset distribution: scored 1 if the phases of training, testing, and validation for the AI/ML methods were clearly reported, scored 0 if unclear or not reported. Performance metric: scored 1 if the study clearly reported the metrics accuracy, sensitivity, specificity, and/or AUC-ROC for assessing the AI/ML model performance, scored 0 if unclear or not reported. AI model: scored 1 if clearly stated the specific AI/ML algorithm used by the study, scored 0 if not clearly stated.

Compared to the original MINORS checklist, this modified version, proposed by [ 9 ], provides a more accurate grading tool for studies focused on applying AI/ML methods in medical research and diagnostic studies within the medical field. Two independent reviewers (F.V. and M.V.C.) evaluated each publication individually.

The initial search identified 654 studies. After the duplicate removal, 479 studies were screened from which 402 articles were excluded after the title/abstract examination, resulting in 77 records for the full-text evaluation. After the full-text assessment, 49 studies were included in the data analysis (Fig.  1 ). Of these excluded articles, 9 studies did not evaluate AI/ML application in knee arthroplasty, 8 were medical imaging analysis studies without explicit reference or applications to knee arthroplasty, 2 used non-human objects, and 9 were inaccessible articles or systematic reviews.

Study characteristics

A total of 2,595,780 patients were identified from 48 of the 49 studies included, with one study [ 10 ] not providing the sample size. Thirty-seven of the 49 studies stated the percentage of female patients, adding up to 1,435,218 female patients, which account for 55.29% of the total patients. The overall average age of the patients was 70.2 years ± 7.9 years old, with 33 out of 49 articles providing an average age of the study population. The study which had the highest number of patients was Hyer et al., 2020 [ 11 ] with 1,049,160 patients (40.41% of all the patients included in the studies). All the study characteristics are reported in Table  1 .

The five most common AI/ML models used were: RF, used in 19 articles; Gradient Boosting Machine (GBM), used in 18 articles (including less generalized versions such as Extreme Gradient Boosting (XGBoost) and Stochastic Gradient Boosting (SGB)); Artificial Neural Network (ANN) used in 17 articles; Logistic regression (LR), used in 16 articles (together with less generalized versions such as Elastic-net penalized logistic regression (EPLR)); and Support Vector Machine (SVM) used in 13 articles.

Regarding the variables reported, the most common input variables were: Age [ 38 , 41 , 45 , 47 , 49 , 50 , 52 ] (44 articles), Sex (33 articles), Comorbidities (29 articles), BMI (27 articles), Race/ ethnicity (26 articles), ASA classification (10 articles). The most common output variables provided by the studies were: post-surgical complications (11 articles), Probability of TKA (7 articles), and length of stay (4 articles).

This review included studies with level of evidence II-IV. Level of evidence II studies consist of Randomized controlled trials (RCTs) and are considered one of the strongest study designs, second only to reviews and meta-analysis which are considered as level of evidence I; Level of evidence III studies are composed of non-randomized controlled trials; the last category of evidence included in the review is Level IV: Case–control studies assessing associations between exposure and outcome.

The following level of evidence was included in the selected articles: 37 level III retrospective cohort studies [ 6 , 8 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 40 , 48 , 51 ]; three level III diagnostic studies [ 20 , 21 , 22 , 54 ]; three level II prospective cohort studies [ 4 , 39 , 53 ]; one level II comparative studies [ 46 ]; three level IV cohort pilot studies [ 42 , 44 , 55 ], one level III multi-center retrospective study [ 47 ]. One study [ 43 ] did not present the level of evidence. All the characteristics are reported in Tables 1 , 2 and 3 .

AI and ML methods

The following section reports the AI and ML methods identified in the reviewed articles. Each section includes the number of articles that used each AI or ML method, its corresponding AUC value, and the evaluated output variable. Table 4 classifies each article regarding the output variable studies and presents the highest AUC score for the respective article.

Random forest

RF is a decision trees-based algorithm introduced in the 2000s and capable of handling a variety of data types; its implementation in many medical fields is sustained by its high performance with large datasets and its ability to integrate both clinical and imaging data to achieve more accurate predictions compared to older models such as LR. This ML method operates by constructing and averaging a multitude of decision tress, a simpler ML method, with each of the tress randomly analyzing selected subset variations of the original data, the model is capable to analyze large and complex subset of data, resulting in a more resistant model to overfitting, while also adding diversity in the analysis. It was the most common AI method, applied in 38.77% of the reviewed articles. Mainly it was used to evaluate outcomes, one of them being a technical outcome: TKA component size prediction (femoral and tibial) [ 35 ]. Eight publications implemented RF for the evaluation of clinical outcomes, some of them being: achievement of Minimal Clinically Important Differences (MCIDs), prediction of Patient Reported Outcomes (PROs), prolonged postoperative opioid prescription, improvement of Knee injury and Osteoarthritis Outcome Score (KOOS) to one-year, dissatisfaction, assessment of sensitization in patients with chronic pain after TKA, etc. [ 4 , 6 , 20 , 26 , 32 , 33 , 45 , 53 ]. Only one article evaluated the post-walking limitation with RF, under the functional outcome category [ 7 , 56 ].

RF was also utilized to analyze the surgical technique by two articles [ 15 , 49 ], which considered the following outputs respectively: characterization of anatomical tissues and surgical corrections, the latter presenting the highest AUC (0.89) for this ML method. Postoperative length of stay (LOS) was predicted using RF only by one article [ 57 ], which presented an AUC of 0.71.

Another application of RF was regarding possible complications such as major complications after primary TKA, blood transfusion, surgical site infection, and disposition of patients at discharge [ 15 , 25 , 38 ]. Lastly, two reviewed articles implemented RF for predicting TKA risk depending on knee OA, evaluating both risk and time [ 16 , 27 ].

Gradient boosting machine

The ML model GBM gained popularity in the 2000s due to the model’s high predictive accuracy even in settings with mixed data types and missing values. GBM works by building decision tress sequentially, rather than in parallel like RF, with each of the tress correcting the predicting errors made by the previous ones. This results in the model being able to analyse complex relationships in data and producing an accurate prediction, even if lacking the randomized selection or diversity of the RF model. It can be used for both classification and regression due to its ability to produce new decision trees by correcting the errors of the previous predictions, gaining more accuracy than popularly used models such as SVM.

It was used by 18 studies, one employing it to predict TKA component size [ 35 ]. The highest AUC value was applied by an article that evaluated the development of acute kidney infection (AKI) after TKA, AUC: 0.89 [ 34 ]. Other studies that evaluated complications with GBM comprised the following outputs: major complications after primary TKA, blood transfusion after TKA, surgical site infection, and disposition of patients at discharge [ 8 , 17 , 38 ]. One study used GBM for the prediction of LOS after TKA [ 37 ], a different study employed this method to evaluate functional outcome: post-TKA walking limitations [ 7 ].

In addition, GB was used by 7 articles to evaluate clinical outcomes: prediction of patient satisfaction, achievement of MCIDs in KOOS 1 year after TKA, prediction of PROs, extended prescription of postoperative opioids, MCIDs attainment 2 years after TKA [ 6 , 18 , 19 , 22 , 26 , 32 , 33 , 53 ]. Only one study evaluated the use of SGB to predict the risk of TKA in comparison to other ML models, resulting in the highest performance together with RF among the algorithms observed, with an AUC: 0.83 [ 16 ].

Artificial Neural Network (ANN) /Multilayer perceptron

Although it originated in the 1940s, the ANN model gained prominence in the 2010s due to the application of deep learning in modeling complex relationships, making it suitable for a wide range of applications. ANN is a computational algorithm consisting of interconnected nodes organized in sequential layers, each analyzing the data to pass the prediction to the following one, mimicking the functioning of human neural network. This model was applied by 17 studies, one of them being for the prediction of LOS, inpatient charges, and discharge disposition before primary TKA [ 43 ]. Five articles analyzed clinical outcomes, the one having the highest AUC for this method (0.86) was regarding the prediction of PROs [ 26 ]; other outputs under this category were: prolonged postoperative opioid prescription, dissatisfaction after TKA, prediction of same-day discharge in patients undergoing TKA [ 6 , 32 , 33 , 50 ]. One article applied ANN for TKA component size prediction (femoral and tibial) [ 44 ], and another study applied it for procedural cost prediction for TKA [ 31 , 58 ].

Regarding complications, ANN was applied to evaluate the disposition of patients at discharge, post-surgical complications such as surgical site infection, and blood transfusion [ 38 ]. Additionally, two articles used this ML method to characterize tissues and surgical corrections based on patient-specific intra-operative assessment [ 15 , 49 ]. Another application of ANN, by four other articles, was related to future clinical intervention outputs: effect of opioid use in risk of knee revision and manipulation in the first year after primary TKA [ 59 ]; identification of influential factors before surgery, and prediction of the risk of TKA surgery [ 23 , 60 ].

Logistic regression

LR is a simply interpretable model for binary classification developed in the early twentieth century; being one of the oldest predictive models, its role is well established in the medical setting to estimate the probability of occurrence of different events. Although, it is to be considered that the advent of newer algorithms able to form wider and more complex associations between inputs and outputs causes this model to be more frequently relegated to a comparator role. The algorithm was used by 16 out of 49 articles. Four articles evaluated complications, which comprised the following outputs: disposition of patient at discharge, predictors of Allogenic Blood Transfusion (ALBT), and post-surgical complications [ 17 , 25 , 38 ]. The future clinical intervention was studied by three articles, specifically regarding the risk and time for a TKA in a patient presenting knee OA [ 27 ]. One article used this machine learning method for TKA component size prediction [ 35 ], and a different publication used it to evaluate post-TKA walking limitations, a type of functional outcome [ 7 ].

Regarding clinical outcome, LR was applied by 7 articles to study: achievement of MCIDs in KOOS 1 year after TKA, extended opioid prescription post-surgery, dissatisfaction after TKA, assessment of sensitization in patients with chronic pain after TKA, prediction of same-day discharge in patients undergoing TKA, and prediction of PROs [ 6 , 22 , 26 , 32 , 33 , 45 , 50 ]. The article that presented the highest AUC (0.88) evaluated the probability of TKA within 5 years [ 47 ].

Support vector machine

SVM is an effective model which can be used for both classification and regression; developed in the 1960s it still is one of the most popular algorithms used to classify disease progression based on imaging data. However, due to its low accuracy in performances with noisy datasets, newly developed algorithms such as K-Nearest Neighbors (kNN) are gaining prominence in this role. SVM is particularly effective when the number of features exceeds the number of samples in the data, being able to handle both linear and non-linear relationships in data. It was used by 13 articles, one of them evaluating the prediction of LOS and complications after TKA [ 29 , 51 ]. Mainly to assess clinical outcomes such as: prolonged postoperative opioid prescription [ 32 ]; improvement of KOOS one year after TKA [ 33 ]; dissatisfaction after TKA [ 6 ]; attainment of MCIDs 2 years after TKA [ 20 , 53 ]. SVM was also employed to analyze subtask segmentation of the TUG test for perioperative TKA [ 24 ]; Risk and Time of TKA in patients with knee OA [ 16 , 27 ]; surgical corrections based on patient-specific intra-operative evaluation [ 49 ]. Additionally, one article used the algorithm to evaluate the characterization of tissues [ 15 , 60 ] while another applied SVM in component sizing for TKA [ 35 ].

Other AI models

Two AI models were employed to evaluate major complications after primary TKA [ 17 ]: AutoPrognosis (AP) and AdaBoost. The ML method Decision tree was utilized in two studies for the analysis of the following outputs: gait comparison between UKA and TKA patient [ 30 ], and subtask segmentation of TUG test for perioperative TKA, the latter also being assessed by the methods: AdaBoost, kNN, Naïve Bayes Classifier (NB) [ 24 ].

Regarding the analysis of post-TKA walking limitation, the model SuperLearner was used [ 7 ]. Both the Cox-PH model and DeepSurv model were used to predict the risk and time of TKA in patients with knee osteoarthritis [ 27 ]; an Ensemble Deep Learning (DL) model based on the use of MRI and radiograph was also compared with traditional ML algorithms to predict the risk of TKA, obtaining promising results [ 40 ]. The prediction of PROs was assessed by the models: NB, kNN, and Multi-Step Adaptive Elastic-Net (MSAENET) [ 26 ].

The models Quadratic Discriminant Analysis (QDA) and LASSO regression were employed to evaluate MCIDs attainment after TKA in different periods. One of the studies made the assessment 1 year after TKA [ 22 ], other two articles made the evaluation 2 years after TKA [ 20 , 53 ]. LASSO regression was also used to analyze mortality and complication after TKA, such as respiratory, cardiovascular, and nervous system and renal complications [ 21 ]. Regarding the prediction of clinical outcomes, the new Skeletal Oncology Research Group Machine Learning Algorithm (SORG-MLA) was validated for the identification of patients at risk of prolonged postoperative opioid use after TKA, obtaining an AUC: 0.75 [ 48 ].

Moreover, the models Linear Discriminant Analysis (LDA), Recursive Partitioning (RP), and NB were employed for the assessment of sensitization in patients with chronic pain after TKA [ 1 ]. The prediction of procedural cost after TKA, the DenseNet was used, presenting an AUC score of 0.813 [ 31 ].

Natural Language Processing Method (NLPM) was utilized to assess surgical technique, using the following outputs: category of surgery, implant model, presence of patellar resurfacing, constraint type, and laterality of surgery [ 46 ]. NLPM was also used to estimate ITS data [ 4 ] and analyze the alteration that opioid use can have in risk of knee revision and manipulation in the first year after primary TKA [ 12 ].

Lastly, the Stochastic Hill Climbing Complexity score was for the prediction of surgical 90-day morbidity, mortality, and complications [ 11 ]. NB was employed to analyze inpatient cost and LOS after TKA [ 32 , 45 ].

Quality assessment by modified MINORS

All 49 of the reviewed articles were evaluated following the modified MINORS checklist to assess quality and risk of bias. All 49 articles clearly reported the study aim, however, 11 studies failed to report the performance metric. Two publications did not report the output feature, while 46 of the studies clearly stated the input feature, and 45 of the articles indicated disclosure. Regarding the item AI model, 45 of the reviewed articles fulfilled this criterion. These findings showed a relatively high degree of quality and low likelihood of bias, only two of the reviewed articles received a score of 5/8, five articles with 6/8 as a score, and the majority, 42 out of 49 publications, scored 7/8 and higher (Table  5 ).

This systematic review evaluated the possible uses of AI/ML models in TKA, highlighting their potential in improving decision-making, component sizing, inpatient costs, perioperative planning, and streamlining the surgical workflow. Implementing these prediction models in TKA can ultimately lead to more accurate predictions, less time-consuming data processing, and higher precision in identifying patterns, all while minimizing user input bias to provide risk-based patient-specific care.

A key finding was the benefits of RF in aiding surgical decision-making when applied in intraoperatively collected surface models and patient-specific intraoperative assessments. RF outperformed both ANN and SVM not only when categorizing various types of anatomical tissue [ 15 ], but also when identifying populations at risk for TKA [ 16 ], and assessing balance and alignment during TKA surgery, aiding the surgeon regarding the optimal choice for the suitable bone recut or soft tissue adjustment [ 49 , 61 ]. This review highlights how the application of RF in all the steps leading to TKA, perioperative and postoperative care can lead to optimal clinical and surgical outcomes, while reducing complications thanks to patient-specific planning. Moreover, by streamlining the surgical workflow and helping to select surgical corrections, this AI model can overcome the risk of data overload and the challenge of data interpretation, while being fast, cost-efficient, and accurate.

The SGB model presented promising results in the Kunze et al. (2021) study, by outranking RF, SVM, and EPLR for the prediction of the component sizing of the implant used in TKA. This model demonstrated the best overall performance regarding minimizing prediction error and maximizing accuracy for both femoral and tibial implant component size prediction. A potential benefit is an ability to predict final component sizes of the prosthetic without reliance on digital or manual templating, therefore being faster than traditional methods. Also, showing good performance across different TKA component manufacturers, streamlining component selection processes, improving inventory control, and reducing shipping costs [ 35 , 62 ].

Regarding prediction models for allogenic blood transfusion, the highest AUC score was reported by the RF and SVM-based models [ 25 ]. With a slightly lower difference of 0.038 in the AUC score, the ANN-based model was still significantly higher than the classic prediction models [ 38 ]. Overall, these results show how the implementation of various ML-based models can result in an improvement of peri-operative complications predictions, ensuring that the identified population at risk, for blood transfusion, receives proper care while also optimizing the operative process and reducing the risk of prolonged LOS, caused by complications, such as blood transfusion, during TKA.

A further finding is the already established importance of LR models when used in healthcare settings, which can lead to the development of patient-specific care and peri-operative planning. The most successful result of LR (AUC 0.88) was achieved by its implementation, together with DenseNet, in identifying a population at higher risk of TKA within 5 years, particularly at less advanced stages of OA [ 47 ]; although, in the more recent study published by Crawford et al. in 2023, compared to other models such as SGB and RF (AUC: 0.83), EPLR scored a lower performance in identification of population at risk of TKA [ 16 ]. Additionally, implementing LR with other models, like the ML-based remote patient monitoring system, can reduce the need for TKA revision, while acquiring continuous data for patients undergoing TKA, in terms of mobility and rehabilitation compliance. This patient monitoring system proved to be reliable, low-maintenance, and a well-received platform for the patient recovering from TKA [ 42 ]. Implementing LR models would result in higher objectivity, cost-effectiveness, and ability to acquire continuous data, together with higher accuracy in identifying at-risk population, overall increasing the success rate for TKA.

Financial aspects are to be considered when proposing a treatment plan to patients, as complications can arise during the surgery and recovery, drastically changing the cost expected beforehand. Although it was shown to be an important element to consider when planning peri-operative care during TKA, the cost-prediction outcome was only analysed in one article. Demonstrating high accuracy when used in clinical medicine, the DenseNet model [ 31 , 63 ] can optimize and provide a cost-efficient organization of resources that can benefit the medical staff by reducing their workload and improving the quality of the arrangement of resources. Simultaneously, this method can identify populations at risk for complications, a benefit that would help reduce the higher cost of the procedure after TKA, making it possible to implement patient-specific payment plans benefitting both patients and healthcare providers.

Going over the performances of the GBM model analysed in different articles, we can observe how this algortihm is simple and efficient, it has been validated to improve both short- and long-term prognoses of TKA patients. Ko et al. successfully used this AI model for the prediction of the development of postoperative AKI after TKA, which can not only increase LOS but also be life-threatening [ 34 ]; while TreeNet GBM proved to be the most successful method when applied for predictors regarding patient satisfaction [ 18 ]. Additionally, GBM showed great results when predicting the disposition of patients at discharge [ 38 ], therefore the model’s implementation could improve the overall patient satisfaction and recovery rate post-TKA, while also assuring patient-specific peri-operative care is applied to prevent and manage possible complications.

Looking at more novel models less implemented up until recently in the healthcare settings, the following AI/ML models: DL-TL-MT, SVM, Deep Surv, and Cox-PH, proved to be of great use to individuate the population of patients at risk and develop patient-specific care. The DL-TL-MT model successfully predicted the risk of OA progression based on knee radiographs in patients that previously underwent TKA [ 36 ]. Presenting the same AUC level (of 0.87), the methods SVM, Deep Surv, and Cox-PH were successfully employed to predict the risk and time of TKA of an OA knee [ 27 ]. The implementation methods prove to be indispensable in predicting the progression of OA, even at an early stage. This ML-based model has great potential as a diagnostic tool for physicians when determining the prognosis for patient at all stages of OA, allowing for early intervention through TKA where needed, therefore reducing the risk of complications and of TKA revision.

The SVM predictor model showed also a very promising results when applied in the different settings, and especially for the segmentation of the TUG test and extraction of information from each subtask perioperative to TKA, solving the problems regarding subjectiveness and other biases [ 24 , 64 ]. The benefits that come with the usage of this AI model would be a more precise segmentation and therefore data extraction, which results in further understanding and classification of improvements in patients, leading to the employment of patient-specific treatments and rehabilitation models.

Looking at the results of the different articles involved in the review, the emergence of ML models in the medical setting becomes an evident matter: most data corroborates the idea that novel AI models present better results and predictive powers, compared to traditional models, when identifying predictors of TKA and analyzing multiple outcomes simultaneously. In the prediction of complications after primary TKA, Devana et al. prove the superiority of AP, compared to traditional models, regarding the discriminative ability and the capability to suggest nonlinear relationships between variables in the outcomes of TKA. Consequently, AP can be a versatile tool that may be utilized for the identification of crucial patient characteristics when predicting outcomes across a variety of datasets, thereby improving the patient outcomes [ 17 ]. Additionally, Harris et al. demonstrated how AI can produce preoperative prediction models for one-year improvement in pain and functioning after TKA; and how the GBM model, which performs well in important interactions, and the QDA model, which performs better in nonlinear association, can be applied to produce an easy-to-use predictive model able to achieve similar or better accuracy with far fewer inputs in respect to traditional predictive models [ 22 ].

Lastly, the NLPM model presents great potential as a newly emerging algorithm, in particular when applied in clinical settings for the interpretation of a text, which has been applied in different studies for the classification of patient satisfaction [ 14 ], knee revisions after TKA due to preoperative opioid use [ 12 ], and for the processing of clinical free text from electronic health records [ 46 ]. The strength of this ML-based model relies on its ability to automate the extraction of embedded information in perioperative notes and patient-centered surveys, decreasing the need for costly manual chart reviewing and improving data quality while being less time-consuming. The use of this model would improve patient feedback and perioperative notes to better patient-specific risk-based care resulting in higher patient satisfaction and a reduction in costs for the healthcare system due to possible lawsuits [ 65 ], together with the reduction of the cost due to manual chart reviews [ 46 ].

Like both the Hinterwimmer et al. 2021, and the Lee et al. 2022 review, this systematic review confirms the great potential of AI/ML methods and their application in orthopedics for cost predictions, diagnostic applications, and identification of risk factors, while also clearing the doubts regarding the inaccuracy and lack of sufficient evaluation of these models. In comparison, this review analyzed 49 articles, including the publications already examined in previous reviews. This more extensive research concluded that not only is it possible to implement these models in the prediction of TKA perioperative care, disease progression of OA, and distinct outcomes applying specific data, but also the prediction of more complex outcomes is now feasible through the application of more novel AI/ML algorithms [ 13 , 17 , 21 , 22 , 27 , 30 ]. Although, as mentioned in several studies, further research may enhance the reliability of AI/ML models and allow for their use in patient preoperative and perioperative care [ 8 , 11 , 19 , 21 , 43 , 50 ].

Limitations

The main limitation of this review derives from the possible bias of information regarding the performance of the different AI models, which, as highlighted by the MINORS table, results as the most at-risk parameter due to the omission by several articles of either AUC score or Accuracy score for the different predictive models examined. Moreover, many of the studies included in this review are retrospective studies obtaining the data, regarding the patients for the testing of the AI/ML prediction models, from national databases and electronic health recordings; limitations by the lack of detailed clinical information, potential misclassification of data, and in many cases a small cohort of patients presenting limited characteristics from which to derive input and compare outputs, which may lead to the results not being generalizable to all patient populations [ 11 , 19 , 21 ]. Validation of analyzed predictive models on larger populations of patients is needed. Lastly, due to the heterogeneity between data, it was not possible to perform a meta-analysis.

Regarding the implementation of AI/ML models in TKA, the articles in this review mostly consider these predictive models to be helpful and suggest that their application in medical settings for perioperative TKA clinical decision-making and prediction of the progression of OA into TKA may result in an improvement of patient satisfaction, risk managing, and cost efficiency. Among the best qualities, for which the AI/ML models outperform the traditional prediction models, frequently reported higher accuracy, cost efficiency, simple application, lack of subjectiveness, and overall reduction of time consumption thanks to the automation of tasks. Therefore, it is possible to conclude that, although the results of the reviewed articles should be further validated by their testing on larger cohorts of patients, the findings of these articles highlight the great potentials that derive from the inclusion of AI/ML predictive models in a further branch of medicine.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Adaptive Boosting

Artificial Intelligence

Acute Kidney Infection

Allogenic Blood Transfusion

Artificial Neural Network

Artificial Neural Network – Transfer Learning – multitask

AutoPrognosis

All patient refined

American society of anesthesiologists’ physical status

America Society of Anesthesiologists

Area Under the Receiver Operating Characteristic Curve

Body mass index

Blood pressure

Blood transfusion

Cartilage and bone classification

Charlson comorbidity score

Centers for Medicare & Medicaids Services

Cox proportional hazards

Cohort pilot study

Comparative study

Deep Convolutional Neural Network

Discharge disposition

Densely Connected Convolutional Network

Deep Learning

Disposition of patient

Decision tree

Elastic-net Penalized Logistic Regression

European Society of Sports Traumatology, Knee Surgery and Arthroscopy

Gradient Boosting Machine

Heamoglobin

Home exercise program

Hip-knee-angle

Impatient charges/costs

Knee Artrhoplasty

K-Nearest Neighbors

Knee Injury and Osteoarthritis Outcome Score

Knee injury and Osteoarthritis Outcome Score for Joint Replacement

Knee society score function

Knee society score

Least Absolute Shrinkage and Selection Operator

Linear Discriminant Analysis

Length of stay

Logistic Regression

Muscle, cartilage, bone classification

Minimally clinically important differences

Multi center retrospective study

Mechanical circulatory support

Methodological Index for Non-Randomized Studies

• Machine Learning

Multilayer perceptron

Multi-Step Adaptive Elastic NETwork

Muscle, tendon, bone, cartilage classification

Multi-Task Logistic Regression

Näive-Bayes

National Health Service

National (Nationwide) Inpatient Sample database

Natural Language Processing Method

Neural Network

Non-Linear-Group Method of Data Handling

Non-steroidal anti-inflammatory drugs

National Surgical Quality Improvement Program

Osteoarthritis

Osteoarthritis initiative

Orthopedic Minimal Data Set database

Office of Statewide Health Planning and Development

Physical activity scale for the Elderly

Posterior cruciate ligament

Physical component summary

Preoperative patient-reported health state

Population Intervention Comparison Outcome

Prosthetic Joint Infection

Preferred Reporting Items for Systematic reviews and Meta-Analysis

Patient reported outcome measures

Prediction of patient-reported outcomes

Pilot study

Quadrant Discriminant Analysis

Quality of Life

Retrospective cohort study

Randomized control trial

Random Forest

Recursive Partitioning

Remote Patient Monitoring

Standard analytical files

Short form – physical component summary

Stochastic Gradient Boosting

Stochastic Hill Climbing

Skeletal Oncology Research Group Machine Learning Algorithm

State-wide Planning and Research Cooperative System

Support Vector Machines

Total Knee Arthroplasty

Timed Up-and-Go

University of California Los Angeles

Unicompartmental Knee Arthroplasty

Veteran’s affairs

Visual analog scale

World Health Organization

Western Ontario and McMaster Universities Osteoarthritis Index

EXtreme Gradient Boosting

Cabitza F, Locoro A, Banfi G. Machine learning in orthopedics: a literature review. Front Bioeng Biotechnol. 2018;6:75.

Article   PubMed   PubMed Central   Google Scholar  

Lee LS, Chan PK, Wen C, Fung WC, Cheung A, Chan VWK, Cheung MH, Fu H, Yan CH, Chiu KY. Artificial intelligence in diagnosis of knee osteoarthritis and prediction of arthroplasty outcomes: a review. Arthroplasty. 2022;4(1):16.

Martín Noguerol T, Paulano-Godino F, Martín-Valdivia MT, Menias CO, Luna A. Strengths, weaknesses, opportunities, and threats analysis of artificial intelligence and machine learning applications in radiology. J Am Coll Radiol. 2019;16(9 Pt B):1239–47.

Article   PubMed   Google Scholar  

Shohat N, Goswami K, Tan TL, Yayac M, Soriano A, Sousa R, Wouthuyzen-Bakker M, Parvizi J. (NINJA) ESGoIAIEatNINoJA: 2020 Frank Stinchfield award: identifying who will fail following irrigation and debridement for prosthetic joint infection. Bone Joint J. 2020;102-B(7_Supple_B):11–9.

Hinterwimmer F, Lazic I, Suren C, Hirschmann MT, Pohlig F, Rueckert D, Burgkart R, von Eisenhart-Rothe R. Machine learning in knee arthroplasty: specific data are key-a systematic review. Knee Surg Sports Traumatol Arthrosc. 2022;30(2):376–88.

Kunze KN, Polce EM, Sadauskas AJ, Levine BR. Development of machine learning algorithms to predict patient dissatisfaction after primary total knee arthroplasty. J Arthroplasty. 2020;35(11):3117–22.

Pua YH, Kang H, Thumboo J, Clark RA, Chew ES, Poon CL, Chong HC, Yeo SJ. Machine learning methods are comparable to logistic regression techniques in predicting severe walking limitation following total knee arthroplasty. Knee Surg Sports Traumatol Arthrosc. 2020;28(10):3207–16.

Jo C, Ko S, Shin WC, Han HS, Lee MC, Ko T, Ro DH. Transfusion after total knee arthroplasty can be predicted using the machine learning algorithm. Knee Surg Sports Traumatol Arthrosc. 2020;28(6):1757–64.

Ogink PT, Groot OQ, Karhade AV, Bongers MER, Oner FC, Verlaan JJ, Schwab JH. Wide range of applications for machine-learning prediction models in orthopedic surgical outcome: a systematic review. Acta Orthop. 2021;92(5):526–31.

Bonakdari H, Pelletier JP, Martel-Pelletier J. A reliable time-series method for predicting arthritic disease outcomes: new step from regression toward a nonlinear artificial intelligence method. Comput Methods Programs Biomed. 2020;189:105315.

Hyer JM, White S, Cloyd J, Dillhoff M, Tsung A, Pawlik TM, Ejaz A. Can we improve prediction of adverse surgical outcomes? Development of a surgical complexity score using a novel machine learning technique. J Am Coll Surg. 2020;230(1):43–52.e41.

Ben-Ari A, Chansky H, Rozet I. Preoperative opioid use is associated with early revision after total knee arthroplasty: a study of male patients treated in the veterans affairs system. J Bone Joint Surg Am. 2017;99(1):1–9.

Bloomfield RA, Williams HA, Broberg JS, Lanting BA, McIsaac KA, Teeter MG. Machine learning groups patients by early functional improvement likelihood based on wearable sensor instrumented preoperative timed-up-and-go tests. J Arthroplasty. 2019;34(10):2267–71.

Bovonratwet P, Shen TS, Islam W, Ast MP, Haas SB, Su EP. Natural language processing of patient-experience comments after primary total knee arthroplasty. J Arthroplasty. 2021;36(3):927–34.

Chan B, Rudan JF, Mousavi P, Kunz M. Intraoperative integration of structured light scanning for automatic tissue classification: a feasibility study. Int J Comput Assist Radiol Surg. 2020;15(4):641–9.

Crawford AM, Karhade AV, Agaronnik ND, Lightsey HM, Xiong GX, Schwab JH, Schoenfeld AJ, Simpson AK. Development of a machine learning algorithm to identify surgical candidates for hip and knee arthroplasty without in-person evaluation. Arch Orthop Trauma Surg. 2023;143(9):5985–92.

Devana SK, Shah AA, Lee C, Roney AR, van der Schaar M, SooHoo NF. A novel, potentially universal machine learning algorithm to predict complications in total knee arthroplasty. Arthroplast Today. 2021;10:135–43.

Farooq H, Deckard ER, Ziemba-Davis M, Madsen A, Meneghini RM. Predictors of patient satisfaction following primary total knee arthroplasty: results from a traditional statistical model and a machine learning algorithm. J Arthroplasty. 2020;35(11):3123–30.

Farooq H, Deckard ER, Arnold NR, Meneghini RM. Machine learning algorithms identify optimal sagittal component position in total knee arthroplasty. J Arthroplasty. 2021;36(7S):S242–9.

Fontana MA, Lyman S, Sarker GK, Padgett DE, MacLean CH. Can machine learning algorithms predict which patients will achieve minimally clinically important differences from total joint arthroplasty? Clin Orthop Relat Res. 2019;477(6):1267–79.

Harris AHS, Kuo AC, Weng Y, Trickey AW, Bowe T, Giori NJ. Can machine learning methods produce accurate and easy-to-use prediction models of 30-day complications and mortality after knee or hip arthroplasty? Clin Orthop Relat Res. 2019;477(2):452–60.

Harris AHS, Kuo AC, Bowe TR, Manfredi L, Lalani NF, Giori NJ. Can machine learning methods produce accurate and easy-to-use preoperative prediction models of one-year improvements in pain and functioning after knee arthroplasty? J Arthroplasty. 2021;36(1):112–117.e116.

Heisinger S, Hitzl W, Hobusch GM, Windhager R, Cotofana S. Predicting total knee replacement from symptomology and radiographic structural change using artificial neural networks-data from the osteoarthritis initiative (OAI). J Clin Med. 2020;9(5):1298.

Hsieh CY, Huang HY, Liu KC, Chen KH, Hsu SJ, Chan CT. Subtask segmentation of timed up and go test for mobility assessment of perioperative total knee arthroplasty. Sensors (Basel). 2020;20(21):6302.

Huang Z, Huang C, Xie J, Ma J, Cao G, Huang Q, Shen B, Byers Kraus V, Pei F. Analysis of a large data set to identify predictors of blood transfusion in primary total hip and knee arthroplasty. Transfusion. 2018;58(8):1855–62.

Huber M, Kurz C, Leidl R. Predicting patient-reported outcomes following hip and knee replacement surgery using supervised machine learning. BMC Med Inform Decis Mak. 2019;19(1):3.

Jamshidi A, Pelletier JP, Labbe A, Abram F, Martel-Pelletier J, Droit A. Machine learning-based individualized survival prediction model for total knee replacement in osteoarthritis: data from the osteoarthritis initiative. Arthritis Care Res (Hoboken). 2021;73(10):1518–27.

Jayakumar P, Moore MG, Furlough KA, Uhler LM, Andrawis JP, Koenig KM, Aksan N, Rathouz PJ, Bozic KJ. Comparison of an artificial intelligence-enabled patient decision aid vs educational material on decision quality, shared decision-making, patient experience, and functional outcomes in adults with knee osteoarthritis: a randomized clinical trial. JAMA Netw Open. 2021;4(2):e2037107.

Johannesdottir KB, Kehlet H, Petersen PB, Aasvang EK, Sørensen HBD, Jørgensen CC, Group CfF-tHaKRC. Machine learning classifiers do not improve prediction of hospitalization > 2 days after fast-track hip and knee arthroplasty compared with a classical statistical risk model. Acta Orthop. 2022;93:117–23.

Jones GG, Kotti M, Wiik AV, Collins R, Brevadt MJ, Strachan RK, Cobb JP. Gait comparison of unicompartmental and total knee arthroplasties with healthy controls. Bone Joint J. 2016;98-B(10 Supple B):16–21.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Karnuta JM, Navarro SM, Haeberle HS, Helm JM, Kamath AF, Schaffer JL, Krebs VE, Ramkumar PN. Predicting inpatient payments prior to lower extremity arthroplasty using deep learning: which model architecture is best? J Arthroplasty. 2019;34(10):2235-2241.e2231.

Katakam A, Karhade AV, Schwab JH, Chen AF, Bedair HS. Development and validation of machine learning algorithms for postoperative opioid prescriptions after TKA. J Orthop. 2020;22:95–9.

Katakam A, Karhade AV, Collins A, Shin D, Bragdon C, Chen AF, Melnic CM, Schwab JH, Bedair HS. Development of machine learning algorithms to predict achievement of minimal clinically important difference for the KOOS-PS following total knee arthroplasty. J Orthop Res. 2022;40(4):808–15.

Ko S, Jo C, Chang CB, Lee YS, Moon YW, Youm JW, Han HS, Lee MC, Lee H, Ro DH. A web-based machine-learning algorithm predicting postoperative acute kidney injury after total knee arthroplasty. Knee Surg Sports Traumatol Arthrosc. 2022;30(2):545–54.

Kunze KN, Polce EM, Patel A, Courtney PM, Levine BR. Validation and performance of a machine-learning derived prediction guide for total knee arthroplasty component sizing. Arch Orthop Trauma Surg. 2021;141(12):2235–44.

Leung K, Zhang B, Tan J, Shen Y, Geras KJ, Babb JS, Cho K, Chang G, Deniz CM. Prediction of total knee replacement and diagnosis of osteoarthritis by using deep learning on knee radiographs: data from the osteoarthritis initiative. Radiology. 2020;296(3):584–93.

Li H, Jiao J, Zhang S, Tang H, Qu X, Yue B. Construction and comparison of predictive models for length of stay after total knee arthroplasty: regression model and machine learning analysis based on 1,826 cases in a single Singapore center. J Knee Surg. 2022;35(1):7–14.

Mohammed H, Huang Y, Memtsoudis S, Parks M, Ma Y. Utilization of machine learning methods for predicting surgical outcomes after total knee arthroplasty. PLoS One. 2022;17(3):e0263897.

Navarro SM, Wang EY, Haeberle HS, Mont MA, Krebs VE, Patterson BM, Ramkumar PN. Machine learning and primary total knee arthroplasty: patient forecasting for a patient-specific payment model. J Arthroplasty. 2018;33(12):3617–23.

Rajamohan HR, Wang T, Leung K, Chang G, Cho K, Kijowski R, Deniz CM. Prediction of total knee replacement using deep learning analysis of knee MRI. Sci Rep. 2023;13(1):6922.

Ramazanian T, Yan S, Rouzrokh P, Wyles CC, O Byrne TJ, Taunton MJ, Maradit Kremers H. Distribution and correlates of hip-knee-ankle angle in early osteoarthritis and preoperative total knee arthroplasty patients. J Arthroplasty. 2022;37(6S):S170–5.

Ramkumar PN, Haeberle HS, Ramanathan D, Cantrell WA, Navarro SM, Mont MA, Bloomfield M, Patterson BM. Remote patient monitoring using mobile health for total knee arthroplasty: validation of a wearable and machine learning-based surveillance platform. J Arthroplasty. 2019;34(10):2253–9.

Ramkumar PN, Karnuta JM, Navarro SM, Haeberle HS, Scuderi GR, Mont MA, Krebs VE, Patterson BM. Deep learning preoperatively predicts value metrics for primary total knee arthroplasty: development and validation of an artificial neural network model. J Arthroplasty. 2019;34(10):2220–2227.e2221.

Rexwinkle JT, Werner NC, Stoker AM, Salim M, Pfeiffer FM. Investigating the relationship between proteomic, compositional, and histologic biomarkers and cartilage biomechanics using artificial neural networks. J Biomech. 2018;80:136–43.

Sachau J, Otto JC, Kirchhofer V, Larsen JB, Kennes LN, Hüllemann P, Arendt-Nielsen L, Baron R. Development of a bedside tool-kit for assessing sensitization in patients with chronic osteoarthritis knee pain or chronic knee pain after total knee replacement. Pain. 2022;163(2):308–18.

Sagheb E, Ramazanian T, Tafti AP, Fu S, Kremers WK, Berry DJ, Lewallen DG, Sohn S, Maradit Kremers H. Use of natural language processing algorithms to identify common data elements in operative notes for knee arthroplasty. J Arthroplasty. 2021;36(3):922–6.

Tolpadi AA, Lee JJ, Pedoia V, Majumdar S. Deep learning predicts total knee replacement from magnetic resonance images. Sci Rep. 2020;10(1):6371.

Tsai CC, Huang CC, Lin CW, Ogink PT, Su CC, Chen SF, Yen MH, Verlaan JJ, Schwab JH, Wang CT, et al. The Skeletal Oncology Research Group Machine Learning Algorithm (SORG-MLA) for predicting prolonged postoperative opioid prescription after total knee arthroplasty: an international validation study using 3,495 patients from a Taiwanese cohort. BMC Musculoskelet Disord. 2023;24(1):553.

Verstraete MA, Moore RE, Roche M, Conditt MA. The application of machine learning to balance a total knee arthroplasty. Bone Jt Open. 2020;1(6):236–44.

Wei C, Quan T, Wang KY, Gu A, Fassihi SC, Kahlenberg CA, Malahias MA, Liu J, Thakkar S, Gonzalez Della Valle A, et al. Artificial neural network prediction of same-day discharge following primary total knee arthroplasty based on preoperative and intraoperative variables. Bone Joint J. 2021;103(8):1358–66.

Yeo I, Klemt C, Robinson MG, Esposito JG, Uzosike AC, Kwon YM. The use of artificial neural networks for the prediction of surgical site infection following TKA. J Knee Surg. 2023;36(6):637–43.

Yi PH, Wei J, Kim TK, Sair HI, Hui FK, Hager GD, Fritz J, Oni JK. Automated detection & classification of knee arthroplasty using deep learning. Knee. 2020;27(2):535–42.

Zhang S, Lau BPH, Ng YH, Wang X, Chua W. Machine learning algorithms do not outperform preoperative thresholds in predicting clinically meaningful improvements after total knee arthroplasty. Knee Surg Sports Traumatol Arthrosc. 2022;30(8):2624–30.

Longo UG, De Salvatore S, Intermesoli G, Pirato F, Piergentili I, Becker R, Denaro V. Metaphyseal cones and sleeves are similar in improving short- and mid-term outcomes in total knee arthroplasty revisions. Knee Surg Sports Traumatol Arthrosc. 2023;31(3):861–82.

Hinterwimmer F, Lazic I, Langer S, Suren C, Charitou F, Hirschmann MT, Matziolis G, Seidl F, Pohlig F, Rueckert D, et al. Prediction of complications and surgery duration in primary TKA with high accuracy using machine learning with arthroplasty-specific data. Knee Surg Sports Traumatol Arthrosc. 2023;31(4):1323–33.

Longo UG, Maffulli N, Denaro V. Minimally invasive total knee arthroplasty. N Engl J Med. 2009;361(6):633–4 author reply 634.

Article   PubMed   CAS   Google Scholar  

Lambrechts A, Wirix-Speetjens R, Maes F, Van Huffel S. Artificial intelligence based patient-specific preoperative planning algorithm for total knee arthroplasty. Front Robot AI. 2022;9:840282.

Berton A, Longo UG, Candela V, Fioravanti S, Giannone L, Arcangeli V, Alciati V, Berton C, Facchinetti G, Marchetti A, et al. Virtual Reality, Augmented Reality, Gamification, and Telerehabilitation: Psychological Impact on Orthopedic Patients' Rehabilitation. J Clin Med. 2020;9(8).

Goplen CM, Kang SH, Randell JR, et al. Effect of preoperative long-term opioid therapy on patient outcomes after total knee arthroplasty: an analysis of multicentre population-based administrative data. Can J Surg. 2021;64(2):E135–43. https://doi.org/10.1503/cjs.007319 .

Google Scholar  

Longo UG, Ciuffreda M, Mannering N, D’Andrea V, Cimmino M, Denaro V. Patellar resurfacing in total knee arthroplasty: systematic review and meta-analysis. J Arthroplasty. 2018;33(2):620–32.

Bravi M, Longo UG, Laurito A, Greco A, Marino M, Maselli M, Sterzi S, Santacaterina F. Supervised versus unsupervised rehabilitation following total knee arthroplasty: a systematic review and meta-analysis. Knee. 2023;40:71–89.

Longo UG, Silva S, Perdisa F, Salvatore G, Filardo G, Berton A, Piergentili I, Denaro V. Gender related results in total knee arthroplasty: a 15-year evaluation of the Italian population. Arch Orthop Trauma Surg. 2023;143(3):1185–92.

Longo UG, Loppini M, Trovato U, Rizzello G, Maffulli N, Denaro V. No difference between unicompartmental versus total knee arthroplasty for the management of medial osteoarthtritis of the knee in the same patient: a systematic review and pooling data analysis. Br Med Bull. 2015;114(1):65–73.

Longo UG, Ciuffreda M, Mannering N, D’Andrea V, Locher J, Salvatore G, Denaro V. Outcomes of posterior-stabilized compared with cruciate-retaining total knee arthroplasty. J Knee Surg. 2018;31(4):321–40.

Stelfox HT, Gandhi TK, Orav EJ, Gustafson ML. The relation of patient satisfaction with complaints against physicians and malpractice lawsuits. Am J Med. 2005;118(10):1126–33.

Download references

Acknowledgements

Not applicable.

This research received no external funding.

Author information

Authors and affiliations.

Fondazione Policlinico Universitario Campus Bio-Medico, Via Alvaro del Portillo, Rome, 200 - 00128, Italy

Umile Giuseppe Longo

Department of Medicine and Surgery, Research Unit of Orthopaedic and Trauma Surgery, Università Campus Bio-Medico Di Roma, Via Alvaro del Portillo, Rome, 21 - 00128, Italy

Umile Giuseppe Longo, Federica Valente, Mariajose Villa Corta & Kristian Samuelsson

IRCCS Ospedale Pediatrico Bambino Gesù, Rome, Italy

Sergio De Salvatore

Orthopedic Unit, Department of Surgery, Bambino Gesù Children’s Hospital, Rome, Italy

Orthopaedic Department, Clinical Institute Sant’Ambrogio, IRCCS - Galeazzi, Milan, Italy

Bruno Violante

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, U.G.L., S.D.S.; methodology, U.G.L., F.V; software.B.V, K.S; validation, U.G.L., B.V.; formal analysis, M.V.C, K.S, F.V; resources, U.G.L., S.DS; data curation, F.V ; writing—original draft preparation M.V.C, B.V., K.S., F.V.; writing—review and editing, M.V.C, U.G.L., ; visualization, K.S., F.M ; supervision, U.G.L.,  All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Umile Giuseppe Longo .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Longo, U.G., De Salvatore, S., Valente, F. et al. Artificial intelligence in total and unicompartmental knee arthroplasty. BMC Musculoskelet Disord 25 , 571 (2024). https://doi.org/10.1186/s12891-024-07516-9

Download citation

Received : 16 October 2023

Accepted : 13 May 2024

Published : 22 July 2024

DOI : https://doi.org/10.1186/s12891-024-07516-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Artificial intelligence
• Orthopaedics
• Joint replacement
• Knee replacement

BMC Musculoskeletal Disorders

ISSN: 1471-2474

• Submission enquiries: [email protected]
• General enquiries: [email protected]