gamification in the workplace a systematic literature review

• DOI: 10.1007/978-3-319-56541-5_29
• Corpus ID: 22261977

Gamification in the Workplace: A Systematic Literature Review

• A. Ferreira-Oliveira , Alexandra M. Araújo , +1 author I. Miguel
• Published in WorldCIST 11 April 2017
• Business, Computer Science

43 Citations

Gamification at workplace: theories, constructs and conceptual frameworks, gamification in business: a review of the studies, continuous learning at work: the power of gamification, understanding the effectiveness of gamification in an industrial work process: an experimental approach, gamification for recruitment and job training: model, taxonomy, and challenges, gamification in dutch businesses: an explorative case study, work gamification: effects on enjoyment, productivity and the role of leadership, preconceptions towards gamifying work: a thematic analysis of responses of a maritime logistics organization, evaluation of an industrial case of gamification in software quality improvement, “prevention is better than cure”: challenges in engaging employees through gamification, 51 references, i play at work—ten principles for transforming work processes through gamification, decision support model for introduction of gamification solution using ahp, towards a gamification of industrial production: a comparative study in sheltered work environments, applying game design elements in the workplace, work and play: an experiment in enterprise gamification, the gamification of spice, gamification and multigamification in the workplace: expanding the ludic dimensions of work and challenging the work/play dichotomy, lude et labora. notes on gamification at work, individuals, teams and organizations score with gamification, the nature, scale and beneficiaries of research impact, related papers.

Showing 1 through 3 of 0 Related Papers

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Internet Interv
• v.6; 2016 Nov

Gamification for health and wellbeing: A systematic review of the literature

Daniel johnson.

a Queensland University of Technology (QUT), GPO Box 2434, Brisbane, QLD 4001, Australia

Sebastian Deterding

b Digital Creativity Labs, University of York, York YO10 5GE, United Kingdom

Kerri-Ann Kuhn

Aleksandra staneva, stoyan stoyanov, leanne hides.

Compared to traditional persuasive technology and health games, gamification is posited to offer several advantages for motivating behaviour change for health and well-being, and increasingly used. Yet little is known about its effectiveness.

We aimed to assess the amount and quality of empirical support for the advantages and effectiveness of gamification applied to health and well-being.

We identified seven potential advantages of gamification from existing research and conducted a systematic literature review of empirical studies on gamification for health and well-being, assessing quality of evidence, effect type, and application domain.

We identified 19 papers that report empirical evidence on the effect of gamification on health and well-being. 59% reported positive, 41% mixed effects, with mostly moderate or lower quality of evidence provided. Results were clear for health-related behaviours, but mixed for cognitive outcomes.

Conclusions

The current state of evidence supports that gamification can have a positive impact in health and wellbeing, particularly for health behaviours. However several studies report mixed or neutral effect. Findings need to be interpreted with caution due to the relatively small number of studies and methodological limitations of many studies (e.g., a lack of comparison of gamified interventions to non-gamified versions of the intervention).

• • A systematic review is conducted to assess the empirical effectiveness of gamification in the health and wellbeing domain.
• • Twenty-one papers are identified that report empirical evidence on the effectiveness of gamification in health and wellbeing.
• • Overall the evidence suggests gamification can have a positive impact for health and wellbeing related interventions.
• • The evidence is strongest for the use of gamification to target behavioural outcomes, particularly physical activity.
• • Further research that isolates the impacts of gamification is needed.

1. Introduction

1.1. background.

The major health challenges facing the world today are shifting from traditional, pre-modern risks like malnutrition, poor water quality and indoor air pollution to challenges generated by the modern world itself. Today, the leading global risks for mortality and chronic diseases – high blood pressure, tobacco use, high blood glucose, physical inactivity, obesity, high cholesterol – are immediately linked to a modern lifestyle characterized by sedentary living, chronic stress, and high intake of energy-dense foods and recreational drugs ( Stevens et al., 2009 ). In addition, following calls from the World Health Organization's (2015/(1946) inclusive conception of health, researchers, civil society, and politicians have been pushing to extend policy goals from preventing and reducing disease towards promoting people's holistic physical, mental, and social well-being ( Carlisle and Hanlon, 2008 , Hanratty and Farmer, 2012 , Huppert and So, 2013 , Marks and Shah, 2004 , Schulte et al., 2015 ).

Practically all modern lifestyle health risks (and resulting diseases) are directly affected by people's individual health behaviours — be it physical activity, diet, recreational drug use, medication adherence, or preventive and rehabilitative exercises ( Glanz et al., 2008 , Schroeder, 2007 ). By one estimate, three quarters of all health care costs in the US are attributable to chronic diseases caused by poor health behaviours ( Woolf, 2008 ), the effective management of which again requires patients to change their behaviours ( Sola et al., 2015 ). Similarly, research indicates that well-being can be significantly improved through small individual behaviours ( Lyubomirsky and Layous, 2013 , Seligman, 2011 ). Behaviour change has therefore become one of the most important and frequently targeted levers for reducing the burden of preventable disease and death and increasing well-being ( Glanz, K., Rimer, B. K., & Viswanath, K, 2008, p. xiii ).

A main factor driving behaviour change is the individual's motivation. Even if different theories contain different motivational constructs, “the processes that direct and energize behaviour” ( Reeve, 2014, p. 8 ) feature prominently across health behaviour change theories ( Glanz and Bishop, 2010 , Michie et al., 2011b ). Motives are a core target of a wide range of established behaviour change techniques ( Michie et al., 2011a,b ).

However, following self-determination theory (SDT), a well-established motivation theory, not all forms of motivation are equal ( Deci and Ryan, 2012 ). A crucial consideration is whether behaviour is intrinsically or extrinsically motivated. Intrinsic motivation describes activities done ‘for their own sake,’ which satisfy basic psychological needs for autonomy, competence, and relatedness, giving rise to the experience of volition, willingness, and enjoyment. Extrinsically motivated activity is done for an outcome separable from the activity itself, like rewards or punishments, which thwarts autonomy need satisfaction and gives rise to experiences of unwillingness, tension, and coercion ( Deci and Ryan, 2012 ). In recent years, SDT has become a key framework for health behaviour interventions and studies. A large number of studies have demonstrated advantages of intrinsic over extrinsic motivation with regard to health behaviours ( Fortier et al., 2012 , Ng et al., 2012 , Patrick and Williams, 2012 , Teixeira et al., 2012b ). Not only is intrinsically motivated behaviour change more sustainable than extrinsically motivated change ( Teixeira, Silva, Mata, Palmeira, & Markland, 2012 ): satisfying the psychological needs that intrinsically motivate behaviour also directly contributes to mental and social well-being ( Ryan, Huta, & Deci, 2008 ; Ryan, Patrick, Deci, & William, 2008 ).

In short, in our modern life world, health and well-being strongly depend on the individual's health behaviours, motivation is a major factor of health behaviour change, and intrinsically motivated behaviour change is desirable as it is both sustained and directly contributes to well-being. This raises the immediate question what kind of interventions are best positioned to intrinsically motivate health behaviour change.

1.2. Computing technology for health behaviour change and well-being

The last two decades have seen the rapid ascent of computing technology for health behaviour change and well-being ( Glanz, K., Rimer, B. K., & Viswanath, K, 2008, pp. 8–9 ), with common labels like persuasive technology ( Fogg, 2003 ) or positive computing ( Calvo and Peters, 2014 ). This includes a broad range of consumer applications for monitoring and managing one's own health and well-being ( Knight et al., 2015 , Martínez-Pérez et al., 2013 , Middelweerd et al., 2014 ), such as the recent slew of “quantified self” ( Wolf, 2009 ) or “personal informatics” tools for collecting and reflecting on information about the self ( Li et al., 2010 ).

One important sector is serious games for health ( Wattanasoontorn et al., 2013 ), games used to drive health-related outcomes. The majority of these are “health behaviour change games” ( Baranowski et al., 2008 ) or “health games” ( Kharrazi et al., 2012 ) affecting the health behaviours of health care receivers (and not e.g. training health care providers) ( Wattanasoontorn et al., 2013 ). Applications and research have mainly targeted physical activity, nutrition, and stroke rehabilitation, with an about equal share of (a) “exergames” or “active video games” directly requiring physical activity as input, (b) behavioural games focusing specific behaviours, (c) rehabilitation games guiding rehabilitative movements, and (d) educational games targeting belief and attitude change as a precondition to behaviour change ( Kharrazi et al., 2012 ). Like serious games in general, health games have seen rapid growth ( Kharrazi et al., 2012 ), with numerous systematic reviews assessing their effectiveness ( DeSmet et al., 2014 , DeSmet et al., 2015 , Gao et al., 2015 , LeBlanc et al., 2013 , Lu et al., 2013 , Papastergiou, 2009 , Primack et al., 2012 , Theng et al., 2015 ).

A main rationale for using games for serious purposes like health is their ability to motivate: Games are systems purpose-built for enjoyment and engagement ( Deterding, 2015b ). Research has confirmed that well-designed games are enjoyable and engaging because playing them provides basic need satisfaction ( Mekler et al., 2014 , Przybylski et al., 2010 , Tamborini et al., 2011 ). Turning health communication or health behaviour change programs into games might thus be a good way to intrinsically motivate users to expose themselves to and continually engage with these programs ( Baranowski et al., 2008 ; though see Wouters et al., 2013 ).

However, the broad adoption of health games has faced major hurdles. One is their high cost of production and design complexity: Health games are typically bespoke interventions for a small target health behaviour and population, and game development is a cost- and time-intensive process, especially if one desires to compete with the degree of “polish” of professional, big studio entertainment games. Thus, there is no developed market and business model for health games, wherefore the entertainment game and the health industries have by and large not moved into the space ( Parker, n.d , Sawyer, 2014 ).

A second adoption hurdle is that most health games are delivered through a dedicated device like a game console, and require users to create committed spaces and times in their life for gameplay. This demand often clashes with people's varied access to technology, their daily routines and rituals, as well as busy and constantly shifting schedules ( Munson et al., 2015 ).

1.3. Gamification: a new model?

One possible way of overcoming these hurdles is presented by gamification, which is defined as “the use of game design elements in non-game contexts” ( Deterding et al., 2011 ; see Seaborn and Fels, 2015 for a review). The underlying idea of gamification is to use the specific design features or “motivational affordances” ( Deterding, 2011 , Zhang, 2008 ) of entertainment games in other systems to make engagement with these more motivating. 1 Appealing to established theories of intrinsic motivation, gamified systems commonly employ motivational features like immediate success feedback, continuous progress feedback, or goal-setting through interface elements like point scores, badges, levels, or challenges and competitions; relatedness support, social feedback, recognition, and comparison through leaderboards, teams, or communication functions; and autonomy support through customizable avatars and environments, user choice in goals and activities, or narratives providing emotional and value-based rationales for an activity (cf. Ryan and Rigby, 2011 , Seaborn and Fels, 2015 ).

Since its emergence around 2010, gamification has seen a groundswell of interest in industry and academia, easily outstripping persuasive technology in publication volume ( Hamari, Koivisto, & Pakkanen, 2014 ). By one estimate, the gamification market is poised to reach 2.8 billion US dollars by 2016 ( Meloni and Gruener, 2012 ). It is little wonder, then, that several scholars have pointed to health gamification as a promising new approach to health behaviour change ( Cugelman, 2013 , King et al., 2013 , Munson et al., 2015 , Pereira et al., 2014 , Sola et al., 2015 ). Popular examples are Nike+ 2 , a system of activity trackers and applications that translate measured physical exertion into so-called “NikeFuel points” which then become enrolled in competitions with friends, the unlocking of achievements, or social sharing; Zombies , Run! 3 , a mobile application that motivates running through wrapping runs into an audio-delivered story of surviving a Zombie apocalypse; or SuperBetter 4 , a web platform that helps people achieve their health goals by building psychological resilience, breaking goals into smaller achievable tasks and wrapping these into layers of narrative and social support.

Conceptually, health gamification sits at the intersection of persuasive technology, serious games, and personal informatics ( Cugelman, 2013 , Munson et al., 2015 ): Like persuasive technology, it revolves around the application of specific design principles or features that drive targeted behaviours and experiences. Several authors have in fact suggested that many game design elements can be mapped to established behaviour change techniques ( Cheek et al., 2015 , Cugelman, 2013 , King et al., 2013 ). Like serious games, gamification aims to drive these behaviours through the intrinsically motivating qualities of well-designed games. Like personal informatics, gamification usually revolves around the tracking of individual behaviours, only that these are then not only displayed to the user, but enrolled in some form of goal-setting and progress feedback. Indeed, many applications commonly classified as gamification are also labelled personal informatics, and gamification is seen as a way to sustain engagement with personal informatics applications (e.g., Morschheuser et al., 2014 ).

1.4. Promises of gamification for health and well-being

The reasons why gamification is potentially relevant to health behaviour change today, and the shortcomings of other digital health and well-being interventions include:

• 1 Intrinsic motivation. Like games, gamified systems can intrinsically motivate the initiation and continued performance of health and well-being behaviours ( Deterding, 2015b for similar arguments regarding gamification in general; King et al., 2013 , Munson et al., 2015 , Pereira et al., 2014 ; cf. Seaborn and Fels, 2015 , Sola et al., 2015 ). In contrast, personal informatics can lack sustained appeal, and persuasive technologies often employ extrinsic motivators like social pressure or overt rewards ( Oinas-Kukkonen and Harjumaa, 2009 ).
• 2 Broad accessibility through mobile technology and ubiquitous sensors. Activity trackers and mobile phones, equipped with powerful sensing, processing, storage, and display capacities, are excellent and widely available platforms to extend a game layer to everyday health behaviours, making gamified applications potentially more accessible than health games which rely on bespoke gaming devices ( King et al., 2013 , Lister et al., 2014 , Sawyer, 2014 ).
• 3 Broad appeal . As wider and wider audiences play games, games and game design elements become approachable and appealing to wider populations ( King et al., 2013 ).
• 4 Broad applicability . Current health gamification domains cover all major chronic health risks: physical activity, diet and weight management, medication adherence, rehabilitation, mental well-being, drug use, patient activation around chronic diseases like Diabetes, cancer, or asthma ( Munson et al., 2015 , Pereira et al., 2014 , Sola et al., 2015 ).
• 5 Cost-benefit efficiency . Retro-fitting existing health systems and enhancing new ones with an engaging “game layer” may be faster, most cost-benefit efficient, and more scalable than the development of full-fledged health games ( Munson et al., 2015 , Sawyer, 2014 ).
• 6 Everyday life fit . Gamified systems using mobile phones or activity trackers can encompass practically all trackable everyday activity, unlike health games requiring people to add dedicated time and space to their life ( Munson et al., 2015 ). Whereas standard health games typically try to fit another additional activity into people's schedules, gamification aims to reorganise already-ongoing everyday conduct in a more well-being conducive manner ( Deterding, 2015b ; see Hassenzahl and Laschke, 2015 ).
• 7 Supporting well-being . Beyond motivating health behaviours, engaging with gamified applications can directly contribute to well-being by generating positive experiences of basic psychological need satisfaction as well as other elements of well-being like positive emotions, engagement, relationships, meaning, and accomplishment (cf. Johnson et al., 2013 for a review on well-being effects on video game play; McGonigal, 2011 , Pereira et al., 2014 ).

In short, gamification may realize what games for health doyen Ben Sawyer (2014) dubbed the “new model for health” games should pursue: sensor-based, data-driven, “seductive, ubiquitous, lifelong health interfaces” for well-being self-care.

Promising as gamification for health and well-being may be, the essential question remains whether gamified interventions are effective in driving behaviour change, health, and well-being, and more specifically, whether they manage to do so via intrinsic motivation. These questions are especially relevant as (a) general-purpose literature reviews on gamification have flagged the lack of high-quality effect studies on gamification ( Hamari et al., 2014b ; cf. Seaborn and Fels, 2015 ), and (b) critics have objected that gamification often effectively entails standard behavioural reinforcement techniques and reward systems that are extrinsically motivating, not emulating the intrinsically motivating features of well-designed games ( Juul, 2011 , Walz and Deterding, 2015 ).

1.5. Research goal and questions

To our knowledge, there is no systematic review on the effectiveness and quality of health and well-being gamification applications available. Existing reviews include a survey spanning several application domains which identified four health-related papers (cf. Seaborn and Fels, 2015 ), a review of gamification features in commercially available health and fitness applications ( Lister et al., 2014 ), a topical review on the use of games, gamification, and virtual environments for diabetes self-management, which identified three studies on gamified applications ( Theng et al., 2015 ), a review focused specifically on the use of (extrinsic) reward systems in health-related gamified applications ( Lewis et al., 2016 ) and a review on the persuasion context of gamified health behaviour support systems ( Alahäivälä and Oinas-Kukkonen, 2016 ). While these reviews offer important and valuable insights, none have examined gamification for both health and well-being nor the effectiveness of gamification. Additionally, existing reviews do not directly consider and evaluate the quality of evidence underlying the conclusions drawn. We therefore conducted a systematic literature review of peer-reviewed papers examining the effectiveness of gamified applications for health and well-being, assessing the quality of evidence provided by studies.

We developed four guiding research questions:

• o What is the number and quality of available effect studies? This follows the observation that gamification research is lacking high-quality effect studies.
• o What effects are reported? This follows the question whether health gamification is indeed effective.
• o What game design elements are used and tested? These questions follow whether health gamification drives outcomes through the same processes of intrinsic motivation that make games engaging, and whether directly supporting well-being through positive experiences.
• o What delivery platforms are used and tested? This probes whether current health gamification does make good on the promise of greater accessibility, pervasiveness, and everyday life fit through mobile phones or multiple platforms.
• o Which theories of motivation ( e.g. , Self-Determination Theory) are used and tested? This explores to what extent health gamification explicitly draws on motivational theory and to whether design incorporating these theories leads to better outcomes.
• o Is gamification shown to be more effective with gaming affinitive audiences? This assesses whether the benefits of gamification are limited to audience already familiar with or drawn to game elements as engaging and motivating.
• o Have the benefits of health gamification been shown to extend to audiences that are not already intrinsically motivated? This explores whether there is evidence of gamification working when users are not already intrinsically motivated to perform the target activity (e.g., users who voluntarily engage with a fitness app can be assumed to already be intrinsically motivated to exercise).
• • RQ4. What health and well-being domains are targeted? Beyond a general scoping of the field, this tests whether the claimed broad applicability of gamification indeed holds.

The protocol for the review was developed and agreed by the authors prior to commencement. It followed all aspects recommended in the reporting of systematic reviews, namely the PRISMA Checklist and MOOSE Guidelines ( Moher et al., 2010 ). All studies that explored the association between gamification and health were considered for this review. “Gamification” was defined and operationalised as “the use of game design elements in non-game contexts” ( Deterding et al., 2011 ). “Health” and “well-being” were collectively defined and operationalised using the World Health Organization's’s’s’s (1946) inclusive definition of health as “a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity”.

2.1. Data collection

The electronic databases in this review were searched on November 19th, 2015 and included those identified as relevant to information technology, social science, psychology and health: Ebscohost (PsychInfo, Medline, CINAHL) ( n  = 33); ProQuest ( n  = 10); Association for Computing Machinery, ACM ( n  = 81); IEEE Xplore ( n  = 36); Web of Science ( n  = 44); Scopus ( n  = 108); Science Direct ( n  = 12) and PubMed ( n  = 39). Three additional studies were identified with a manual search of the reference lists of key studies, including existing gamification reviews, identified during the database search process.

2.2. Search terms

Based on prior practice in systematic reviews on gamification and health and well-being ( Alahäivälä and Oinas-Kukkonen, 2016 , Lewis et al., 2016 , Seaborn and Fels, 2015 ), we used full and truncated search terms capturing gamification, health outcomes, and well-being in the following search string:

Gamif* AND (health OR mental OR anxi* OR depres* OR wellbeing OR well-being).

Mental health related search terms (“mental”, “anxi*” and “depres*”) were added as initial searches failed to capture some expected results.

2.3. Inclusion/exclusion criteria

2.3.1. inclusion criteria.

Our review focused on high quality scholarly work reporting original research on the impact and effectiveness of gamification for health and wellbeing. From this focus, we developed the following inclusion criteria:

• 1 Peer-reviewed (incl. peer-reviewed conference papers)
• 2 Full papers (incl. full conference papers)
• 3 Empirical research (qualitative and quantitative)
• 4 Explained research methods
• 5 Explicitly state and described gamification as research subject
• 6 Clearly described gamification elements (type of game design elements)
• 7 Effect reported in terms of:
• a. Impact (affect, behaviour, cognition), and/or
• b. User experience — any subjective measure of experience while using the gamified or non-gamified version of the intervention
• 8 Clearly described outcomes related to health and well-being

Criteria 1–4 were chosen to ensure focus on high-quality work reporting original research. Criteria 3, 4, and 7 were also included to enable assessment of quality of evidence. Criteria 5–6 ensured the paper reported on gamification, not serious games or persuasive technology mislabeled as gamification (a common issue, cf. Seaborn and Fels, 2015 ). Criteria 7–8 were chosen to assess reported health and well-being outcomes and potential mediators, with user experience included given its prevalence as an outcome measure in gamification research (see Table 1 ).

Full paper details and quality of evidence ratings.

2.3.2. Exclusion criteria

Our exclusion criteria mirror the focus on high quality scholarly work that reports the impact and effectiveness of gamification for health and well-being. They were particularly framed to exclude duplicate reporting of earlier versions of studies fully reported later. We excluded papers with the following features:

• 1 Extended abstracts or ‘work-in-progress’ papers
• 2 Study protocols
• 3 Covers complete games (serious games) not gamification
• 4 Gamification is mentioned but not evaluated

Criteria 1–2 exclude peer-reviewed yet early and incomplete versions of studies. Criteria 3–4 exclude studies that mislabel serious games as gamification (see above) or fail to report the concrete intervention in sufficient detail to assess whether it constituted gamification.

2.4. Quality assessment tool

We used the quality assessment method presented by Connolly et al. (2012) . The tool was explicitly developed to assess the strength of evidence of a total body of work relative to a particular review question. Connolly et al. (2012) used the tool to assess the overall weight of empirical evidence for positive impact and outcomes of games. We applied the tool to our more focused interest in the empirical evidence for the effectiveness of gamification in the health and wellbeing domain. Each final paper included in the review was read and given a score of 1–3 (where 3 denotes high, 2 denotes medium and 1 denotes low on that criterion) across the following five criteria:

• 1 How appropriate is the research design for addressing the research questions of this review (higher weighting for inclusion of a control group)
• a. High — 3 RCT
• b. Medium — 2, quasi-experimental controlled study
• c. Low — 1, case study, single subject-experimental, pre-test/post-test design
• 2 How appropriate are the methods and analysis?
• 3 How generalizable are the findings of this study to the target population with respect to the size and representativeness of the sample? To what extent would the findings be relevant across age groups, gender, ethnicity, etc.
• 4 How relevant is the particular focus of the study (incl. Conceptual focus, context, sample and measure) for addressing the question of this review?
• 5 To what extent can the study findings be trusted in answering the study question?

The total weight of evidence for each paper is calculated by adding the scores of all five dimensions, with a range from 5 to 15. Connolly et al.'s (2012 p. 665) analysis of the empirical evidence regarding games and serious games found a mean rating of 8.56 and a mode of 9, which gave us a baseline to evaluate gamification studies against. Connolly et al. (ibid.) found 70 of 129 or 54% of studies to be above the mode, constituting “stronger evidence”. We elected to categorise in slightly more detail, with papers with a rating 8 or below categorised as “weaker evidence”, papers with a rating above 8 to 12 as “moderate evidence”, and papers with a rating above 12 as “stronger evidence”.

2.5. Modalities and game design elements

Based on an initial survey, we categorised delivery modalities as mobile (phone), website, social network application, analog, or bespoke device. Given the lack of consensus in the literature regarding definitions and categorizations, game design elements were coded using an adaptation of the systemisation provided by Hamari, Koivisto and Sarsa (2014) . Hamari and colleagues identified the following typology: points, leaderboards, achievements/badges, levels, story/theme, clear goals, feedback, rewards, progress and challenge. In the current review, we elected to combine points and badges with other digital rewards (e.g., virtual roses, coins, digital in-app equipment) into a single category labelled ‘rewards’. Additionally, we also coded for the inclusion of an ‘avatar’ or ‘social interaction,’ as these were found to be commonly employed game design elements in the reviewed papers.

2.6. Effects

We categorised health and well-being effects as relating to affect (mood), behaviour (i.e., involving real world actions), or cognition (e.g., sense of empowerment, motivation, stress, knowledge of domain). These categories were chosen based on the three-component model of attitudes ( Breckler, 1984 , Vaughan and Hogg, 1995 ) with the primary adaptation being the inclusion of knowledge of the target domain as part of the cognition category (knowledge was only assessed in one study ( Allam et al., 2015 ). In addition, multiple studies also assessed user experience (e.g. attitudes towards the gamified intervention itself), which we coded separately. Furthermore, we coded effects as positive, negative, or mixed/neutral, the latter meaning that results were inconclusive or positive for one group and negative for another. If a study assessed health and well-being impacts for multiple dimensions, these were counted separately. For example, a study that finds positive effects on stress and life satisfaction would be counted as two positive impacts on cognition. In contrast, a study that finds a positive impact on life satisfaction for one group of users and negative impact for another would be coded as one neutral/mixed impact on cognition.

2.7. Inter-rater reliability

All studies were independently coded by a second reviewer. Inter-rater reliability was determined by the intra-class correlation coefficient (ICC) ( Shrout and Fleiss, 1979 ). This statistic allows for the appropriate calculation of weighted values of rater agreement and accounts for proximity, rather than equality of ratings. A two-way mixed effects, average measures model with absolute agreement was utilized. Independent ratings demonstrated an excellent level of inter-rater reliability (2-way mixed ICC = 0.91; 95% CI 0.77–0.96).

Our search identified 365 papers. After removing duplicates 221 papers remained. Of these 191 were removed based on screening of title and abstract. The remaining 30 articles were considered and assessed as full texts. Of them eleven did not pass the inclusion and exclusion criteria. Nineteen final eligible studies remained and were individually assessed for this review. The study selection process is reported as recommended by the PRISMA group ( Moher et al., 2010 ) in Fig. 1 .

Flow diagram.

The final 19 articles eligible for review were then rated for quality of evidence (in relation to the current papers review question, see Table 1 ). Following Connolly et al. (2012) we calculated the mean (10.3) and mode (10.5) as a means of determining which papers provided relatively weaker or stronger evidence. However, we departed from the approach taken by Connolly and colleagues who assigned papers to two categories (weaker and stronger quality of evidence) and instead categorised papers into three categories (weaker, moderate and stronger evidence). This decision was made as an equal number of papers fell above and below the mode of 10.5 (also the median), which in turn meant that classifying papers with the modal/median score as either weaker or stronger evidence arbitrarily resulted in that category appearing as a majority. Based on this, 8 papers (42%) were categorised as providing weaker evidence, 3 papers (16%) were categorised as providing moderate evidence and 8 papers (42%) were categorised as providing stronger evidence. See Fig. 2 for a histogram displaying quality of evidence ratings.

Histogram of quality of evidence ratings.

A closer look into methodologies helps unpack these ratings. The majority ( n  = 11) of studies collected data at multiple timepoints (two or more) from multiple groups or conditions; 6 studies collected data from a single group at multiple timepoints, two from a single group at a single time point. Notably, more than half ( n  = 10) of the studies did not compare gamified and non-gamified versions of the interventions studied. Sample sizes ranged from 5 to 251, sampling methods included both convenient and systematic.

Chief modalities employed were mobile applications ( n  = 7) and websites ( n  = 6), with several studies offering an intervention across both. Two studies each used analog techniques, social networking sites, or bespoke devices, namely a modified fork and a Wii console and Wii Fit board. Game design elements included avatars, challenges, feedback, leaderboards, levels, progress indicators, rewards and story/theme and social interaction (see Table 2 ). A total of 46 instances of implemented gamification elements were found across the 19 papers. The most commonly employed elements were rewards ( n  = 16), leaderboards ( n  = 6) and avatars ( n  = 6).

Frequency of user of game design elements.

There was a broad variety without discernible patterns in outcome measures (including surveys/questionnaires, interviews, diary entries, videos, log files and equipment readings such as blood glucose readings), target audiences, or contexts, including medical settings, home recovery, self-assessment, health monitoring, stress management, improving eating behaviours, and increasing physical activity.

Overall (see Table 3 ), positive effects of gamified interventions were reported in the majority of cases ( n  = 22, 59%), with a significant proportion of neutral or mixed effects ( n  = 15, 41%) and no purely negative effects reported. The majority of assessed outcomes were behavioural ( n  = 19, 51%) or cognitive ( n  = 17, 46%). Affect was rarely assessed ( n  = 1, 3%).

Positive, mixed/neutral and negative health and well-being impacts of gamification.

Beyond health and well-being impacts, 12 studies assessed user experience impacts, with 5 (42%) reporting positive, 5 (42%) reporting mixed and 2 (16%) reporting negative impacts.

4. Discussion

For the most part, gamification has been well received; it has been shown to foster positive impacts on affect, behaviour, cognition and user experience. The majority of studies reported gamification had a positive influence on health and well-being. In those cases where gamification had mixed or negative effects, the primary issues seemed to be: 1) the context in which gamification was used (e.g., mindfulness), 2) the manner in which gamification was applied (e.g., exaggerated feedback), or 3) a mismatch between the gamification techniques used and the target audience (e.g., non-beginners feeling that gamification interfered with access to the target activities).

4.1. RQ1. What evidence is there for the effectiveness of gamification applied to health and wellbeing?

We assessed evidence based on the number, quality and the reported effects of available studies. We identified a total of 19 studies assessing the effects of gamified health and wellbeing interventions published since 2012 (avg. 5 studies/year). The most comparable serious games for health meta-analysis in terms of inclusion and exclusion criteria is DeSmet et al. (2014) , which found 53 studies published between 1989 and 2013 (avg. 2 studies/year). This provides evidence that health gamification research like gamification research in general is progressing at a fast pace (cf. Hamari et al., 2014a,b ).

Quality of evidence ratings of existing research conducted by two raters, indicated an equal number of papers were of weak ( n  = 8) or strong ( n  = 8) quality, and the remainder ( n  = 3) were of moderate quality. This suggests that health and wellbeing research is approximately in line with the low evidence quality of gamification research in general (cf. Hamari et al., 2014a , Hamari et al., 2014b ) or perhaps slightly better. It is also consistent with the quality of research found in (serious) game research in general: our study found a mean quality rating of 10.3 (with 42% of papers below the mean and classified as providing weaker evidence). In comparison, Connolly et al. (2012 p. 665) reported a mean rating of 8.56 (with 46% of papers classified as providing weaker evidence). While the number of studies included in the current review precludes any firm conclusions, the slightly higher mean quality score found in the current study could indicate the quality of evidence for empirical effectiveness is slightly higher in gamification in health and wellbeing than the broader serious games literature. More broadly, it is worth noting that the small number and low quality ratings of studies included in this review reflect the relative infancy of the gamification field and the formative nature of research conducted to date.

It should also be noted that this analysis of quality of evidence is not intended as a critique of the peer review the selected papers underwent. The papers were categorised as providing lower, moderate or stronger evidence solely with respect to the weight of empirical evidence for health and well-being effects; studies may well be considered differently based on other aims and criteria.

The impact of gamified interventions on health and well-being was found to be predominantly positive (59%). However, a significant portion (41%) of studies reported mixed or neutral effects. More specifically, findings were largely positive for behavioural impacts (13 positive, 6 mixed or neutral), whereas the evidence for cognitive outcomes is less clear-cut, with an approximately equal number of reported positive ( n  = 8) and mixed/neutral ( n  = 9) impacts. Notably, no direct negative impacts on health and wellbeing were reported, although 2 of 12 studies that additionally assessed user experience reported negative impacts on the latter. This picture is more positive than comparable general gamification reviews (cf. Hamari et al., 2014a , Seaborn and Fels, 2015 ). Current results suggest gamification of health and wellbeing interventions can lead to positive impacts, particularly for behaviours, and is unlikely to produce negative impacts. That being said, gamification should be used with caution when the user experience is critical, e.g. where users can voluntarily opt in and out of the intervention. For example, Spillers and Asimakopoulos (2014) documented user complaints about the poor usability of gamified running apps, which resulted in individual users ceasing to use them. Boendermaker et al. (2015) similarly suggest that gamification may detract from usability and user experience by adding task demands to the interface.

4.2. RQ2. How is gamification being applied to health and wellbeing applications?

The majority of papers ( n  = 7) explored mobile devices or websites as the delivery platform ( n  = 6). Positive effects were also found outside the digital domain including a gamified physical display in the classroom ( Jones et al., 2014b , Jones et al., 2014a ) and a sensor-equipped fork designed to influence children's eating habits ( Kadomura et al., 2014 ). This is in line with the identified promises of everyday life fit and broad accessibility of gamification through mobile and ubiquitous sensor technology. That being said, there are few studies directly testing the differences and effects of everyday life fit and accessibility in mobile/ubiquitous versus PC/bespoke device-based interventions. Boendermaker et al. (2015) found no difference in effectiveness between a web-based and mobile gamified cognitive bias modification training for alcohol use, but did not explicitly design and control for everyday life fit and accessibility as independent variables.

Although the assessed studies included a broad range of game design elements, there was a clear focus on rewards , constituting 16 of a total of 46 instantiations of game design elements across studies (35%), followed by leaderboards and avatars (6 instantiations or 13% each). A notable 84% of all individual studies involved rewards in some form (16 out of 19 studies). Not a single included study captured effects of game design elements on intrinsic motivation as a direct outcome (e.g. motivation to exercise) or mediator for other health and wellbeing outcomes. Taken together with the fact that the majority of studies focused purely behavioural outcomes (see above), this indicates that the dominant theoretical and practical logic of the studied health and wellbeing gamification interventions is positive reinforcement ( Deterding, 2015a, pp. 43–45 ). In other words, the promise of intrinsically motivating health behaviour by taking learnings from game design is currently neither explored nor tested.

Eighteen of the 19 included studies implemented multiple game elements, and no study tested for the independent effects of individual elements. This makes it difficult to attribute effects clearly to individual game elements, and again underlines the need for more rigorously designed studies. With this caveat, the strongest evidence available does support that rewards 5 drive health behaviours: Hamari and Koivisto (2015) found rewards in the form of points and achievements to be associated with improvements in desire to exercise. Thorsteinsen et al. (2014) saw points (in combination with leaderboards) to contribute significantly to increased physical activity. Chen and Pu (2014) similarly found that rewards (badges and points) and leaderboards led to an increase in physical activity among dyads working cooperatively (or working in a hybrid cooperative/competitive mode), but not among dyads working competitively. Allam et al. (2015) found that rewards (points, badges and medals in combination with leaderboards) were associated with increased physical activity and sense of empowerment as well as decreased health care utilization among Rheumatoid Arthritis patients. Cafazzo et al. (2012) saw rewards (in the form of points that could be redeemed for prizes) to contribute to the frequency of blood glucose measurement among individuals with type 1 diabetes. Riva et al. (2014) similarly found a positive impact of points (with leaderboards) on outcomes related to chronic back pain, including reduced medication misuse, lowered pain burden, and increased exercise. With a group of highly trait-anxious participants, Dennis and O'Toole (2014) found rewards (in the form of points) associated with reduced subjective anxiety and stress reactivity.

In contrast to these positive outcomes, Maher et al. (2015) report mixed results: rewards (in combination with leaderboards) led to a short-term (8 week follow-up) increase in moderate to vigorous physical activity, but no long-term effects (20 week follow-up). Similarly, they found no impact of gamification on self-reported general or mental quality of life. Studying a mobile application designed to increase routine walking, Zuckerman and Gal-Oz (2014) similarly found no differences between gamified (points and leaderboards) and non-gamified versions. Relatedly, in a qualitative study of gamified mobile running applications, Spillers and Asimakopoulos (2014) observed poor usability of gamified applications leading to users stopping to use them.

Avatars are commonly employed as a gamification technique to represent the user in the application context. Again, the majority of studies found avatars were associated with positive outcomes. Kuramoto et al. (2013) developed an application with an avatar that ‘grew stronger’ the longer users were standing instead of sitting on public transport. They found evidence for increased motivation to stand. Dennis and O'Toole (2014) compared a gamified mobile attention-bias modification training for anxiety using virtual characters with a placebo training and found it to significantly reduce subjective anxiety and stress reactivity. In a series of two studies, Jones et al., 2014a , Jones et al., 2014b found that avatars (in combination with rewards, levels and narrative) led to increased fruit and vegetable consumption among children. Assessing the effectiveness of a gamified (avatar and backstory) application designed to moderate alcohol use, Boendermaker et al. (2015) observed a positive impact on motivation to train; however, participants reported greater task demand associated with the gamified version of the application.

Social Interaction was also commonly employed as a means to engage users and was found to increase user experiences of fun and motivation in the context of moderating alcohol consumption ( Boendermaker et al., 2015 ), to have a positive influence on physical activity (Juho Hamari and Koivisto, 2015 , Maher et al., 2015 , Spillers and Asimakopoulos, 2014 ) and flourishing mental health ( Hall et al., 2013 ). Less commonly employed gamed design elements across studies included levels ( Cafazzo et al., 2012 ; Juho Hamari and Koivisto, 2015 , Kuramoto et al., 2013 , Ludden et al., 2014 ), progress ( Ahtinen et al., 2013 , Ludden et al., 2014 , Spillers and Asimakopoulos, 2014 ), story/theme ( Boendermaker et al., 2015 , Jones et al., 2014b , Jones et al., 2014a ), challenges ( Ludden et al., 2014 , Spillers and Asimakopoulos, 2014 ) and feedback ( Kadomura et al., 2014 ).

With respect to theories of motivation, very few studies provide insight regarding the extent to which gamification that draws on relevant theory is more effective. Only a minority of studies ( n  = 8) explicitly discuss motivational theory and very few studies ( n  = 3) are conducted in a manner that assesses whether a motivational construct is associated with positive outcomes. Most commonly, self-determination theory and intrinsic/extrinsic motivation were the theories discussed in relation to health gamification ( Hall et al., 2013 ; Juho Hamari and Koivisto, 2015 , Riva et al., 2014 , Spillers and Asimakopoulos, 2014 , Zuckerman and Gal-Oz, 2014 ). Other theories (relevant to motivation) that were considered include design strategies to reduce attrition and guides for behaviour change ( Ahtinen et al., 2013 ), empowerment ( Allam et al., 2015 , Riva et al., 2014 ) and the transtheoretical model of behaviour change ( Reynolds et al., 2013 ).

As discussed above, most studies considered multiple gamification elements simultaneously making it difficult to isolate the effects of individual elements. In some cases, this also makes it more difficult to consider the impact of specific theories of motivation. Hamari and Koivisto (2015) found a positive impact of social norms and recognition providing support for self-determination theory in terms of relatedness of social influence. Similarly, although mixed evidence was found for the impact of the gamification elements used, Zuckerman and Gal-Oz (2014) interpret their results as confirming the value of Nicholson's (2012) concept of ‘meaningful’ gamification and the self-determination driven ideas of informational feedback and customizable elements. Further affirming the notion of ‘meaningful’ gamification, Ahtinen et al. (2013) discuss how their findings highlight the importance of meaningful experiences rather than rewards.

4.3. RQ3. What audiences are targeted? What effect differences between audiences are observed?

A broad range of audiences were targeted throughout the research reviewed. While some studies focussed on younger participants (ranging from Kindergarten age ( Jones et al., 2014b , Kadomura et al., 2014 ) to adolescents ( Cafazzo et al., 2012 ), the majority of studies were conducted with adults. Regardless, positive outcomes have been found for children ( Jones et al., 2014a , Kadomura et al., 2014 ), adolescents ( Cafazzo et al., 2012 ) and young adults ( Kuramoto et al., 2013 , Zuckerman and Gal-Oz, 2014 ).A small number of studies focussed on specific audiences, such primary school teachers ( Ludden et al., 2014 ), participants with specific health issues like chronic back pain Riva et al., 2014 , rheumatoid arthritis ( Allam et al., 2015 ), or high levels of trait anxiety ( Dennis and O'Toole, 2014 ). It is not immediately clear from the reviewed studies what relationship exists between existing gaming affinity or expertise and the effectiveness of gamification as previous experience with digital games is not commonly reported.

Beyond demographics, factors relevant to the potential effectiveness of gamification seem to include the users' personality ( Hall et al., 2013 ), as well as their level of knowledge, expertise, abilities, and basic motivation to engage in the target activity initially. In a study where 15 first-time Wii Fit users were asked to use a Wii balance board to increase their fitness, findings about the effectiveness of gamification were mixed. Only beginners responded positively to gamified elements incorporated into the exercise activities, while these same features had a negative effect on experienced fitness users, leading them to abandon the system as a fitness tool ( Reynolds et al., 2013 ). Non-beginners reported that gamified features slowed down the pace of the exercise, leading to their disengagement, and feedback was disliked, as praising was considered exaggerated.

Importantly, the studies reviewed suggest that the benefits of health gamification extend beyond audiences who have pre-existing motivations to engage in the target activity. Although many ( n  = 11) of the studies involved participants who were likely to have pre-existing motivation, of the studies conducted with participants without existing motivations ( n  = 8), the majority ( n  = 7) showed some positive results. Positive impacts of gamification were found with young children around eating behaviours ( Jones et al., 2014a , Jones et al., 2014b , Kadomura et al., 2014 ); university students regarding alcohol consumption ( Boendermaker et al., 2015 ); commuters with respect to standing Kuramoto et al., 2013 and teachers in relation to positive psychology training. Furthermore, when comparing beginners and experts, Reynolds and colleagues found positive impacts of gamification on exercise behaviour only for the beginners (who are presumably less intrinsically motivated than experts).

4.4. RQ4. What health and well-being domains are targeted?

Across fields, the most popular and successful context for the application of gamification is physical health ( n  = 13) and more specifically, its use for motivating individuals to increase their physical activity, or to engage in self-monitoring of fitness levels ( n  = 10). Notably, a positive impact of gamification on physical activity related outcomes are observed in 8 of the 10 studies with mixed effects observed by Maher et al. (2015) and Spillers and Asimakopoulos (2014) .

Motivation to exercise is increased largely through “fun” activities, through cooperating, competing, and sharing a common goal with peers or exercise buddies (e.g., Chen and Pu, 2014 ), or through various other social incentives (e.g., Spillers and Asimakopoulos, 2014 ). There is evidence that gamification features may be more motivating than exercise alone ( Chen and Pu, 2014 ). Some elements can stimulate increased exercise and reduce physical fatigue ( Kuramoto et al., 2013 . Gamifying fitness is a way to attract users, encourage participation and motivate behaviour change ( Reynolds et al., 2013 ). There is also evidence to suggest that social influence may play a key role in the influence of gamification on willingness to exercise (Juho Hamari and Koivisto, 2015 ). While gamified elements can provide motivation to maintain or increase physical activity, such outcomes may not be sustained over time ( Thorsteinsen et al., 2014 ); these responses are not necessarily consistent for all types of users ( Reynolds et al., 2013 ); and not all types of elements help users achieve their fitness goals or positively impact user adoption ( Spillers and Asimakopoulos, 2014 ). Nevertheless, these studies combined lend support to the use of gamification as a viable intervention strategy in fitness contexts. Outside of activity, within the domain of physical health a positive influence of gamification was also found in three studies of nutrition ( Jones et al., 2014b , Jones et al., 2014a , Kadomura et al., 2014 ).

The remaining studies exploring the impact of gamification within the domain of physical health examined illness related issues. Gamification was found to have a positive influence on healthcare utilization ( Allam et al., 2015 ), the reduction of medication misuse ( Allam et al., 2015 , Riva et al., 2014 ) and blood glucose monitoring ( Cafazzo et al., 2012 ). In two studies these changes were also associated with a positive influence on patient empowerment ( Allam et al., 2015 , Riva et al., 2014 ).

In the domain of mental health, gamification has been shown to have positive effects on wellbeing, personal growth and flourishing ( Hall et al., 2013 , Ludden et al., 2014 ) as well as stress and anxiety ( Dennis and O'Toole, 2014 ). This supports the identified promise of gamification to directly support wellbeing . More mixed results were found with respect to substance use, with evidence of an increased motivation to train with a gamified version of a tool (designed to alter positive associations with alcohol in memory), alongside evidence of lowered ease of use. However, in a study of mental wellness training, which involved concentration, relaxation and other techniques to encourage changes in thoughts and negative beliefs, gamification was received with skepticism by just over half of the users ( Ahtinen et al., 2013 ). Participants suggested that points, rewards and achievements were a poor fit in the context of mental wellness and mindfulness. However, it is not clear to what extent this point of view is related to the specific types of gamification used in the study and whether the finding would extend to a broader sample.

4.5. Limitations

As noted throughout the discussion, the small number and wide variability in the design, quality and health behaviour targets of the gamification studies included in this review limits the conclusions which can be made. There is a need for more well-designed studies comparing gamified and non-gamified interventions: we need randomized controlled trials and double-blind experiments that tease out the effect of individual game design elements on mediators like user experience or motivation and health and wellbeing outcomes, with adequately powered sample sizes, control groups and long-term follow up assessments of outcomes. The studies included in this review typically conflated the assessment of multiple game design elements at once, often involved small sample sizes, did not feature control groups, or only focused on user experience outcomes. Additionally, very few studies have explored the long-term or sustained effects of gamified products, which means that current support for gamification may in part reflect its novelty.

Finally, the heuristic used (positive, negative, neutral) in the current review to evaluate impact, was considered appropriate given the heterogeneity of included studies. However, once more studies on individual gaming elements are completed, future reviews should consider using a more complex heuristic to evaluate impact.

5. Conclusions

As the main contributors to health and wellbeing have shifted towards personal health behaviours, policymakers and health care providers are increasingly looking for interventions that motivate positive health behaviour change, particularly interventions leveraging the capabilities of computing technology. Compared to existing approaches like serious games for health or persuasive technology, gamification has been framed as a promising new alternative that embodies a “new model for health”: “seductive, ubiquitous, lifelong health interfaces” for well-being self-care ( Sawyer, 2014 ). More specifically, proponents of gamification for health and wellbeing have highlighted seven potential advantages of gamification: (1) supporting intrinsic motivation (as games have been shown to motivate intrinsically), (2) broad accessibility through mobile technology and ubiquitous sensors, (3) broad appeal across audiences (as gaming has become mainstream), (4) broad applicability across health and wellbeing risks and factors, (5) cost-benefit efficiency of enhancing existing systems (versus building bespoke games), (6) everyday life fit (reorganising existing activity rather than adding additional demands to people's lives), (7) direct wellbeing support (by providing positive experiences).

That being said, little is known whether and how effectively gamification can drive positive health and wellbeing outcomes, let alone deliver on these promises. In response, we conducted a systematic literature review, identifying 19 papers that report empirical evidence on the effect of gamification on health and wellbeing. Just over half (59%) of the studies reported positive effects, whereas 41% reported mixed or neutral effects. This suggests that gamification could have a positive effect on health and wellbeing, especially when applied in a skilled way. The evidence is strongest for the use of gamification to target behavioural outcomes, particularly physical activity, and weakest for its impact on cognitions. There is also initial support for gamification as a tool to support other physical health related outcomes including nutrition and medication use as well as mental health outcomes including wellbeing, personal growth, flourishing, stress and anxiety. However, evidence for the impact of gamification on the user experience, was mixed. Further research that isolates the impacts of gamification (e.g., randomized controlled trials) is needed to determine its effectiveness in the health and wellbeing domain.

In terms of the highlighted promises, little can be said conclusively. No intervention examined intrinsic motivation support (1), as the majority of studies subscribed to a behaviorist reinforcement paradigm. Most studies did employ mobile and/or ubiquitous technology (2), yet no study directly assessed whether they differed in accessibility compared to stationary delivery modes. The range of participant samples employed across studies suggests likely broad appeal across audiences (3) and the wide range of health and wellbeing issues addressed across studies does support broad applicability (4) in principle. None of the studies included assessed cost-benefit efficiency (5) or everyday life fit (6). On a positive note, multiple studies found evidence that gamified interventions did directly support participants' wellbeing (7).

Acknowledgements

Funding for this project was provided by the Young and Well Cooperative Research Centre, the Digital Creativity Labs (digitalcreativity.ac.uk), jointly funded by EPSRC/AHRC/InnovateUK under grant no EP/M023265/1, and the Movember Foundation (via the Mindmax project).

1 Authors like Deterding et al. (2011) caution to not delimit gamification to a specific design goal like motivation, but grant that motivating behaviours is indeed the overwhelming use case for gamification, as borne out by systematic reviews.

2 https://secure-nikeplus.nike.com/plus/

3 https://zombiesrungame.com

4 http://superbetter.com

5 Because leaderboards were only ever found implemented in conjunction with rewards, we report jointly on both here.

• Ahtinen A., Mattila E., Välkkynen P., Kaipainen K., Vanhala T., Ermes M.…Lappalainen R. Mobile mental wellness training for stress management: feasibility and design implications based on a one-month field study. JMIR mHealth and uHealth. 2013; 1 (2) [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Alahäivälä T., Oinas-Kukkonen H. Understanding persuasion contexts in health gamification: a systematic analysis of gamified health behavior change support systems literature. Int. J. Med. Inform. 2016 [ PubMed ] [ Google Scholar ]
• Allam A., Kostova Z., Nakamoto K., Schulz P.J. The effect of social support features and gamification on a Web-based intervention for rheumatoid arthritis patients: randomized controlled trial. J. Med. Internet Res. 2015; 17 (1) [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Baranowski T., Buday R., Thompson D.I., Baranowski J. Playing for real: video games and stories for health-related behavior change. Am. J. Prev. Med. 2008; 34 (1):74–82. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Boendermaker W.J., Boffo M., Wiers R.W. Exploring elements of fun to motivate youth to do cognitive bias modification. Games Health J. 2015; 4 (6):434–443. [ PubMed ] [ Google Scholar ]
• Breckler S.J. Empirical validation of affect, behavior, and cognition as distinct components of attitude. J. Pers. Soc. Psychol. 1984; 47 (6):1191–1205. http://www.ncbi.nlm.nih.gov/pubmed/6527214 (Retrieved from) [ PubMed ] [ Google Scholar ]
• Cafazzo J.A., Casselman M., Hamming N., Katzman D.K., Palmert M.R. Design of an mHealth app for the self-management of adolescent type 1 diabetes: a pilot study. J. Med. Internet Res. 2012; 14 (3):13. http://gateway.library.qut.edu.au/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2012-32354-015&site=ehost-live (Retrieved from) [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Calvo R.A., Peters D. MIT Press; Cambridge MA: 2014. Positive Computing: Technology for Wellbeing and Human Potential. [ Google Scholar ]
• Carlisle S., Hanlon P. “well-being” as a focus for public health? A critique and defence. Crit. Public Health. 2008; 18 (3):263–270. [ Google Scholar ]
• Cheek C., Fleming T., Lucassen M.F., Bridgman H., Stasiak K., Shepherd M., Orpin P. Integrating health behavior theory and design elements in serious games. JMIR Ment. Health. 2015; 2 (2) [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Chen Y., Pu P. Proceedings of the Second International Symposium of Chinese CHI. ACM; 2014. HealthyTogether: exploring social incentives for mobile fitness applications; pp. 25–34. [ Google Scholar ]
• Connolly T.M., Boyle E.A., MacArthur E., Hainey T., Boyle J.M. A systematic literature review of empirical evidence on computer games and serious games. Comput. Educ. 2012; 59 (2):661–686. [ Google Scholar ]
• Cugelman B. Gamification: what it is and why it matters to digital health behavior change developers. JMIR Serious Games. 2013; 1 (1) [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Deci E.L., Ryan R.M. Motivation, personality, and development within embedded social contexts: an overview of self-determination theory. In: Ryan R.M., editor. The Oxford Handbook of Human Motivation. Oxford University Press; U.S.A.: 2012. pp. 85–107. [ Google Scholar ]
• Dennis T.A., O'Toole L. Mental health on the go: effects of a gamified attention bias modification mobile application in trait anxious adults. Clin. Psychol. Sci. 2014; 2 (5):576–590. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• DeSmet A., Van Ryckeghem D., Compernolle S., Baranowski T., Thompson D., Crombez G.…De Bourdeaudhuij I. A meta-analysis of serious digital games for healthy lifestyle promotion. Prev. Med. 2014; 69 :95–107. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• DeSmet A., Shegog R., Van Ryckeghem D., Crombez G., De Bourdeaudhuij I. A systematic review and meta-analysis of interventions for sexual health promotion involving serious digital games. Games Health J. 2015; 4 (2):78–90. [ PubMed ] [ Google Scholar ]
• Deterding S. Gamification: Using Game Design Elements in Non-gaming Contexts, a Workshop at CHI. 2011. Situated motivational affordances of game elements: a conceptual model. (Retrieved from http://www.quilageo.com/wp-content/uploads/2013/07/09-Deterding.pdf ) [ Google Scholar ]
• Deterding S. The Ambiguity of Games: Histories and Discourses of a Gameful World. In: Walz S.P., editor. The Gameful World: Approaches, Issues, Applications. MIT Press; Cambridge, MA, London: 2015. pp. 23–64. (Retrieved from http://papers.ssrn.com/sol3/Papers.cfm?abstract_id=2463983 ) [ Google Scholar ]
• Deterding S. The lens of intrinsic skill atoms: a method for gameful design. Hum. Comput. Interact. 2015; 30 (3–4):294–335. [ Google Scholar ]
• Deterding S., Dixon D., Khaled R., Nacke L. “From game design elements to gamefulness: Defining ‘gamification,’”. in Proceedings of the 15th International Academic MindTrek Conference; New York, NY, USA: Envisioning Future Media Environments, MindTrek 2011; 2011. pp. 9–15. [ Google Scholar ]
• Fogg B.J. Morgan Kaufmann; Amsterdam: 2003. Persuasive Technology: Using Computers to Change What We Think and Do. [ Google Scholar ]
• Fortier M.S., Duda J.L., Guerin E., Teixeira P.J. Promoting physical activity: development and testing of self-determination theory-based interventions. Int. J. Behav. Nutr. Phys. Act. 2012; 9 :20. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Gao Z., Chen S., Pasco D., Pope Z. A meta-analysis of active video games on health outcomes among children and adolescents: a meta-analysis of active video games. Obes. Rev. Off. J. Int. Assoc. Study Obes. 2015; 16 (9):783–794. [ PubMed ] [ Google Scholar ]
• Glanz K., Bishop D.B. The role of behavioral science theory in development and implementation of public health interventions. Annu. Rev. Public Health. 2010; 31 :399–418. [ PubMed ] [ Google Scholar ]
• Glanz K., Rimer B.K., Viswanath K., editors. Health Behaviour and Health Education: Theory, Research, and Practice. Jossey-Bass; San Francisco: 2008. [ Google Scholar ]
• Hall M., Caton S., Weinhardt C. vol. 8029 LNCS. Karlsruhe Service Research Institute, Karlsruhe Institute of Technology; Germany: 2013. Well-being's Predictive Value a Gamified Approach to Managing Smart Communities; pp. 13–22. (Retrieved from http://www.scopus.com/inward/record.url?eid=2-s2.0-84880864705&partnerID=40&md5=4ac62e5a34f93564c5df31ccffd6e60e ). [ Google Scholar ]
• Hamari J., Koivisto J. “Working out for likes”: an empirical study on social influence in exercise gamification. Comput. Hum. Behav. 2015; 50 :333–347. [ Google Scholar ]
• Hamari J., Koivisto J., Pakkanen T. Do persuasive technologies persuade? — a review of empirical studies. In: Spagnolli A., Chittaro L., Gamberini L., editors. Persuasive Technology. Springer International Publishing; 2014. pp. 118–136. [ Google Scholar ]
• Hamari J., Koivisto J., Sarsa H. 2014 47th Hawaii International Conference on System Sciences. 2014. Does gamification work? — a literature review of empirical studies on gamification; pp. 3025–3034. [ Google Scholar ]
• Hanratty B., Farmer S. The new UK focus on well-being: what will it mean for tackling social inequalities in health? J. Public Health. 2012; 34 (1):2–4. [ PubMed ] [ Google Scholar ]
• Hassenzahl M., Laschke M. 2015. Pleasurable Troublemakers. The Gameful World: Approaches, Issues, Applications; pp. 167–195. (Retrieved from https://www.researchgate.net/profile/Marc_Hassenzahl/publication/266794803_Pleasurable_Troublemakers/links/553a0ed40cf2239f4e7dc302.pdf ) [ Google Scholar ]
• Huppert F.A., So T.T.C. Flourishing across Europe: application of a new conceptual framework for defining well-being. Soc. Indic. Res. 2013; 110 (3):837–861. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Johnson D., Jones C.M., Scholes L., Colder Carras M. 2013. Videogames and Wellbeing: A Comprehensive Review. (Retrieved from https://www.youngandwellcrc.org.au/wp-content/uploads/2014/03/Videogames_and_Wellbeing.pdf ) [ Google Scholar ]
• Jones B.A., Madden G.J., Wengreen H.J., Aguilar S.S., Desjardins E.A. Gamification of dietary decision-making in an elementary-school cafeteria. PLoS One. 2014; 9 (4) [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Jones B.A., Madden G.J., Wengreen H.J. The FIT game: preliminary evaluation of a gamification approach to increasing fruit and vegetable consumption in school. Prev. Med. 2014; 68 :76–79. [ PubMed ] [ Google Scholar ]
• Juul J. The Ludologist. 2011. Gamification backlash roundup. (Retrieved from https://www.jesperjuul.net/ludologist/gamification-backlash-roundup ) [ Google Scholar ]
• Kadomura A., Tsukada K., Siio I. EducaTableware: sound emitting tableware for encouraging dietary education. J. Inf. Process. 2014; 22 (2):325–333. [ Google Scholar ]
• Kharrazi H., Lu A.S., Gharghabi F., Coleman W. A scoping review of health game research: past, present, and future. Games Health J. 2012; 1 (2):153–164. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• King D., Greaves F., Exeter C., Darzi A. “Gamification”: influencing health behaviours with games. J. R. Soc. Med. 2013; 106 (3):76–78. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Knight E., Stuckey M.I., Prapavessis H., Petrella R.J. Public health guidelines for physical activity: Is there an app for that? A review of android and apple app stores. JMIR mHealth and uHealth. 2015; 3 (2) [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Kuramoto I., Ishibashi T., Yamamoto K., Tsujino Y. Design, User Experience, and Usability. Health, Learning, Playing, Cultural, and Cross-cultural User Experience. Springer; Berlin Heidelberg: 2013. Stand up, heroes!: gamification for standing people on crowded public transportation; pp. 538–547. [ Google Scholar ]
• LeBlanc A.G., Chaput J.-P., McFarlane A., Colley R.C., Thivel D., Biddle S.J.H.…Tremblay M.S. Active video games and health indicators in children and youth: a systematic review. PLoS One. 2013; 8 (6) [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lewis Z.H., Swartz M.C., Lyons E.J. What's the point?: a review of reward systems implemented in gamification interventions. Games Health J. 2016; 5 (2):93–99. [ PubMed ] [ Google Scholar ]
• Li I., Dey A., Forlizzi J. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM; 2010. A stage-based model of personal informatics systems; pp. 557–566. [ Google Scholar ]
• Lister C., West J.H., Cannon B., Sax T., Brodegard D. Just a fad? Gamification in health and fitness apps. JMIR Serious Games. 2014; 2 (2) [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lu A.S., Kharrazi H., Gharghabi F., Thompson D. A systematic review of health videogames on childhood obesity prevention and intervention. Games Health J. 2013; 2 (3):131–141. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Ludden G.D.S., Kelders S.M., Snippert B.H.J. Persuasive Technology. vol. 8462. Springer International Publishing; 2014. “This is your life!”: the design of a positive psychology intervention using metaphor to motivate; pp. 179–190. [ Google Scholar ]
• Lyubomirsky S., Layous K. How do simple positive activities increase well-being? Curr. Dir. Psychol. Sci. 2013; 22 (1):57–62. [ Google Scholar ]
• Maher C., Ferguson M., Vandelanotte C., Plotnikoff R., De Bourdeaudhuij I., Thomas S.…Olds T. A web-based, social networking physical activity intervention for insufficiently active adults delivered via Facebook app: randomized controlled trial. J. Med. Internet Res. 2015; 17 (7) [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Marks N., Shah H. A well-being manifesto for a flourishing society. J. Public Ment. Health. 2004; 3 (4):9–15. [ Google Scholar ]
• Martínez-Pérez B., de la Torre-Díez I., López-Coronado M. Mobile health applications for the most prevalent conditions by the World Health Organization: review and analysis. J. Med. Internet Res. 2013; 15 (6) [ PMC free article ] [ PubMed ] [ Google Scholar ]
• McGonigal J. Penguin; 2011. Reality Is Broken: Why Games Make Us Better and How They Can Change the World. (Retrieved from http://fronteiras.rtietz.com/velho/mc_gonigal.pdf ) [ Google Scholar ]
• Mekler E.D., Bopp J.A., Tuch A.N., Opwis K. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM; 2014. A systematic review of quantitative studies on the enjoyment of digital entertainment games; pp. 927–936. [ Google Scholar ]
• Meloni W., Gruener W. Consumer and Enterprise Market Trends. 2012. Gamification in 2012: Market Update. (Encinitas, CA. Retrieved from http://gamingbusinessreview.com/wp-content/uploads/2012/05/Gamification-in-2012-M2R3.pdf ) [ Google Scholar ]
• Michie S., Ashford S., Sniehotta F.F., Dombrowski S.U., Bishop A., French D.P. A refined taxonomy of behaviour change techniques to help people change their physical activity and healthy eating behaviours: the CALO-RE taxonomy. Psychol. Health. 2011; 26 (11):1479–1498. [ PubMed ] [ Google Scholar ]
• Michie S., van Stralen M.M., West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement. Sci.: IS. 2011; 6 :42. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Middelweerd A., Mollee J.S., van der Wal C.N., Brug J., Te Velde S.J. Apps to promote physical activity among adults: a review and content analysis. Int. J. Behav. Nutr. Phys. Act. 2014; 11 :97. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Moher D., Liberati A., Tetzlaff J., Altman D.G., PRISMA Group Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int. J. Surg. 2010; 8 (5):336–341. [ PubMed ] [ Google Scholar ]
• Morschheuser B.S., Rivera-Pelayo V., Mazarakis A., Zacharias V. Interaction and reflection with quantified self and gamification: an experimental study. J. Lit. Technol. 2014; 15 (2) (Retrieved from http://www.literacyandtechnology.org/uploads/1/3/6/8/136889/bm_5.pdf ) [ Google Scholar ]
• Munson S., Poole E., Perry D.B., Peyton T. Gamification and health. In: Walz S.P., editor. The Gameful World: Approaches, Issues, Applications. MIT Press; Cambridge, London: 2015. pp. 597–623. [ Google Scholar ]
• Ng J.Y.Y., Ntoumanis N., Thøgersen-Ntoumani C., Deci E.L., Ryan R.M., Duda J.L., Williams G.C. Self-determination theory applied to health contexts: a meta-analysis. Perspect. Psychol. Sci. J. Assoc. Psychol. Sci. 2012; 7 (4):325–340. [ PubMed ] [ Google Scholar ]
• Nicholson S. Proceedings of Games + Learning + Society 8.0 (GLS 8.0) 2012. A user-centered theoretical framework for meaningful gamification. (Retrieved from http://scottnicholson.com/pubs/meaningfulframework.pdf ) [ Google Scholar ]
• Oinas-Kukkonen H., Harjumaa M. Persuasive systems design: key issues, process model, and system features. Commun. Assoc. Inf. Syst. 2009; 24 (1):28. (Retrieved from http://aisel.aisnet.org/cgi/viewcontent.cgi?article=3424&context=cais ) [ Google Scholar ]
• Papastergiou M. Exploring the potential of computer and video games for health and physical education: a literature review. Comput. Educ. 2009; 53 (3):603–622. [ Google Scholar ]
• Parker, S. G. (n.d.). Advancing the Field of Health Games: A Progress Report on Health Games Research. (Retrieved from http://www.rwjf.org/en/library/research/2011/03/advancing-the-field-of-health-games.html (Original work published 2011)).
• Patrick H., Williams G.C. Self-determination theory: its application to health behavior and complementarity with motivational interviewing. Int. J. Behav. Nutr. Phys. Act. 2012; 9 :18. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Pereira P., Duarte E., Rebelo F., Noriega P. Design, User Experience, and Usability. User Experience Design for Diverse Interaction Platforms and Environments. Springer International Publishing; 2014. A review of gamification for health-related contexts; pp. 742–753. [ Google Scholar ]
• Primack B.A., Carroll M.V., McNamara M., Klem M.L., King B., Rich M.…Nayak S. Role of video games in improving health-related outcomes: a systematic review. Am. J. Prev. Med. 2012; 42 (6):630–638. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Przybylski A.K., Rigby C.S., Ryan R.M. A motivational model of video game engagement. Rev. Gen. Psychol. J. Div. 1, Am. Psychol. Assoc. 2010; 14 (2):154–166. [ Google Scholar ]
• Reeve J. Wiley; 2014. Understanding Motivation and Emotion. (Retrieved from https://books.google.com.au/books?id=zfOyBwAAQBAJ ) [ Google Scholar ]
• Reynolds L., Sosik V.S., Cosley D. Proceedings of the First International Conference on Gameful Design, Research, and Applications. ACM; 2013. When Wii doesn't fit: how non-beginners react to Wii fit's gamification; pp. 111–114. (http://doi.org/10.1145/2583008.2583027) [ Google Scholar ]
• Riva S., Camerini A.-L., Allam A., Schulz P.J. Interactive sections of an Internet-based intervention increase empowerment of chronic back pain patients: randomized controlled trial. J. Med. Internet Res. 2014; 16 (8) [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Ryan R.M., Rigby C.S. Praeger; CA: 2011. Glued to Games: How Video Games Draw Us in and Hold Us Spellbound. Santa Barbara. [ Google Scholar ]
• Ryan R.M., Patrick H., Deci E.L., William G.C. Facilitating health behaviour change and its maintenance: interventions based on self-determination theory. Eur. Health Psychol. 2008; 10 :2–5. http://openhealthpsychology.com/ehp/index.php/contents/article/viewFile/ehp.v10.i1.p2/32 Retrieved from. [ Google Scholar ]
• Ryan R.M., Huta V., Deci E.L. Living well: a self-determination theory perspective on eudaimonia. J. Happiness Stud. 2008; 9 (1):139–170. [ Google Scholar ]
• Sawyer B. MIT; Presented at the MIT Media Lab Talks, Cambridge, MA: 2014. Games for Health. https://www.media.mit.edu/video/view/wellbeing-2014-11-05 Retrieved from. [ Google Scholar ]
• Schroeder S.A. We can do better — improving the health of the American people. N. Engl. J. Med. 2007; 357 (12):1221–1228. [ PubMed ] [ Google Scholar ]
• Schulte P.A., Guerin R.J., Schill A.L., Bhattacharya A., Cunningham T.R., Pandalai S.P.…Stephenson C.M. Considerations for incorporating “well-being” in public policy for workers and workplaces. Am. J. Public Health. 2015; 105 (8):e31–e44. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Seaborn K., Fels D.I. Gamification in theory and action: a survey. Int. J. Hum. Comput. Stud. 2015; 74 :14–31. [ Google Scholar ]
• Seligman M. Nicholas Brealey Publishing; London: 2011. Flourish: A New Understanding of Happiness and Wellbeing. [ Google Scholar ]
• Shrout P.E., Fleiss J.L. Intraclass correlations: uses in assessing rater reliability. Psychol. Bull. 1979; 86 (2):420–428. http://www.ncbi.nlm.nih.gov/pubmed/18839484 (Retrieved from) [ PubMed ] [ Google Scholar ]
• Sola D., Couturier J., Voyer B. Unlocking patient activation in chronic disease care. Br. J. Healthc. Manag. 2015 [ Google Scholar ]
• Spillers F., Asimakopoulos S. Design, User Experience, and Usability. User Experience Design Practice. Springer; 2014. A diary study comparing mobile health applications; pp. 358–369. [ Google Scholar ]
• Stevens G., Mascarenhas M., Mathers C. Global health risks: progress and challenges. Bull. World Health Organ. 2009; 87 (9):646. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Tamborini R., Grizzard M., David Bowman N., Reinecke L., Lewis R.J., Eden A. Media enjoyment as need satisfaction: the contribution of hedonic and nonhedonic needs. J. Commun. 2011; 61 (6):1025–1042. [ Google Scholar ]
• Teixeira P.J., Silva M.N., Mata J., Palmeira A.L., Markland D. Motivation, self-determination, and long-term weight control. Int. J. Behav. Nutr. Phys. Act. 2012; 9 :22. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Teixeira P.J., Palmeira A.L., Vansteenkiste M. The role of self-determination theory and motivational interviewing in behavioral nutrition, physical activity, and health: an introduction to the IJBNPA special series. Int. J. Behav. Nutr. Phys. Act. 2012; 9 :17. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Theng Y.-L., Lee J.W.Y., Patinadan P.V., Foo S.S.B. The use of videogames, gamification, and virtual environments in the self-management of diabetes: a systematic review of evidence. Games Health J. 2015; 4 (5):352–361. [ PubMed ] [ Google Scholar ]
• Thorsteinsen K., Vittersø J., Svendsen G.B. Increasing physical activity efficiently: an experimental pilot study of a website and mobile phone intervention. Int. J. Telemed. Appl. 2014; 2014 :746232. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Vaughan G.M., Hogg M.A. Prentice-Hall Inc. xv; Upper Saddle River, NJ: 1995. Introduction to Social Psychology. [ Google Scholar ]
• Walz S.P., Deterding S. MIT Press; 2015. An Introduction to the Gameful World; pp. 1–13. (Retrieved from https://books.google.com/books?hl=en&lr=&id=KDxTBgAAQBAJ&oi=fnd&pg=PA1&dq=An%2BIntroduction%2Bto%2Bthe%2BGameful%2BWorld&ots=hJ64TO7KDT&sig=cbtDFwM7_rzlVlSUrAALfmeS8r0 ) [ Google Scholar ]
• Wattanasoontorn V., Boada I., García R., Sbert M. Serious games for health. Entertain. Comput. 2013; 4 (4):231–247. [ Google Scholar ]
• Wolf G.I. Quantified self. 2009. http://web.archive.org/web/20091106094426/http://aether.com/quantifiedself (Retrieved December 5, 2016, from)
• Woolf S.H. The power of prevention and what it requires. JAMA. 2008; 299 (20):2437–2439. [ PubMed ] [ Google Scholar ]
• World Health Organization WHO Definition of Health. 1946. http://www.who.int/about/definition/en/print.html (Retrieved December 5, 2016, from)
• Wouters P., van Nimwegen C., van Oostendorp H., van der Spek E.D. A meta-analysis of the cognitive and motivational effects of serious games. J. Educ. Psychol. 2013; 105 (2):249–265. [ Google Scholar ]
• Zhang P. Motivational affordances: reasons for ICT design and use. Commun. ACM. 2008; 51 (11):145–147. [ Google Scholar ]
• Zuckerman O., Gal-Oz A. Deconstructing gamification: evaluating the effectiveness of continuous measurement, virtual rewards, and social comparison for promoting physical activity. Pers. Ubiquit. Comput. 2014; 18 (7):1705–1719. [ Google Scholar ]

To read this content please select one of the options below:

Please note you do not have access to teaching notes, a systematic review of gamification in organizational learning.

The Learning Organization

ISSN : 0969-6474

Article publication date: 6 December 2022

Issue publication date: 27 April 2023

Gamification appears to be one of the most important levers for improving performance and engagement in training programs. Moreover, gamification has been considered as critical to researchers in organizational learning. The main purpose of this study is to provide a general overview of gamification studies in the organizational context.

Design/methodology/approach

This systematic literature review has been synthesized by reviewing the majority of literature reviews as well as carefully selected primary research studies published between 2010 and 2020.

The result of the study revealed that gamification has become a popular technique to improve employee and organizational capabilities. The features of the gamification have been reviewed from three mechanics, dynamics and aesthetics perspectives. Additionally, three primary, middle and optimal affordances and core functions of gamification in organizational learning have been identified and examined.

Originality/value

This paper fills a gap in the literature regarding the gamification field for systematic review results in an organizational learning context.

• Systematic review
• Gamification
• Organizational learning
• Affordances
• Core functions

Khodabandelou, R. , Roghanian, P. , Gheysari, H. and Amoozegar, A. (2023), "A systematic review of gamification in organizational learning", The Learning Organization , Vol. 30 No. 2, pp. 251-272. https://doi.org/10.1108/TLO-05-2022-0057

Emerald Publishing Limited

Copyright © 2022, Emerald Publishing Limited

Related articles

All feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

Revealing the theoretical basis of gamification: : A systematic review and analysis of theory in research on gamification, serious games and game-based learning

New Citation Alert added!

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

Information & Contributors

Bibliometrics & citations, view options.

• Çakar V Karadağ A (2024) Development and Evaluation of a Serious Game on Pressure Injury Prevention for the Training of Patient Relatives Simulation and Gaming 10.1177/10468781241237251 55 :4 (576-599) Online publication date: 1-Aug-2024 https://dl.acm.org/doi/10.1177/10468781241237251
• Hadan H Zhang-Kennedy L Nacke L Mäkelä V (2024) Gamification and Gaming in Cryptocurrency Education Simulation and Gaming 10.1177/10468781231223762 55 :2 (196-223) Online publication date: 10-Apr-2024 https://dl.acm.org/doi/10.1177/10468781231223762
• Low H Ellefson M (2024) Punnett Farms Simulation and Gaming 10.1177/10468781231220728 55 :2 (302-322) Online publication date: 10-Apr-2024 https://dl.acm.org/doi/10.1177/10468781231220728
• Show More Cited By

Index Terms

Applied computing

Computers in other domains

Personal computers and PC applications

Computer games

Human-centered computing

Human computer interaction (HCI)

Information systems

Information systems applications

Multimedia information systems

Massively multiplayer online games

Software and its engineering

Software organization and properties

Contextual software domains

Virtual worlds software

Interactive games

Recommendations

Gamification mechanics for behavioral change: a systematic review and proposed taxonomy.

In the last few years, gamification has been proven as an effective strategy to improve people's motivation and performance. Many authors have reported success examples of gamification in areas such as education, entertainment, health and business. This ...

Challenging games help students learn

In this paper, we investigate the impact of flow (operationalized as heightened challenge and skill), engagement, and immersion on learning in game-based learning environments. The data was gathered through a survey from players (N = 173) of two ...

Gamification, Serious Games, Ludic Simulation, and other Contentious Categories

This paper provides a conceptual framework for gamification, ludic simulations, and serious games. Central to this framework is the spectrum of design that differentiates work and play. Work and play help define software in purpose as games, ...

Information

Published in.

Elsevier Science Publishers B. V.

Netherlands

Publication History

Author tags.

• Gamification
• Serious games
• Game-based learning
• Systematic review
• Self-determination theory
• Research-article

Contributors

Other metrics, bibliometrics, article metrics.

• 31 Total Citations View Citations
• 0 Total Downloads
• Downloads (Last 12 months) 0
• Downloads (Last 6 weeks) 0
• Tyack A Mekler E (2024) Self-Determination Theory and HCI Games Research: Unfulfilled Promises and Unquestioned Paradigms ACM Transactions on Computer-Human Interaction 10.1145/3673230 31 :3 (1-74) Online publication date: 15-Jun-2024 https://dl.acm.org/doi/10.1145/3673230
• Min A Trueblood Miller W Connelly K Nippert-Eng C Lin H Shih P (2024) Enhancing Epilepsy Awareness and Cooperative Care in Elementary and Middle Schools Proceedings of the ACM on Human-Computer Interaction 10.1145/3641003 8 :CSCW1 (1-25) Online publication date: 26-Apr-2024 https://dl.acm.org/doi/10.1145/3641003
• Scott-Sharoni S Mehrotra S Salubre K Song M Misu T Akash K (2024) Can we enhance prosocial behavior? Using post-ride feedback to improve micromobility interactions Proceedings of the 16th International Conference on Automotive User Interfaces and Interactive Vehicular Applications 10.1145/3640792.3675739 (294-304) Online publication date: 22-Sep-2024 https://dl.acm.org/doi/10.1145/3640792.3675739
• Hong Y Saab N Admiraal W (2024) Approaches and game elements used to tailor digital gamification for learning Computers & Education 10.1016/j.compedu.2024.105000 212 :C Online publication date: 1-Apr-2024 https://dl.acm.org/doi/10.1016/j.compedu.2024.105000
• Perez-Aranda J Medina-Claros S Urrestarazu-Capellán R (2024) Effects of a collaborative and gamified online learning methodology on class and test emotions Education and Information Technologies 10.1007/s10639-023-11879-2 29 :2 (1823-1855) Online publication date: 1-Feb-2024 https://dl.acm.org/doi/10.1007/s10639-023-11879-2
• Dahalan F Alias N Shaharom M (2024) Gamification and Game Based Learning for Vocational Education and Training: A Systematic Literature Review Education and Information Technologies 10.1007/s10639-022-11548-w 29 :2 (1279-1317) Online publication date: 1-Feb-2024 https://dl.acm.org/doi/10.1007/s10639-022-11548-w
• Yu D Yue A Rau P (2024) A Gamified Approach on Inducing Energy Conservation Behavior Cross-Cultural Design 10.1007/978-3-031-60898-8_10 (147-160) Online publication date: 29-Jun-2024 https://dl.acm.org/doi/10.1007/978-3-031-60898-8_10

View options

Login options.

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Share this publication link.

Copying failed.

Share on social media

Affiliations, export citations.

• Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
• Download citation
• Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Gamification for health and wellbeing: A systematic review of the literature

Affiliations.

• 1 Queensland University of Technology (QUT), GPO Box 2434, Brisbane, QLD 4001, Australia.
• 2 Digital Creativity Labs, University of York, York YO10 5GE, United Kingdom.
• PMID: 30135818
• PMCID: PMC6096297
• DOI: 10.1016/j.invent.2016.10.002

Background: Compared to traditional persuasive technology and health games, gamification is posited to offer several advantages for motivating behaviour change for health and well-being, and increasingly used. Yet little is known about its effectiveness.

Aims: We aimed to assess the amount and quality of empirical support for the advantages and effectiveness of gamification applied to health and well-being.

Methods: We identified seven potential advantages of gamification from existing research and conducted a systematic literature review of empirical studies on gamification for health and well-being, assessing quality of evidence, effect type, and application domain.

Results: We identified 19 papers that report empirical evidence on the effect of gamification on health and well-being. 59% reported positive, 41% mixed effects, with mostly moderate or lower quality of evidence provided. Results were clear for health-related behaviours, but mixed for cognitive outcomes.

Conclusions: The current state of evidence supports that gamification can have a positive impact in health and wellbeing, particularly for health behaviours. However several studies report mixed or neutral effect. Findings need to be interpreted with caution due to the relatively small number of studies and methodological limitations of many studies (e.g., a lack of comparison of gamified interventions to non-gamified versions of the intervention).

Keywords: Gamification; Health; Systematic review; Wellbeing.

PubMed Disclaimer

Flow diagram.

Histogram of quality of evidence…

Histogram of quality of evidence ratings.

Similar articles

• The Effects of Gamification on Computerized Cognitive Training: Systematic Review and Meta-Analysis. Vermeir JF, White MJ, Johnson D, Crombez G, Van Ryckeghem DML. Vermeir JF, et al. JMIR Serious Games. 2020 Aug 10;8(3):e18644. doi: 10.2196/18644. JMIR Serious Games. 2020. PMID: 32773374 Free PMC article.
• Understanding persuasion contexts in health gamification: A systematic analysis of gamified health behavior change support systems literature. Alahäivälä T, Oinas-Kukkonen H. Alahäivälä T, et al. Int J Med Inform. 2016 Dec;96:62-70. doi: 10.1016/j.ijmedinf.2016.02.006. Epub 2016 Feb 18. Int J Med Inform. 2016. PMID: 26944611
• Recommendations for Implementing Gamification for Mental Health and Wellbeing. Cheng VWS. Cheng VWS. Front Psychol. 2020 Dec 7;11:586379. doi: 10.3389/fpsyg.2020.586379. eCollection 2020. Front Psychol. 2020. PMID: 33365001 Free PMC article. Review.
• The Effects of mHealth-Based Gamification Interventions on Participation in Physical Activity: Systematic Review. Xu L, Shi H, Shen M, Ni Y, Zhang X, Pang Y, Yu T, Lian X, Yu T, Yang X, Li F. Xu L, et al. JMIR Mhealth Uhealth. 2022 Feb 3;10(2):e27794. doi: 10.2196/27794. JMIR Mhealth Uhealth. 2022. PMID: 35113034 Free PMC article. Review.
• Gamification of Cognitive Assessment and Cognitive Training: A Systematic Review of Applications and Efficacy. Lumsden J, Edwards EA, Lawrence NS, Coyle D, Munafò MR. Lumsden J, et al. JMIR Serious Games. 2016 Jul 15;4(2):e11. doi: 10.2196/games.5888. JMIR Serious Games. 2016. PMID: 27421244 Free PMC article.
• The regulatory status of health apps that employ gamification. Freyer O, Wrona KJ, de Snoeck Q, Hofmann M, Melvin T, Stratton-Powell A, Wicks P, Parks AC, Gilbert S. Freyer O, et al. Sci Rep. 2024 Sep 9;14(1):21016. doi: 10.1038/s41598-024-71808-2. Sci Rep. 2024. PMID: 39251786 Free PMC article.
• Gamified Mobile App (MobERAS) for Telemonitoring Patients in the Postoperative Period Based on the Enhanced Recovery after Surgery Program: Development and Validation Study. Santiago AE, Cruz VPG, Furtado RS, Cândido EB, Brandão WC, Silva Filho AL. Santiago AE, et al. JMIR Perioper Med. 2024 Aug 14;7:e56033. doi: 10.2196/56033. JMIR Perioper Med. 2024. PMID: 39141909 Free PMC article.
• The Impact of a Gamified Intervention on Daily Steps in Real-Life Conditions: Retrospective Analysis of 4800 Individuals. Mazéas A, Forestier C, Harel G, Duclos M, Chalabaev A. Mazéas A, et al. J Med Internet Res. 2024 Aug 12;26:e47116. doi: 10.2196/47116. J Med Internet Res. 2024. PMID: 39133533 Free PMC article.
• A randomized trial of behavioral interventions yielding sustained reductions in distracted driving. Ebert JP, Xiong RA, Khan N, Abdel-Rahman D, Leitner A, Everett WC, Gaba KL, Fisher WJ, McDonald CC, Winston FK, Rosin RM, Volpp KG, Barnett IJ, Wiebe DJ, Halpern SD, Delgado MK. Ebert JP, et al. Proc Natl Acad Sci U S A. 2024 Aug 6;121(32):e2320603121. doi: 10.1073/pnas.2320603121. Epub 2024 Jul 29. Proc Natl Acad Sci U S A. 2024. PMID: 39074277 Free PMC article. Clinical Trial.
• PEMOCS: Evaluating the effects of a concept-guided, PErsonalised, MOtor-Cognitive exergame training on cognitive functions and gait in chronic Stroke-study protocol for a randomised controlled trial. Huber SK, Knols RH, Held JPO, Betschart M, de Bruin ED. Huber SK, et al. Trials. 2024 Jul 4;25(1):451. doi: 10.1186/s13063-024-08283-7. Trials. 2024. PMID: 38965612 Free PMC article.
• Ahtinen A., Mattila E., Välkkynen P., Kaipainen K., Vanhala T., Ermes M.…Lappalainen R. Mobile mental wellness training for stress management: feasibility and design implications based on a one-month field study. JMIR mHealth and uHealth. 2013;1(2) - PMC - PubMed
• Alahäivälä T., Oinas-Kukkonen H. Understanding persuasion contexts in health gamification: a systematic analysis of gamified health behavior change support systems literature. Int. J. Med. Inform. 2016 - PubMed
• Allam A., Kostova Z., Nakamoto K., Schulz P.J. The effect of social support features and gamification on a Web-based intervention for rheumatoid arthritis patients: randomized controlled trial. J. Med. Internet Res. 2015;17(1) - PMC - PubMed
• Baranowski T., Buday R., Thompson D.I., Baranowski J. Playing for real: video games and stories for health-related behavior change. Am. J. Prev. Med. 2008;34(1):74–82. - PMC - PubMed
• Boendermaker W.J., Boffo M., Wiers R.W. Exploring elements of fun to motivate youth to do cognitive bias modification. Games Health J. 2015;4(6):434–443. - PubMed

Publication types

• Search in MeSH

Related information

• Cited in Books

LinkOut - more resources

Full text sources.

• Elsevier Science
• Europe PubMed Central
• PubMed Central

Other Literature Sources

• scite Smart Citations

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

A systematic literature review of gamification in/for cultural heritage: leveling up, going beyond.

1. Introduction

2. methodological considerations.

• What heritage is being gamified?
• Are the gamification projects more institution-oriented or tourist-oriented?
• What gamification design dimensions are present in the project?
• What technologies were used in the gamification project?
• What are the target publics of the gamification project?
• What future directions do the authors wish to see explored?

3. Findings and Discussion

3.1. q1. what heritage is being gamified, 3.2. q2. are the gamification projects more institution-oriented or tourism-oriented, 3.3. q3. what gamification design dimensions are present in the project.

• Intrinsic Motivation Heuristics 1.1. “Purpose and Meaning”, affordances aimed at helping users identify a meaningful goal that will be achieved through the system and can benefit the users themselves or other people; 1.2. “Challenge and Competence”, affordances aimed at helping users satisfy their intrinsic need of competence through accomplishing difficult challenges or goals; 1.3. “Completeness and Mastery”, affordances aimed at helping users satisfy their intrinsic need of competence by completing series of tasks or collecting virtual achievements; 1.4. “Autonomy and Creativity”, affordances aimed at helping users satisfy their intrinsic need of autonomy by offering meaningful choices and opportunities for self-expression; 1.5. “Relatedness”, affordances aimed at helping users satisfy their intrinsic need of relatedness through social interaction, usually with other users; 1.6. “Immersion”, affordances aimed at immersing users into the system to improve their aesthetic experience, usually by means of a theme, narrative or story, which can be real or fictional.
• Extrinsic Motivation Heuristics 2.1. “Ownership and Rewards” affordances aimed at motivating users through extrinsic rewards or possession of real or virtual goods. Ownership is different from competence when acquiring goods is perceived by the user as the reason for interacting with the system, instead of feeling competent; 2.2. “Scarcity”, affordances aimed at motivating users through feelings of status or exclusivity by means of acquisition of difficult or rare rewards, goods or achievements; 2.3. “Loss Avoidance”, affordances aimed at leading users to act with urgency, by creating situations in which they could loose acquired or potential rewards, goods or achievements if they do not act immediately.
• Context Dependent Heuristics 3.1. “Feedback”, affordances aimed at informing users of their progress and the next available actions or challenges; 3.2. “Unpredictability”, affordances aimed at surprising users with variable tasks, challenges, feedback or rewards; 3.3. “Change and Disruption”, affordances aimed at engaging users with disruptive tendencies by allowing them to help improve the system, in a positive rather than destructive way [ 96 ].

3.4. Q4. What Technologies Were Used in the Gamification Project?

3.5. q5. what are the target publics of the gamification project.

• “General public”, for everyone or unspecified;
• “Children”, students between 5 and 12 years old;
• “Teenagers”, students and adolescents between 13 and 17 years old;
• “Young Adults”, students and young workers between 18 and 25 years old;
• “Tourists or visitors”, specifically those not from that location but visiting in some capacity, regardless of age;
• “Residents”, specifically those living in that location, regardless of age.

3.6. Q6. What Future Directions Do the Authors Wish to See Explored?

• “Continue the work”, meaning that the project would remain ongoing following the publication of the text;
• “Expand the work”, meaning that the project would not just continue but increase its areas of applicability, scope or objects of interest;
• “Looking for new ideas”, meaning that the authors will (themselves) seek to engage in concrete debates in the future;
• “More reflection is needed”, meaning that the authors point to specific questions left unanswered or unearthed by their study, and leave it up for debate for the broader academic community;
• “Unspecified”, meaning no discernable mention of any future work or orientation.

4. Concluding Remarks

Author contributions, data availability statement, conflicts of interest.

• Osatuyi, B.; Osatuyi, T.; De La Rosa, R. Systematic Review of Gamification Research in IS Education: A Multi-Method Approach. Commun. Assoc. Inf. Syst. 2018 , 42 , 95–124. [ Google Scholar ] [ CrossRef ]
• Prakash, D.; Manchanda, P. Designing a Comprehensive Gamification Model and Pertinence in Organizational Context to Achieve Sustainability. Cogent Bus. Manag. 2021 , 8 , 1962231. [ Google Scholar ] [ CrossRef ]
• Heide, A.; Želinský, D. ‘Level up your money game’: An analysis of gamification discourse in financial services. J. Cult. Econ. 2021 , 14 , 711–731. [ Google Scholar ] [ CrossRef ]
• Ouariachi, T.; Wim, E. Escape rooms as tools for climate change education: An exploration of initiatives. Environ. Educ. Res. 2020 , 26 , 1193–1206. [ Google Scholar ] [ CrossRef ]
• Cavada, M. Evaluate Space after COVID-19: Smart City Strategies for Gamification. Int. J. Hum. Comput. Interact. 2023 , 39 , 319–330. [ Google Scholar ] [ CrossRef ]
• Xu, F.; Tian, F.; Buhalis, D.; Weber, J.; Zhang, H. Tourists as Mobile Gamers: Gamification for Tourism Marketing. J. Travel Tour. Mark. 2016 , 33 , 1124–1142. [ Google Scholar ] [ CrossRef ]
• Alivizatou, M. Digital Intangible Heritage: Inventories, Virtual Learning and Participation. Herit. Soc. 2019 , 12 , 116–135. [ Google Scholar ] [ CrossRef ]
• Seitsonen, O. Crowdsourcing Cultural Heritage: Public Participation and Conflict Legacy in Finland. J. Community Archaeol. Herit. 2017 , 4 , 115–130. [ Google Scholar ] [ CrossRef ]
• Deterding, S.; Dixon, D.; Khaled, R.; Nacke, L. From Game Design Elements to Gamefulness: Defining “Gamification”. In Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, Tampere, Finland, 28–30 September 2011; pp. 9–15. [ Google Scholar ] [ CrossRef ]
• Dicheva, D.; Dichev, C.; Agre, G.; Angelova, G. Gamification in education: A systematic mapping study. J. Educ. Technol. Soc. 2015 , 18 , 75–88. [ Google Scholar ]
• Sampaio, M.C.; Sousa, M.J.; Dionísio, A. The Use of Gamification in Knowledge Management Processes: A Systematic Literature Review. J. Knowl. Manag. 2020 , 24 , 2329–2353. [ Google Scholar ] [ CrossRef ]
• Hassan, L.; Hamari, J. Gamification of E-Participation: A Literature Review. In Proceedings of the 52nd Hawaii International Conference on System Sciences (HICSS’52), Maui, HI, USA, 5–8 January 2019. [ Google Scholar ] [ CrossRef ]
• Khan, I.; Melro, A.; Amaro, A.C.; Oliveira, L. Systematic Review on Gamification and Cultural Heritage Dissemination. J. Digit. Media Interact. 2020 , 3 , 19–41. [ Google Scholar ] [ CrossRef ]
• Marques, C.G.; Pedro, J.P.; Dionísio, M.; Almeida, P.; Silva, C.P. A systematic literature review of Gamification in Cultural Heritage: Where are we? Where do we go? J. Tour. Herit. Res. 2022 , 5 , 64–83. [ Google Scholar ]
• Amado, J.; Costa, A.P.; Crusoé, N. A técnica da análise de conteúdo. In Manual de Investigação Qualitativa em Educação , 1st ed.; Amado, J., Ed.; University of Coimbra Press: Coimbra, Portugal, 2013; pp. 301–351. [ Google Scholar ]
• Pasca, M.G.; Renzi, M.F.; Pietro, L.; Mugion, R.G. Gamification in Tourism and Hospitality Research in the Era of Digital Platforms: A Systematic Literature Review. J. Serv. Theory Pract. 2021 , 31 , 691–737. [ Google Scholar ] [ CrossRef ]
• Deggim, S.; Kersten, T.P.; Tschirschwitz, F.; Hinrichsen, N. Segeberg 1600-reconstructing a historic town for virtual reality visualization as an immersive experience. ISPRS Arch. 2017 , 42 , 87–94. [ Google Scholar ]
• Vayanou, M.; Loumos, G.; Kargas, A.; Ioannidis, Y.; Sidiropoulou, O. Designing performative, gamified cultural experiences for groups. In Proceedings of the Conference on Human Factors in Computing Systems, Glasgow, UK, 4–9 May 2019. [ Google Scholar ]
• Luimula, M.; Trygg, N.B. Cultural heritage in a pocket: Case study “Turku castle in your hand”. In Proceedings of the 7th IEEE International Conference on Cognitive Infocommunications, Wroclaw, Poland, 16–18 October 2016. [ Google Scholar ] [ CrossRef ]
• Pantile, D.; Frasca, R.; Mazzeo, A.; Ventrella, M.; Verreschi, G. New Technologies and Tools for Immersive and Engaging Visitor Experiences in Museums: The Evolution of the Visit-Actor in Next-Generation Storytelling, through Augmented and Virtual Reality, and Immersive 3D Projections. In Proceedings of the 12th International Conference Signal Image Technology & Internet-Based Systems (SITIS 2016), Naples, Italy, 28 November–1 December 2016; pp. 463–467. [ Google Scholar ]
• Georgiev, V.; Nikolova, A. Virtual reality simulations for presenting cultural-historic content in e-learning for kids. Digit. Present. Preserv. Cult. Sci. Herit. 2021 , 11 , 267–272. [ Google Scholar ] [ CrossRef ]
• Cordeiro, P.A.N.; Sousa, J.P.; Carvalho, A. Digitization and Gamification in Cultural Heritage: The Portuguese context in the framework of national and international policies and some practical examples. In Proceedings of the 2021 16th Iberian Conference on Information Systems and Technologies (CISTI), Chaves, Portugal, 23 June 2021. [ Google Scholar ] [ CrossRef ]
• Alatrash, S.; Arnab, S.; Antlej, K. Interactive storytelling experience for museums in the era of COVID-19. In Proceedings of the European Conference on Games Based Learning, Brighton, UK, 24 September 2021; pp. 795–802. [ Google Scholar ]
• MacLeod, K.; Reid, A.J.; Donald, I.; Smith, K. Cumbrae Craft: A Virtual Environment for Teaching Cultural Heritage to Primary School Children. In Proceedings of the European Conference on Games-Based Learning, Brighton, UK, 23 September 2021; pp. 499–508. [ Google Scholar ]
• Vlizos, S.; Sharamyeva, J.A.; Kotsopoulos, K. Interdisciplinary Design of an Educational Applications Development Platform in a 3D Environment Focused on Cultural Heritage Tourism. In Proceedings of the Emerging Technologies and the Digital Transformation of Museums and Heritage Sites: First International Conference, RISE IMET 2021, Nicosia, Cyprus, 2–4 June 2021; pp. 79–96. [ Google Scholar ]
• Xiang, Y.; Wang, J.; Fa, J.; Gu, N.; Lo, C.H. Disseminating Intangible Cultural Heritage through Gamified Learning Experiences and Service Design. In Proceedings of the Cross-Cultural Design. Applications in Cultural Heritage, Tourism, Autonomous Vehicles, and Intelligent Agents: 13th International Conference, CCD 2021, Held as Part of the 23rd HCI International Conference, HCII 2021, Virtual, 24–29 July 2021; pp. 116–128. [ Google Scholar ]
• Alatrash, S.; Arnab, S.; Antlej, K. The Potential of Implementing Interactive Storytelling Experience for Museums. In Proceedings of the Digital Heritage. Progress in Cultural Heritage: Documentation, Preservation, and Protection: 8th International Conference, EuroMed 2020, Virtual, 2–5 November 2020; pp. 713–721. [ Google Scholar ]
• Roy, S.; Singh, P.P.; Padun, A. Game-Based Learning for the Awareness of Culture & Tradition: An Exploratory Case Study on the Indigenous Naga Tribe. In Design for Tomorrow—Volume 2: Proceedings of ICoRD 2021 ; Springer: Singapore, 2021; pp. 293–304. [ Google Scholar ]
• Liberato, D.; Nunes, M.; Liberato, P. Wine and Food Tourism Gamification. Exploratory Study in Peso da Régua. Smart Innov. Syst. Technol. 2021 , 497–508. [ Google Scholar ] [ CrossRef ]
• Kontiza, K.; Liapis, A.; Jones, C.E. Reliving the Experience of Visiting a Gallery: Methods for Evaluating Informal Learning in Games for Cultural Heritage. In Proceedings of the 15th International Conference on the Foundations of Digital Games, Bugibba, Malta, 15–18 September 2020. [ Google Scholar ] [ CrossRef ]
• Lopez-Martinez, A.; Iglesias, C.A.; Carrera, A. Gamified Smart Objects for Museums Based on Automatically Generated Quizzes Exploiting Linked Data. In Proceedings of the 16th International Conference on Intelligent Environments, Madrid, Spain, 20–23 July 2020; pp. 132–139. [ Google Scholar ]
• Guarneri, M.; De Collibus, M.F.; Francucci, M.; Ciaffi, M. The Importance of Artworks 3D Digitalization at the Time of COVID Epidemy: Case Studies by the Use of a Multi-wavelengths Technique. In Proceedings of the 2020 IEEE 5th International Conference on Image, Vision and Computing, Beijing, China, 10–12 July 2020; pp. 113–117. [ Google Scholar ]
• Lah, H.A.; Ghani, R.A.; Mahmood, A.S.; Rahman, M.N.A.; Sulaiman, I.; Omar, A.; Mustafa, W.A. Preserving the Nation’s Legacy Through Emerging Technology. J. Phys. Conf. Ser. 2020 , 1529 , 042059. [ Google Scholar ] [ CrossRef ]
• Ovallos-Gazabon, D.; Meléndez-Pertuz, F.; Collazos-Morales, C.; Zamora-Musa, R.; Cardenas, C.A.; González, R.E.R. Innovation, Technology and User Experience in Museums: Insights from Scientific Literature. In Proceedings of the Computational Science and Its Applications–ICCSA 2020: 20th International Conference, Cagliari, Italy, 1–4 July 2020; pp. 819–832. [ Google Scholar ]
• Fakhour, M.; Azough, A.; Kaghat, F.-Z.; Meknassi, M. A Cultural Scavenger Hunt Serious Game Based on Audio Augmented Reality. Adv. Intell. Syst. Comput. 2020 , 1102 , 1–8. [ Google Scholar ]
• Markopoulos, E.; Markopoulos, P.; Liumila, M.; Almufti, Y.; Romano, C. Digital Cultural Strategies Within the Context of Digital Humanities Economics. In Proceedings of the Advances in Human Factors in Wearable Technologies and Game Design: Proceedings of the AHFE 2019 International Conference on Human Factors and Wearable Technologies, and the AHFE International Conference on Game Design and Virtual Environments, Washington, DC, USA, 24–28 July 2019; pp. 283–295. [ Google Scholar ] [ CrossRef ]
• Tsai, T.H.; Chiang, Y.W. Research Study on Applying SLAM-Based Augmented Reality Technology for Gamification History Guided Tour. In Proceedings of the IEEE International Conference on Architecture, Construction, Environment and Hydraulics (ICACEH), Xiamen, China, 20–22 December 2019; pp. 116–119. [ Google Scholar ] [ CrossRef ]
• Jia, X.; Tan, Y.; Xiong, Q.; Li, Y.; Hai, W. Designing edutainment games for intangible cultural heritage: The BNREG project. Digit. Present. Preserv. Cult. Sci. Herit. 2019 , 9 , 281–286. [ Google Scholar ] [ CrossRef ]
• Cauchi, M.; Scerri, D. Enriching tourist UX via a location-based AR treasure hunt game. In Proceedings of the IEEE 9th International Conference on Consumer Electronics, Berlin, Germany, 8–11 September 2019; pp. 199–204. [ Google Scholar ] [ CrossRef ]
• Ivanjko, T. Crowdsourcing image descriptions using gamification: A comparison between game-generated labels and professional descriptors. In Proceedings of the 2019 42nd International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, Croatia, 20–24 May 2019; pp. 537–541. [ Google Scholar ]
• Quattrini, R.; Pierdicca, R.; Lucidi, A.; Di Stefano, F.; Malinverni, E.S. The European Research Night: New Ways for Communicating Science with ICT and Videomapping. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2019 , 42 , 647–654. [ Google Scholar ]
• Bălutoiu, M.A.; Grădinaru, A.; Moldoveanu, A.; Moldoveanu, F.; Morar, A.; Nazare, A.K.; Rădoi, M. Libquest—Revitalize libraries and reading through gamification. In Proceedings of the International Scientific Conference eLearning and Software for Education, Bucharest, Romania, 11–12 April 2019; pp. 173–180. [ Google Scholar ] [ CrossRef ]
• Corallo, A.; Fortunato, L.; Renna, C.; Sarcinella, M.L.; Spennato, A.; De Blasi, C. Mobile app for promoting cultural heritage: Geostatistic and textual analysis. In Proceedings of the IMEKO International Conference on Metrology for Archaeology and Cultural Heritage, MetroArchaeo, Florence, Italy, 4–6 December 2019; pp. 194–201. [ Google Scholar ]
• Wang, N.; Li, J.; Shi, Q.; Shi, D.; Mi, H. Encourage Self-exploration Through an Interactive Chinese Scroll Painting Design. In Proceedings of the International Conference on Human-Computer Interaction—HCII 2019: HCI in Games, Orlando, FL, USA, 26–31 July 2019; pp. 442–452. [ Google Scholar ]
• Weng, Y.; Shen, T.; Chen, S.; Xiao, B. Gamification in Local Intangible Cultural Heritage Museums for Children: A Case Design ; Springer International Publishing: Cham, Switzerland, 2019; pp. 233–245. [ Google Scholar ]
• Anastasovitis, E.; Ververidis, D.; Nikolopoulos, S.; Kompatsiaris, I. Digiart: Building new 3D cultural heritage worlds. In Proceedings of the 3DTV-Conference, Copenhagen, Denmark, 7–9 June 2017. [ Google Scholar ]
• Cunha, C.R.; Mendonça, V.; Morais, E.P.; Carvalho, A. The role of gamification in material and immaterial cultural heritage. In Proceedings of the 31st International Business Information Management Association Conference (IBIMA). International Business Information Management Association (IBIMA), Milan, Italy, 25–26 April 2018; pp. 6121–6129. [ Google Scholar ]
• Basaraba, N. Co-constructing cultural heritage through a web-based interactive digital narrative. In Proceedings of the Interactive Storytelling: 11th International Conference on Interactive Digital Storytelling, ICIDS 2018, Dublin, Ireland, 5–8 December 2018; pp. 642–645. [ Google Scholar ]
• Bazzurri, F.; Picardello, M.A. Optimization Techniques for Photogrammetry Applied to Cultural Heritage and the Action of Transformation Groups. In Proceedings of the Augmented Reality, Virtual Reality, and Computer Graphics: 5th International Conference, AVR 2018, Otranto, Italy, 24–27 June 2018; pp. 332–348. [ Google Scholar ]
• Tan, K.L.; Lim, C.K. Digital Heritage Gamification: An Augmented-Virtual Walkthrough to Learn and Explore Historical Places. In Proceedings of the 2nd International Conference on Applied Science and Technology, Kedah, Malaysia, 3–5 April 2017. [ Google Scholar ]
• Noev, N.; Goynov, M.; Sapundjiev, V.; Valev, I. A Serious Educational Game of Bulgarian Military and Historical Heritage. Digit. Present. Preserv. Cult. Sci. Herit. 2017 , 7 , 249–254. [ Google Scholar ] [ CrossRef ]
• Slavova-Petkova, S. Project “Models and Applications of Serious Games in Education of Cultural Heritage and National Identity”. Digit. Present. Preserv. Cult. Sci. Herit. 2017 , 7 , 243–248. [ Google Scholar ] [ CrossRef ]
• Aydin, S. Decoding kashgar: Participatory digital heritage making via digital online interaction and gamification. In Proceedings of the Adjunct Publication of the 2017 ACM International Conference on Interactive Experiences for TV and Online Video, Hilversum, The Netherlands, 14 June 2017; pp. 93–97. [ Google Scholar ]
• Stefanov, K.; Bontchev, B.; Boytchev, P.; Georgiev, A.; Grigorov, A. Reusing Components from Cultural Heritage Games—The RAGE Project Approach. Digit. Present. Preserv. Cult. Sci. Herit. 2017 , 6 , 49–61. [ Google Scholar ] [ CrossRef ]
• Papathanasiou-Zuhrt, D.; Weiss-Ibanez, D.-F.; Di Russo, A. The Gamification of Heritage in the UNESCO Enlisted Medieval Town of Rhodes. In Proceedings of the CEUR Workshop, Nanjing, China, 4 December 2017; pp. 60–70. [ Google Scholar ]
• Ćosović, M.; Brkić, B. Game-Based Learning in Museums—Cultural Heritage Applications. Information 2020 , 11 , 22. [ Google Scholar ] [ CrossRef ]
• Lu, Y.S.; Yuan, F.; Lin, J.W.; Yuan, K.Y. TouristGo: A Location-Based Mobile Game to Improve Tourist Experience by Visiting Path Optimization. Pers. Ubiquitous Comput. 2020 , 24 , 405–418. [ Google Scholar ] [ CrossRef ]
• Luigini, A.; Parricchi, M.; Basso, A.; Basso, D. Immersive and Participatory Serious Games for Heritage Education, Applied to the Cultural Heritage of South Tyrol. Interact. Des. Archit. 2019 , 43 , 42–67. [ Google Scholar ] [ CrossRef ]
• Konstantinov, O.; Kovatcheva, E.; Palikova, N. Gamification in Cultural and Historical Heritage Education. In Proceedings of the 12th International Technology, Education and Development Conference—INTED, Valencia, Spain, 5–7 March 2018; pp. 8443–8451. [ Google Scholar ]
• Chrast, O.; Beseda, J. New Technologies and New ICT and Gamification-Based Approaches for the Cultural Heritage Education and Comparison of their use in some Czech Republic’s Cultural Heritage Sites, Cultural Monuments and Virtual Museums. In Proceedings of the Overcoming the Challenges and Barriers in Open Education, Prague, Czechia, 25–26 June 2018; pp. 401–411. [ Google Scholar ]
• Michala, M.; Alexakos, C.; Tsolis, D. Digitization and Gamification Technologies in the Design, Development and Evaluation of a Digital Secondary Education Program for Greek Art and Culture. In Proceedings of the EDULEARN18: 10th International Conference on Education and New Learning Technologies, Palma, Spain, 2–4 July 2018; pp. 7746–7755. [ Google Scholar ]
• Liestøl, G. The Photo Positioning Puzzle Creating Engaging Applications for Historical Photographs by Combining Mobile Augmented Reality and Gamification. In Proceedings of the 2018 3RD Digital Heritage International Congress (Digitalheritage) held Jointly with 2018 24th International Conference on Virtual Systems & Multimedia (VSMM 2018), San Francisco, CA, USA, 26–30 October 2018; pp. 245–252. [ Google Scholar ]
• Merchan, M.J.; Perez, E.; Merchan, P.; Salamanca, S.; Moreno, M.D. Making History a New Challenge. 3D Gamification on the Study of Arts Subjects. In Proceedings of the 11th International Conference of Education, Research and Innovation (ICERI2018), Seville, Spain, 12–14 November 2018; pp. 4081–4088. [ Google Scholar ]
• Micheletti, A. Motivating and Involving Users through Gamification: A Proposal. Digit. Libr. Multimed. Arch. 2017 , 701 , 20–30. [ Google Scholar ]
• Montusiewicz, J.; Barszcz, M.; Dziedzic, K.; Kesik, J.; Milosz, M.; Tokovarov, M. The Concept of a 3D Board Game to Recognize Architectural Monuments. In Proceedings of the INTED2017: 11th International Technology, Education and Development Conference, Valencia, Spain, 6–8 March 2017; pp. 8665–8674. [ Google Scholar ]
• Montusiewicz, J.; Milosz, M.; Kesik, J.; Barszcz, M.; Dziedzic, K.; Tokovarov, M.; Kopniak, P. Developing an Educational Board Game Using Information Technology. In Proceedings of the 9th International Conference on Education and New Learning Technologies (EDULEARN17), Barcelona, Spain, 3–5 July 2017; pp. 7396–7405. [ Google Scholar ]
• Moumoutzis, N.; Pappas, N.; Xanthaki, C.; Perrakis, S.; Maragkoudakis, Y.; Christodoulakis, S.; Paneva-Marinova, D. Gamification and Cultural Institutions in Cultural Heritage Promotion: A Successful Example from Italy. Cult. Trends 2021 , 31 , 3–22. [ Google Scholar ]
• Benito-Santos, A.; Dorn, A.; Gomez, A.G.L.; Palfinger, T.; Sanchez, R.T.; Wandl-Vogt, E. Outdoor Education, the Enhancement and Sustainability of Cultural Heritage: Medieval Madrid. Sustainability 2021 , 13 , 1106. [ Google Scholar ] [ CrossRef ]
• Sprugnoli, R.; Guerini, M.; Moretti, G.; Tonelli, S. Architectural Jewels of Lublin: A Modern Computerized Board Game in Cultural Heritage Education. ACM J. Comput. Cult. Herit. 2021 , 14 , 1–21. [ Google Scholar ] [ CrossRef ]
• Slavec, A.; Sajincic, N.; Starman, V. Use of Smartphone Cameras and Other Applications While Traveling to Sustain Outdoor Cultural Heritage. Sustainability 2021 , 13 , 7312. [ Google Scholar ] [ CrossRef ]
• Liritzis, I.; Volonakis, P. Cyber-Archaeometry: Novel Research and Learning Subject Overview. Educ. Sci. 2021 , 11 , 86. [ Google Scholar ] [ CrossRef ]
• Matthys, M.; De Cock, L.; Vermaut, J.; Van de Weghe, N.; De Maeyer, P. An Animated Spatial Time Machine in Co-Creation: Reconstructing History Using Gamification Integrated into 3D City Modelling, 4D Web and Transmedia Storytelling. ISPRS Int. J. Geo-Inf. 2021 , 10 , 460. [ Google Scholar ] [ CrossRef ]
• Ceccacci, S.; Generosi, A.; Leopardi, A.; Mengoni, M.; Mandorli, A.F. The Role of Haptic Feedback and Gamification in Virtual Museum Systems. ACM J. Comput. Cult. Herit. 2021 , 14 , 1–14. [ Google Scholar ] [ CrossRef ]
• Widarti, E.; Suyoto, S.; Emanuel, A.W.R. Mobile Application Design for Heritage Tourism Uses Gamification Approach in Indonesia. Int. J. Eng. Pedagog. 2020 , 10 , 89–102. [ Google Scholar ] [ CrossRef ]
• Lee, J.; Yi, J.H.; Kim, S. Cultural Heritage Design Element Labeling System with Gamification. IEEE Access 2020 , 8 , 127700–127708. [ Google Scholar ] [ CrossRef ]
• Nofal, E.; Panagiotidou, G.; Reffat, R.M.; Hameeuw, H.; Boschloos, V.; Vande Moere, A. Situated Tangible Gamification of Heritage for Supporting Collaborative Learning of Young Museum Visitors. ACM J. Comput. Cult. Herit. 2020 , 13 , 1–24. [ Google Scholar ] [ CrossRef ]
• Senatore, L.; Gallozzi, A.; Cigola, M.; Strollo, R.M. Citizen Science and Gamification for Cultural Heritage. EGA-Rev. Expres. Graf. Arquit. 2020 , 25 , 232–239. [ Google Scholar ] [ CrossRef ]
• Lopez-Martinez, A.; Carrera, A.; Iglesias, C.A. Empowering Museum Experiences Applying Gamification Techniques Based on Linked Data and Smart Objects. Appl. Sci. 2020 , 10 , 5419. [ Google Scholar ] [ CrossRef ]
• Uribe, S.A.N.; Rincon, A.A.A.; Castillo, C.A.R.; Valencia, M.; Lozano, H.D.; Guzman, C. Project for the Innovation in the Gastronomic Market of Bogota Through the Design of a Gamification Experience for Mobile Devices. In Proceedings of the International Conference of Applied Business and Management (ICABM2020), Porto, Portugal, 25–26 June 2020; pp. 246–275. [ Google Scholar ]
• Varinlioglu, G.; Halici, S.M. Gamification of Heritage through Augmented Reality. In Proceedings of the eCAADe SIGraDi 2019 Architecture in the Age of the 4th Industrial Revolution, Porto, Portugal, 11–13 September 2019; pp. 513–518. [ Google Scholar ]
• Lee, B.C. The Effect of Gamification on Psychological and Behavioral Outcomes: Implications for Cruise Tourism Destinations. Sustainability 2019 , 11 , 3002. [ Google Scholar ] [ CrossRef ]
• Di Paola, F.; Inzerillo, L.; Alogna, Y.A. Gaming Approach for Cultural Heritage Knowledge and Dissemination. In Proceedings of the International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Avila, Spain, 1–5 September 2019; pp. 421–428. [ Google Scholar ]
• Prandi, C.; Melis, A.; Prandini, M.; Delnevo, G.; Monti, L.; Mirri, S.; Salomoni, P. Gamifying Cultural Experiences Across the Urban Environment. Multimed. Tools Appl. 2019 , 78 , 3341–3364. [ Google Scholar ] [ CrossRef ]
• Grammatikopoulou, A.; Laraba, S.; Sahbenderoglu, O.; Dimitropoulos, K.; Douka, S.; Grammalidis, N. An adaptive framework for the creation of exergames for intangible cultural heritage (ICH) education. J. Comput. Educ. 2019 , 6 , 417–450. [ Google Scholar ] [ CrossRef ]
• Bujari, A.; Ciman, M.; Gaggi, O.; Palazzi, C.E. Using gamification to discover cultural heritage locations from geo-tagged photos. Pers. Ubiquitous Comput. 2017 , 21 , 235–252. [ Google Scholar ] [ CrossRef ]
• Luigini, A.; Basso, A. Heritage Education for Primary Age through an Immersive Serious Game ; Springer International Publishing: Cham, Switzerland, 2021; pp. 157–174. [ Google Scholar ]
• Bugeja, M.; Grech, E.M. Using Technology and Gamification as a Means of Enhancing Users’ Experience at Cultural Heritage Sites. Stud. Comput. Intell. 2020 , 859 , 69–89. [ Google Scholar ] [ CrossRef ]
• O’Connor, S.; Colreavy-Donnelly, S.; Dunwell, I. Fostering Engagement with Cultural Heritage Through Immersive VR and Gamification ; Springer Series Cultural Computing; Springer International Publishing: Cham, Switzerland, 2020; pp. 301–321. [ Google Scholar ]
• Geronikolakis, E.; Zikas, P.; Kateros, S.; Lydatakis, N.; Georgiou, S.; Kentros, M.; Papagiannakis, G. A True AR Authoring Tool for Interactive Virtual Museums ; Springer Series Cultural Computing; Springer International Publishing: Cham, Switzerland, 2020; pp. 225–242. [ Google Scholar ]
• Wilhelmsson, U.; Backlund, P. Everyone Is Not a Gamer! Developing Cultural Heritage Experiences for Diverse Audiences ; Springer Series Cultural Computing; Springer International Publishing: Cham, Switzerland, 2020; pp. 263–281. [ Google Scholar ]
• Liestøl, G.; Ritter, C.S.; Ibrus, I. Audiovisual industries and tourism: Forms of convergence. In Emergence of Cross-Innovation Systems: Audiovisual Industries Co-Innovating with Education, Health Care and Tourism ; Emerald Publishing: Bingley, UK, 2019; pp. 165–172. [ Google Scholar ]
• Synnes, K.; Artopoulos, G.; Smaniotto Costa, C.; Menezes, M.; Redaelli, G. CyberParks Songs and Stories—Enriching Public Spaces with Localized Cultural Heritage Material such as Digitized Songs and Stories ; Springer International Publishing: Cham, Switzerland, 2019; pp. 224–237. [ Google Scholar ]
• Rodney, H. Heritage: Critical Approaches ; Routledge: England, UK, 2013. [ Google Scholar ]
• Law no. 107/2001 of September 8—Base Law of Cultural Heritage [Lei n.º 107/2001 de 8 de Setembro-Lei de Bases do Património Cultural]. Diário da República . 2001. 1.ª Série-A, n. 209. Available online: https://files.dre.pt/1s/2001/09/209a00/58085829.pdf (accessed on 21 November 2022).
• Meske, C.; Brockmann, T.; Wilms, K.; Stieglitz, S. Social Collaboration and Gamification. In Gamification-Using Game Elements in Serious Contexts ; Stieglitz, S., Lattemann, C., Robra-Bissantz, S., Zarnekow, R., Brockmann, T., Eds.; Springer: Berlin/Heidelberg, Germany, 2016; pp. 93–109. [ Google Scholar ]
• Tondello, G.F.; Kappen, D.L.; Mekler, E.D.; Ganaba, M.; Nacke, L.E. Heuristic Evaluation for Gameful Design. In Proceedings of the ACM SIGCHI Annual Symposium on Computer-Human Interaction in Play (CHI PLAY 2016), Austin, TX, USA, 16–19 October 2016; pp. 315–323. [ Google Scholar ] [ CrossRef ]
• Ryan, R.M.; Deci, E.L. Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. Contemp. Educ. Psychol. 2000 , 25 , 54–67. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Chou, Y. Actionable Gamification: Beyond Points, Badges, and Leaderboards ; Octalysis Media; Packt Publishing Ltd.: Birmingham, UK, 2015. [ Google Scholar ]
• Xu, F.; Buhalis, D.; Weber, J. Serious games and the gamification of tourism. Tour. Manag. 2017 , 60 , 244–256. [ Google Scholar ] [ CrossRef ]
• Groot, J. Consuming History: Historians and Heritage in Contemporary Popular Culture ; Routledge: England, UK, 2008. [ Google Scholar ]
• Tulloch, H.; Randell-Moon, H.E.K. The Politics of Gamification: Education, Neoliberalism and the Knowledge Economy. Rev. Educ. Pedagog. Cult. Stud. 2018 , 40 , 204–226. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Marques, C.G.; Pedro, J.P.; Araújo, I. A Systematic Literature Review of Gamification in/for Cultural Heritage: Leveling up, Going Beyond. Heritage 2023 , 6 , 5935-5951. https://doi.org/10.3390/heritage6080312

Marques CG, Pedro JP, Araújo I. A Systematic Literature Review of Gamification in/for Cultural Heritage: Leveling up, Going Beyond. Heritage . 2023; 6(8):5935-5951. https://doi.org/10.3390/heritage6080312

Marques, Célio Gonçalo, João Paulo Pedro, and Inês Araújo. 2023. "A Systematic Literature Review of Gamification in/for Cultural Heritage: Leveling up, Going Beyond" Heritage 6, no. 8: 5935-5951. https://doi.org/10.3390/heritage6080312

Article Metrics

Article access statistics, supplementary material.

• Externally hosted supplementary file 1 Link: https://www.jthr.es/index.php/journal/article/view/417 Description: Marques, C. G., Pedro, J. P., Dionísio, M., Almeida, P., & Silva, C. P. A systematic literature review of Gamification in Cultural Heritage: Where are we? Where do we go?. Journal of Tourism and Heritage Research. 2022, 5(4), 64-83.

Further Information

Mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

• Open access
• Published: 31 January 2023

Gamification of e-learning in higher education: a systematic literature review

• Amina Khaldi   ORCID: orcid.org/0000-0003-4935-1840 1 ,
• Rokia Bouzidi 1 &
• Fahima Nader 1  

Smart Learning Environments volume  10 , Article number:  10 ( 2023 ) Cite this article

31k Accesses

68 Citations

11 Altmetric

Metrics details

In recent years, university teaching methods have evolved and almost all higher education institutions use e-learning platforms to deliver courses and learning activities. However, these digital learning environments present significant dropout and low completion rates. This is primarily due to the lack of student motivation and engagement. Gamification which can be defined as the application of game design elements in non-game activities has been used to address the issue of learner distraction and stimulate students’ involvement in the course. However, choosing the right combination of game elements remains a challenge for gamification designers and practitioners due to the lack of proven design approaches, and there is no one-size-fits-all approach that works regardless of the gamification context. Therefore, our study focused on providing a comprehensive overview of the current state of gamification in online learning in higher education that can serve as a resource for gamification practitioners when designing gamified systems. In this paper, we aimed to systematically explore the different game elements and gamification theory that have been used in empirical studies; establish different ways in which these game elements have been combined and provide a review of the state-of-the-art of approaches proposed in the literature for gamifying e-learning systems in higher education. A systematic search of databases was conducted to select articles related to gamification in digital higher education for this review, namely, Scopus and Google Scholar databases. We included studies that consider the definition of gamification as the application of game design elements in non-game activities, designed for online higher education. We excluded papers that use the term of gamification to refer to game-based learning, serious games, games, video games, and those that consider face-to-face learning environments. We found that PBL elements (points, badges, and leaderboards), levels, and feedback and are the most commonly used elements for gamifying e-learning systems in higher education. We also observed the increasing use of deeper elements like challenges and storytelling. Furthermore, we noticed that of 39 primary studies, only nine studies were underpinned by motivational theories, and only two other studies used theoretical gamification frameworks proposed in the literature to build their e-learning systems. Finally, our classification of gamification approaches reveals the trend towards customization and personalization in gamification and highlights the lack of studies on content gamification compared to structural gamification.

Introduction

In recent years, most universities use e-learning platforms to deliver courses. Teaching in the form of e-learning is a modern supplement, and sometimes even an alternative to traditional education (Górska, 2016 ). Especially since the last few years, with the spread of the Covid-19 crisis, higher education institutions had to shift from traditional teaching to online teaching as an alternative to resume learners' learning (Sofiadin & Azuddin, 2021 ). However, over time, these digital environments brought several challenges. On one hand, student motivation decreases, resulting in a lack of engagement and participation in courses. On the other hand, instructors struggle to maintain learners’ attention, leading to the eventual abandonment of online education systems. To solve this problem and create engaging e-learning platforms, the gamification technique was proposed.

Game technologies create opportunities for higher education institutions to redesign and innovate their e-learning models to support learning experiences among learners (Alhammad & Moreno, 2018 ). The introduction and growing expansion of gamification in education and learning contexts promotes critical reflection on the development of projects that transform students’ learning experiences (Garone & Nesteriuk, 2019 ). However, is it that simple to create effective gamified e-learning systems especially in the context of higher education?

Early applied work on gamification of educational settings suggested positive-learning, but mixed results have been obtained (Seaborn & Fels, 2015 ). While gamification in general learning systems is known to have a positive impact on student motivation, evidence on its effectiveness in higher education settings is mixed and still uncertain due to the complicated environment in the higher education context. First, the level of difficulty of study is higher at the university than at lower levels of education, and students are more aware of the importance of education they have chosen (Urh et al., 2015 ). Moreover, tertiary education is characterized by the variety of students’ profiles, needs and learning methods; thereby, each game element and even each combination of game elements affects each student differently. Given this diversity of features in the higher education context and the increasing number of inter- and multidisciplinary programs, the process of applying gamification is becoming more complex.

The purpose of this systematic review was to provide a comprehensive overview of the current state of gamification in e-learning in higher education. We focused on identifying how designers currently deal with gamification in the digital higher education context, what game elements they use, how these elements are combined, and what gamification theories are used. In addition, this study sought to find data on existing gamification approaches in the literature, especially those suggested to be applied in digital higher education. Our study differs from previous studies in several ways. In our study, we first wanted to compare our results with previous research’s results that addressed the same research questions in terms of trends in the use of game elements, i.e. whether designers who develop gamified e-learning systems still use classic game elements such as points, badges, and leaderboards, or whether they expand the list of game elements used to include deeper game elements like challenges, storytelling, and so on. We then focused on the underpinning gamification theories used in empirical work, and specifically we sought to understand whether empirical research is beginning to use the various gamification frameworks available in the literature, or whether it is still relying on theories and methods that are highly theoretical and do not provide clear guidance to designers when choosing the right set of game elements (Toda et al., 2020 ). Also, in our study, we sought to find out how game elements are combined in gamified learning systems in higher education. Previous studies have not fully explored this point, with the exception of the study (Dichev & Dicheva, 2017 ). Finally, we proposed a classification of gamification approaches proposed in the context of e-learning in higher education based on several relevant criteria.

The remainder of this manuscript has the following structure. " Related works " section, briefly reviews some of the most relevant review papers. " Systematic literature review methodology " section, systematic literature review methodology, presents the approach we followed in conducting our paper retrieval. " Results of the search " section, results of the research, presents responses to our defined research questions. " Discussion and limitations " section is dedicated for discussion of the results; and finally, we conclude.

Related works

Prior reviews.

This section briefly reviews some of the relevant literature reviews on gamification in higher education related to the topic of our systematic review. The objective is to be able to compare our findings later in the results section to prior reviews’ findings and to shed a more realistic light on any advances in gamification in e-learning in the context of higher education.

Dichev and Dicheva ( 2017 ) critically reviewed the advancement of educational gamification. This review paper was the only one to address the issue of combining game elements in gamified learning systems. The authors found that in all reviewed works, no justification is given for the selection of particular game elements. The study concluded that there is a need for further studies to improve our understanding of how individual game elements are associated with behavioral and motivational outcomes and how they function in an educational context.

Ozdamli ( 2018 ) examined 313 studies on gamification in education. It used content analysis to determine trends in gamification research. The study sought to determine the distribution of empirical research based on a variety of criteria, namely: distribution of studies based on years, number of authors, type of publication, paradigms, research sample, environments, theory/model/strategy, learning area and distribution of game components, mechanics and dynamics. The author found that motivational theories are the most frequently used approach in gamification studies and that the most frequently used game components are goals, rewards and progression sticks.

Khalil et al. ( 2018 ) reviewed the state of the art on gamification in MOOCs (Massive Open Online Course) by answering eight research questions. One of these questions sought to identify elements of gamification that have been implemented or proposed for implementation in MOOCs. The study found that the most commonly used elements in the application of gamification in MOOCs are badges, leaderboards, progress, and challenges. According to the study, progress and challenges are used more frequently in MOOCs than points.

The paper (Alhammad & Moreno, 2018 ) studied gamification in the context of software engineering (SE) education. The study sought to understand how gamification was applied in the SE curriculum and what game elements were used. The study identified four gamification approaches from the primary studies analyzed: papers that implemented gamification by following an existing gamification approach in the literature, papers that adapted psychological and educational theories as gamification approaches, papers that designed and followed their own gamification approach, and finally, papers that did not follow any specific gamification approach. In addition, leaderboards, points and levels were found to be the most frequently used gaming components. Similarly, challenges, feedback, and rewards were the most commonly used mechanics, and progression was the most commonly used dynamic.

Majuri et al. ( 2018 ) reviewed 128 empirical research papers in the literature on gamification in education and learning. It was found that points, challenges, badges and leaderboards are the most commonly used gamification affordances in education which are affordances that refer to achievement and progression while social and immersion-oriented affordances are much less common.

In the paper (Zainuddin et al., 2020 ), the authors addressed a research question related to our research area, namely the underlying theoretical models used in gamification research. It was found that in the studies that implicitly mention their theoretical underpinnings, self-determination theory is the most commonly used, followed by flow theory and goal-setting, while the other studies do not provide any theoretical content.

More recently, van Gaalen et al. ( 2021 ) reviewed 44 research studies in the health professions education literature. The study addressed the question of what game attributes are used in gamified environments, and sought to understand the use of theory throughout the gamification process. The study used Landers ( 2014 )’s framework to categorize the identified game elements into game attributes and revealed that in most reviewed studies the game attributes ‘assessment’ and/or ‘conflict/challenge’ were embedded in the learning environment. Regarding the use of theory in gamification processes, most of the identified studies on gamification in health professions education were not theory-based, or theoretical considerations were not included or not yet developed.

Finally, the authors of the paper (Kalogiannakis et al., 2021 ) performed a systematic literature review on gamification in science education by reviewing 24 empirical research papers. A research question related to our field of study was addressed in this review, namely, what learning theory is used, and what game elements are incorporated into gaming apps. The findings of the studyshowed that most articles did not provide details about the theoretical content or the theory on which they were based. The few articles that used theoretical frameworks were based on self-determination theory SDT, flow theory, goal-setting theory, cognitive theory of multimedia learning and motivation theory. In addition, the study found that the most common game elements and mechanics used in gamified science education environments were competitive setup, leaderboards, points and levels.

Systematic literature review methodology

In this paper of systematic review, we followed a methodology to identify how gamification technique has been used in digital learning environments, specifically in higher education. We sought to identify the game elements that have been used the most, the way they have been combined, and the different frameworks proposed in the literature for gamification of e-learning systems in higher education. A systematic literature review is a means of identifying, evaluating and interpreting all available research relevant to a particular research question, or topic area, or phenomenon of interest (Kitchenham, 2004 ). Kitchenham ( 2004 ) summarizes the stages of a systematic review in three main phases: Planning the Review, Conducting the Review, and Reporting the Review. The first phase ‘Planning the Review’ includes the formulation of research questions, identification of key concepts and constructing the search queries. The second phase ‘Conducting the Review’ consists on study selection based on inclusion and exclusion criteria. Finally, the third phase ‘Reporting the Review’ relates to data extraction and responding to research questions. In the following, we detail the main steps of each phase.

Search strategy

We started by identifying the main goal of this systematic literature review by clearly formulating the following research questions:

Which game elements and gamification theories are used in gamified learning systems?

How these game elements are combined?

Which gamification design approaches are available in the literature?

Then, we constructed a list of key concepts that are: gamification, e-learning and higher education. After that, we identified the alternative terms for each of the key concepts as some authors may refer to the same concept using a different term. For the concept of gamification, we identified this list of free text terms: gamify, game elements, game dynamics, game mechanics, game components, game aesthetics and gameful. For the two other concepts of e-learning and higher education, we identified these terms: education, educational, learning, teaching, course, syllabus, syllabi, curriculum, and curricula.

We formulated two search queries based on the terms identified previously:

For research questions 1and 2:

(gamif* OR gameful OR “game elements” OR “game mechanics” OR “game dynamics” OR “game components” OR “game aesthetics”) AND (education OR educational OR learning OR teaching OR course OR syllabus OR syllabi OR curriculum OR curricula).

For research question 3:

(gamif* OR gameful OR “game elements” OR “game mechanics” OR “game dynamics” OR “game components” OR “game aesthetics”) AND (education OR educational OR learning OR teaching OR course OR syllabus OR syllabi OR curriculum OR curricula) AND (framework OR method OR design OR model OR approach OR theory OR strategy).

We conducted our research by searching the databases using the search query formulated previously. We performed our search in the Scopus and Google Scholar databases as the first is one of the most professional indexing databases and the second is the most popular, so it helps to identify further eligible studies. The search was performed in December 2021. Although the Scopus database indexed the publication abstracts, most of the articles were not available through Scopus, and the articles were retrieved from the following publishers:

SEMANTIC SCHOLAR,

(Hallifax et al. ) SAGE,

Science Direct.

The exception was some articles that could not be accessed. We also performed a backward snowballing search to identify further relevant studies by scanning and searching the references of papers marked as potentially relevant (Dichev & Dicheva, 2017 ; Mora et al., 2017 ; Gari & Radermacher, 2018 ; Khalil et al., 2018 ; Ozdamli, 2018 ; Subhash & Cudney, 2018 ; da Silva et al., 2019 ; Hallifax et al., 2019a , 2019b ; Legaki & Hamari, 2020 ; Zainuddin et al., 2020 ; Saleem et al., 2021 ; Swacha, 2021 ; van Gaalen et al., 2021 ) in search of other relevant studies.

Inclusion and exclusion criteria

In the following table, we summarized the inclusion and exclusion criteria that we considered when we screened full text articles (Table 1 ).

Study selection

To select the relevant studies for this systematic review, a manual screening was performed. First, we reviewed the titles and abstracts of different records that were retrieved. Then, citations were imported to Endnote and duplicate records were removed. After that, we read the full text of all retained articles for inclusion and exclusion based on the eligibility criteria. In case of uncertainty, discussion was organized with the research team to reach consensus about the articles in question.

Data extraction

We developed a data extraction form that was refined and discussed until consensus was obtained. The extraction form was then used by the review author to extract data from all included studies. In this part of this paper, we have considered two types of papers: papers representing case studies to extract the game elements used in the developed e-learning systems, the underpinning theories behind the gamification process and the way game elements were combined with each other. The second type of retrieved papers is about framework proposals, from which we could identify models, approaches, and design processes proposed in the literature for gamifying digital learning environments in tertiary education level.

Results of the search

General results.

In this literature review, we reported the most extensive overview of the empirical research literature on gamification of e-learning in higher education to date. The selection process of relevant studies is shown in Fig.  1 . We analyzed a total of 90 papers to respond to the three research questions formulated previously. First, we retrieved 39 papers in the form of empirical studies carried out at university level and analyzed them to identify what game elements are used, what gamification theories are used to guide the gamification process, and how these game elements are combined. We then identified a variety of 51 papers of type theoretical proposals intended to guide the gamification process. Since higher education is part of general learning systems, we included in this review papers that propose gamification approaches for general contexts and general learning systems. Indeed, we identified 16 papers for general application of gamification, 18 papers for gamifying general learning systems and 17 approaches intended to be applied to e-learning systems in higher education.

Flow diagram of the articles selection process

Answering research questions

In following, we will answer the three research questions formulated at the beginning of this article:

Education applications of gamification refer to using game elements for scholastic development in formal and informal settings (Seaborn & Fels, 2015 ). In our case, we were interested in collecting relevant experimental studies on gamification of e-learning systems in higher education. In the following table (Table 2 ), we list and examine 39 experimental studies that have implemented a digital learning system at the higher education level to answer RQ1 . For each study, we analyzed the game elements that were incorporated and the gamification approaches that were followed during the gamification process. For ease of reference, the game elements that were used in e-learning systems to improve student engagement and the underpinning theories are summarized in Table 2 . More detailed descriptions of the 39 empirical studies are presented in “Appendix”.

By analyzing the game elements listed in Table 2 , we noticed that PBL elements (points, badges, and leaderboards), levels, and feedback are the most commonly used elements for gamifying e-learning systems in higher education. This is in line with other reviews’ findings, e.g. (Dichev & Dicheva, 2017 ).

Furthermore, in response to what (Dichev & Dicheva, 2017 ) stated about the fact that gamification with “deeper game elements” (Enders, 2013 ) by incorporating game design principles involving game mechanics and dynamics such as challenges, choice, low-risk failure, role-play or narrative is still scarce, we noted in our systematic literature review that recent studies explore new game elements. Indeed, among the 39 studies analyzed in Table 2 , there are 20 primary studies that used “deeper game elements” (Enders, 2013 ) like challenges and storytelling (narrative). Among these, challenges are the most popular ones.

In Seaborn and Fels ( 2015 ), the authors noted that till 2015, the majority of applied research on gamification was not grounded in theory and did not use gamification frameworks in the design of the system under study. Likewise, in this systematic review, by analyzing the 39 empirical studies listed in Table 2 , we noticed that most studies were not underpinned by gamification theories. This is in line with the findings of other recent studies, such as van Gaalen et al. ( 2021 ) and Kalogiannakis et al. ( 2021 ). Indeed, of the 39 primary studies analyzed in our systematic review, only nine papers (Smith, 2017 ; Kyewski & Krämer, 2018 ; Pilkington, 2018 ; Tsay et al., 2018 ; van Roy & Zaman, 2019 ; De-Marcos et al., 2020 ; Facey-Shaw et al., 2020 ; Sanchez et al., 2020 ; Dikcius et al., 2021 ) adapted theoretical approaches and used them as gamification approaches. These are a set of social and motivational theories resumed in a variety of six different theories, namely: self-determination theory-SDT, Social comparison theory, social exchange theory-SET, cognitive evaluation theory-CET, situated motivational affordance theory, theory of gamified learning (Landers, 2014 ) and user-centered design (Nicholson, 2012 ). Self-determination theory is considerably the most popular one. These findings are correlated with other reviews’ findings such as Zainuddin et al. ( 2020 ) and Kalogiannakis et al. ( 2021 ). Only two other primary studies Marín et al. ( 2019 ) and Dias ( 2017 ) used existing theoretical gamification frameworks to build their gamified e-learning systems. For the remaining papers, some built their owngamification design based on guidelines from the literature whereas others did not cite any theory. Hence, we notice that this distribution is in line with (Alhammad & Moreno, 2018 )’s review findings regarding the use of four different categories of gamification approaches in primary studies, namely, papers that followed existing gamification frameworks, papers that adapted motivational theories to their needs, papers that built their own approach, and finally, those that didn’t follow any specific approach. We also noticed that motivational theories are the most frequently used approach, as noted in Ozdamli ( 2018 ).

For this research question, we sought to identify how game elements are combined in gamified learning systems in higher education. Previous studies have not fully explored this point except the paper (Dichev & Dicheva, 2017 ). By analyzing the different empirical studies involved in this systematic literature review (listed in Table 2 ), we noticed the lack of detailed information about how instructors and designers combined different game elements. Indeed, in all reviewed papers, the authors listed only the game elements employed to gamify their learning systems. In addition, no study provided any justification of the choice made about the sets of game elements to use, nor the way they combined them in the gamified learning systems.

In the reviewed collection, five studies employed one single game element (Coleman, 2018 ; Garnett & Button, 2018 ; Kyewski & Krämer, 2018 ; Facey-Shaw et al., 2020 ; Dikcius et al., 2021 ), three other studies gamified systems using two game elements (Fajiculay et al., 2017 ; Smith, 2017 ; Donnermann et al., 2021 ), five other studies used three game elements (Hisham & Sulaiman, 2017 ; Kasinathan et al., 2018 ; Romero-Rodriguez et al., 2019 ; Khaleel et al., 2020 ; Sanchez et al., 2020 ) while the remaining ones used more than three elements.

This happens due to the lack of studies that provide clear guidelines and justifications for the combination of game elements (Toda et al., 2020 ).

In this section, we will approach RQ3 . We first synthesize the current literature on gamification approaches in a general context. Then, we present a set of gamification approaches for general learning systems. Finally, we list a set of approaches proposed specifically for higher education within e-learning environments. We briefly described each approach in the table below (Table 3 ).

In the table above, we investigated a total of 51 gamification approaches in three different contexts. The first set of approaches (the first 16 rows of Table 3 ) was designed for general use, i.e., for all contexts such as learning, health, marketing and entrepreneurship. While the second set of approaches (the next 18 rows of Table 3 ) targeted general learning contexts, i.e., without any restriction on educational level. Finally, the third set of approaches (the last 17 rows of Table 3 ) was intended to be applied in a specific context, namely digital higher education.

Given our review’s main interest in e-learning in higher education, we will classify the last 17 approaches of Table 3 , which correspond to those designed for e-learning systems in higher education, into several classes based on different relevant criteria that we will detail below. The paper (Saggah et al., 2020 ) proposes categorizing gamification design frameworks into three categories: scenario-based, high-level approach, and Gamification elements guidance. Inspired by this categorization, we propose our categorization, which will be used to classify the different gamification approaches in e-learning in higher education. A description of each category is given in what follows, and our classification results are shown in Table 4 .

Level of detail

High-level approach This group categorizes papers that provide an overview of the design process that serves as a general high-level guideline containing the global phases without detailing which game elements to use and how to implement them.

Gamification elements guidance This group categorizes papers that provide a conceptualization of the gamification elements that can be used in educational environments. These studies can include implementation guidance.

Scenario based This group categorizes papers that provide a descriptive outline of the design process. In other words, these papers propose gamification approaches by describing their application through real empirical studies experimented in real learning environments.

Type from student perspective (adaptive gamification/one size fits all gamification) Adaptive gamification considers that users have different motivations, so it consists of personalizing learning experiences according to each learner profile. Whereas ‘one size fits all’ gamification uses the same gamified system (gamification elements, rules, etc.) for all learners. For ease of use, we will use ‘A’ character for adaptive approaches and x for ‘one size fits all’ ones.

Profundity from pedagogical perspective (structural gamification versus content gamification) structural gamification refers to the application of game design elements to motivate the learner through an instructional content without changing it (Garone & Nesteriuk, 2019 ). It can be made by using clear goals, rewards for achievements, progression system and status, challenge and feedback (Garone & Nesteriuk, 2019 ). Content gamification is the application of elements, mechanics and game thinking to make the content more game-like (Garone & Nesteriuk, 2019 ). It is a one-time structure created only for a specific content or learning objectives and hence cannot be reused for any content (Sanal, 2019 ). Garone and Nesteriuk ( 2019 ) states that elements that can be used in content gamification are story and narrative; challenge, curiosity and exploration; characters and avatars; interactivity, feedback and freedom to fail (Kapp, 2014 ). According to Kapp ( 2014 ), the combination of both structural and content gamification, is the most effective way to build high engaging and motivating environments. For ease of use, we will use ‘C’ character for content approaches and x for structural ones.

Validation This group categorizes papers that provided a validation of the proposed approach through empirical evidence showing its application to e-learning systems in higher education.

Table 4 represents the results of our classification of gamification approaches in the context of e-learning in higher education. Regarding the level of detail, we noticed that most of the analyzed approaches (with a number of 9 out of a total of 17) are of the type of gamification elements guidance (Urh et al., 2015 ; Huang & Hew, 2018 ; Alsubhi & Sahari, 2020 ; Kamunya et al., 2020 ; Winanti et al., 2020 ; Alsubhi et al., 2021 ; Júnior & Farias, 2021 ; Sofiadin & Azuddin, 2021 ; Yamani, 2021 ). This number is followed by a number of 5 approaches of type scenario based (Mi et al., 2018 ; Legaki et al., 2020 ; Al Ghawail et al., 2021 ; Bencsik et al., 2021 ; Fajri et al., 2021 ), and finally, only 2 approaches are categorized as high-level approaches (Carreño, 2018 ; de la Peña et al., 2021 ). It is worth saying that scenario-based approaches are, in most cases, the most difficult to reproduce in other educational environments, as they are very specific, and each environment has its own characteristics. In contrast, high-level approaches are more general and need to be tailored according to the context. Finally, gamification elements guidance approaches can strongly help implement gamified learning environments as they provide a handy catalog of elements that can be injected easily into learning environments.

Furthermore, Table 4 shows that most of the suggested design approaches in the literature are not empirically explored (for example, by using a control and comparing gamified and non-gamified systems). Indeed, of the 17 gamification approaches in the context of e-learning in higher education analyzed, only four approaches have been applied and evaluated by empirical evidence (Huang & Hew, 2018 ; Alsubhi et al., 2021 ; de la Peña et al., 2021 ; Júnior & Farias, 2021 ). Among those four studies, one work was validated with experts (Alsubhi et al., 2021 ).

Moreover, Table 4 shows that of the 17 gamification approaches proposed for application to online learning systems in the context of higher education, two approaches (Carreño, 2018 ; Kamunya et al., 2020 ) fall into the category of adaptive gamification. This shows the trendy nature of personalization in higher education. Finally, Table 4 shows that the 17 approaches that have been proposed to gamify online learning systems in higher education focus solely on structured gamification, neglecting the content side of online learning systems.

Discussion and limitations

Through this systematic review, we identified several papers on the gamification of e-learning in the higher education context. In recent years, the research on gamification in e-learning has been getting traction, and the number of research articles and systematic reviews of research articles is increasing. As a summary of the existing approaches of gamification in e-learning in higher education, we notice the following points:

Gamification of e-learning in higher education: a trending area of research

The systematic review showed that gamification of learning systems is nowadays a hot topic, and research in this field is growing rapidly as well as for e-learning in higher education context, as it is shown by Fig. 2 .

Number of publications per year

Gamification design gaps and tendencies

In general, gamification theory helps in training and shaping participant behavior, however, in our systematic literature review, we observed from RQ1 that the majority of applied research on gamification is not grounded in theory and did not use gamification frameworks in the design of the learning system under study. This highlights the fact that there is a real gap between theoretical and applied work on gamification. One reason may be that existing approaches are very theoretical and cannot strongly assist designers and practitioners when gamifying learning systems, as pointed out by Toda et al. ( 2020 ). This also explains our results to the second research question RQ2 regarding the lack of detail on the combination of game elements used in the experimental studies and the motivation behind choosing specific game elements over others.

To better understand this phenomenon and to find a rationale for this lack of using theory and, thus, the lack of logic behind the use of certain game elements over others and their random linking and combination in gamified learning systems, we addressed the research question RQ3. In the latter, we analyzed the gamification approaches available in the literature and classified them into different categories based on a variety of criteria. Our results revealed that the gamification elements guidance approaches that provide taxonomies of game elements that can be incorporated into learning systems constitute the majority of the approaches that have been proposed for application in online learning in higher education. Those did not provide the psychological and behavioral changes that correspond to each game element. Instead, the older gamification theory was based simply on the behavioral outcomes that come from using gamification and the motivational needs behind it and did not provide details on how to implement them or details on what elements to use.

Using appropriate game elements can lead to higher levels of user motivation, whereas inappropriate game elements can demotivate users (Hallifax et al., 2019a , 2019b ). Thus, it is essential to choose the right combination of game elements that perfectly matches the desired behavior change. To do this, we must first explore the effect of each game element separately (Dichev & Dicheva, 2017 ). Thus, further studies are needed to improve our understanding of how individual game elements relate to behavioral and motivational outcomes so that we can identify their contribution in studies that mix multiple game elements (Dichev & Dicheva, 2017 ). An example of such study was provided in the health domain in the paper (Hervas et al., 2017 ). The latter proposed a taxonomy of gamification elements used in the domain of health by relating them to psychological fundamentals on behavioral changes, like Self-efficacy, Social influence, and Behavioral momentum. This work can facilitate researchers' empirical validation of gamification theory by building contexts and scenarios from ready-made taxonomies of gamification elements that target a specific behavioral outcome.

On the other hand, through our systematic literature review, we can see from RQ3 the recent emergence of data-driven approaches through machine learning techniques (Knutas et al., 2019 ; Duggal et al., 2021 ). These techniques help to create gamification designs suitable for the gamified context, especially when it comes to customizing the game elements to be incorporated into the final gamified system to the students' profiles.

In many learning environments, pedagogy assumes that all learners have homogeneous characteristics (Kamunya et al., 2020 ). However, Schöbel and Söllner ( 2016 ) argue that most gamification projects are not working because they are designed for a group of system users without considering the personal needs of each user. Hence the advantage of personalized training to the learner where all learners differ in preference, style and abilities with regard to the learning processes with or without technology mediation (Naik & Kamat, 2015 ). In this context, we noted the existence of two gamification approaches designed for online learning in higher education (Carreño, 2018 ; Kamunya et al., 2020 ). This is put into practice by tailoring the gamification elements to users' individual preferences. A recent related problem is the lack of adaptation of gamification to the content being gamified.

Another recent and relevant issue is the extreme lack of content gamification. Indeed, the motivational impact of certain game elements varies with the user activity or the domain of gamified systems (Hallifax et al., 2019a , 2019b ). Therefore, there is a great need for further exploration and experimentation in this immature area to provide a gamified design to satisfy users’ preferences as well as the task at hand. In other words, personalization in gamification should extend to content, as it does with user profiles, for example, by applying machine learning techniques to tailor the choice of game elements to gamified content.

Another common study design issue illuminated by our review is the lack of validation of the proposed gamification approaches through statistical analyses. In addition, most applied research on the gamification of online learning systems in higher education has not explored the gamification frameworks suggested in the literature.

Conclusion and future work

In this work, we conducted a review of the literature on gamification elements used in digital higher education, the way they are combined, and the different gamification approaches proposed in the literature to gamify learning systems. We analyzed a total of 90 papers to answer the three research questions formulated for this study.

This review identified points, badges, leaderboards, levels, feedback, and challenges as the most commonly used game elements in digital higher education. However, in terms of using gamification theory, our review found that the majority of applied gamification research is not theory-based and has not used gamification frameworks in the design of gamified learning systems. Although some experimental studies attempt to adapt psychological and educational theories available in the literature as gamification approaches, the resulting systems are not very clear, and there is no rationale for choosing certain game elements over others. Consequently, it can be concluded that these gamification approaches cannot strongly assist designers and practitioners in gamifying their learning systems. In addition, theoretical gamification approaches in e-learning in higher education should focus on understanding the effect of each single game design element and the behavioral changes that outcome from its use.

Moreover, based on the results of this review, we can observe the trend towards data-driven approaches through the use of machine learning techniques, especially in adaptive gamification approaches. This involves the adaptation of gamification elements to user profiles. On the other hand, although we have noticed the increasing use of gamification elements that are suitable for content gamification and make the content more game-like, such as storytelling and challenges, there is still a lack of gamification approaches that address content gamification. In fact, this is still an immature research area in gamification design in e-learning in higher education.  Future works should pay more attention to the pedagogical side of learning systems and the task under gamification. Apart from that, further research is required to compare theory-driven to data-driven gamification approaches, in terms of which one is the better or perhaps evaluate the effectiveness of a combination of the two, and go so far as to propose a hybrid gamification approach, which does not exist yet and might solve several gamification design issues.

Regarding future work, efforts should focus on building a holistic approach by considering all the aspects that constitute the environment. Among those,  personalization according to students’ profiles, gamified subject, educational context, learner’s culture, learner’s preferences, level, playing motivations and experience with games.

Finally, we have seen that most of the design approaches suggested in the literature are not empirically explored. Therefore, statistical analyses and comparative studies should be conducted to draw more robust and generalizable conclusions to validate the existing gamification approaches in the literature.

Availability of data and materials

All data generated or analyzed during this study are included in this published article.

Adams, S. P., & Du Preez, R. (2021). Supporting student engagement through the gamification of learning activities: A design-based research approach. Technology, Knowledge and Learning, 27 , 119–138.

Article   Google Scholar  

Ahmed, H. D., & Asiksoy, G. (2021). The effects of gamified flipped learning method on student’s innovation skills, self-efficacy towards virtual physics lab course and perceptions. Sustainability (switzerland), 13 (18), 10163.

Al Ghawail, E. A., Yahia, S. B., et al. (2021). Gamification model for developing E-learning in Libyan Higher Education. Smart education and e-learning 2021 (pp. 97–110). Springer.

Google Scholar  

Alcivar, I., & Abad, A. (2016). Design and evaluation of a gamified system for ERP training. Computers in Human Behavior, 58 , 109–118.

Alhammad, M. M., & Moreno, A. M. (2018). Gamification in software engineering education: A systematic mapping. Journal of Systems and Software, 141 , 131–150.

Allen, M. W. M. W. (2007). Designing successful e-learning: Forget what you know about instructional design and do something interesting . Wiley.

Alsubhi, M., Ashaari, N., et al. (2021). Design and evaluation of an engagement framework for e-learning gamification. International Journal of Advanced Computer Science and Applications , 12 .

Alsubhi, M. A., & Sahari, N. (2020). A conceptual engagement framework for gamified E-learning platform activities. International Journal of Emerging Technologies in Learning, 15 (22), 4–23.

Andrade, F. R. H., Mizoguchi, R., et al. (2016). The bright and dark sides of gamification . Springer.

Book   Google Scholar  

Aparicio, A. F., Vela, F. L. G., et al. (2012). Analysis and application of gamification. In Proceedings of the 13th international conference on Interacción Persona-Ordenador . Elche, Spain: Association for Computing Machinery. Article 17.

Aşıksoy, G. (2018). The effects of the gamified flipped classroom environment (GFCE) on students’ motivation, learning achievements and perception in a physics course. Quality and Quantity, 52 , 129–145.

Asiksoy, G., & Canbolat, S. (2021). The effects of the gamified flipped classroom method on petroleum engineering students’ pre-class online behavioural engagement and achievement. International Journal of Engineering Pedagogy, 11 (5), 19–36.

Bencsik, A., Mezeiova, A., et al. (2021). Gamification in higher education (case study on a management subject). International Journal of Learning, Teaching and Educational Research, 20 (5), 211–231.

Bennani, S., Maalel, A., et al. (2021). Towards an adaptive gamification model based on ontologies. In 2021 IEEE/ACS 18th international conference on computer systems and applications (AICCSA) .

Bernik, A. (2021). Gamification framework for E-learning systems in higher education. Tehnički Glasnik, 15 (2), 184–190.

Bernik, A., Radošević, D., et al. (2017). Research on efficiency of applying gamified design into University’s e-courses: 3D modeling and programming. Journal of Computer Science, 13 (12), 718–727.

Bernik, A., Radošević, D., et al. (2019). Achievements and usage of learning materials in computer science hybrid courses. Journal of Computer Science, 15 (4), 489–498.

Böckle, M., Micheel, I., et al. (2018). A design framework for adaptive gamification applications.

Buckley, P., & Doyle, E. (2017). Individualising gamification: An investigation of the impact of learning styles and personality traits on the efficacy of gamification using a prediction market. Computers and Education, 106 , 43–55.

Carreño, A. M. (2018). A framework for agile design of personalized gamification services.

Castro, T. C., & Gonçalves, L. S. (2018). The use of gamification to teach in the nursing field. Revista Brasileira De Enfermagem, 71 (3), 1038–1045.

Cechetti, N. P., Bellei, E. A., et al. (2019). Developing and implementing a gamification method to improve user engagement: A case study with an m-Health application for hypertension monitoring. Telematics Informatics, 41 , 126–138.

Chou, Y. K. (2015). Actionable gamification: Beyond points, badges, and leaderboards . Createspace Independent Publishing Platform.

Coleman, J. D. (2018). Engaging undergraduate students in a co-curricular digital badging platform. Education and Information Technologies, 23 (1), 211–224.

de la Peña, D., Lizcano, D., et al. (2021). Learning through play: Gamification model in university-level distance learning. Entertainment Computing, 39 , 100430.

De-Marcos, L., Garcia-Cabot, A., et al. (2020). Gamifying massive online courses: Effects on the social networks and course completion rates. Applied Sciences (switzerland), 10 (20), 1–17.

Deterding, S., Dixon, D., et al. (2011b). From game design elements to gamefulness: Defining gamification.

Deterding, S., Dixon, D., et al. (2011a). From game design elements to gamefulness: Defining "gamification". In Proceedings of the 15th international academic mindtrek conference: envisioning future media environments (pp. 9–15). Tampere, Finland: Association for Computing Machinery.

Deterding, S. (2012). Gamification: Designing for motivation. Interactions, 19 (4), 14–17.

Dias, J. (2017). Teaching operations research to undergraduate management students: The role of gamification. The International Journal of Management Education, 15 (1), 98–111.

Dichev, C., & Dicheva, D. (2017). Gamifying education: What is known, what is believed and what remains uncertain: A critical review. International Journal of Educational Technology in Higher Education, 14 (1), 9.

Dicheva, D., Dichev, C., et al. (2015). Gamification in education: A systematic mapping study. Educational Technology & Society, 18 , 75–88.

Dikcius, V., Urbonavicius, S., et al. (2021). Learning marketing online: The role of social interactions and gamification rewards. Journal of Marketing Education, 43 (2), 159–173.

Donath, L., Mircea, G., et al. (2020). E-learning platforms as leverage for education for sustainable development. European Journal of Sustainable Development, 9 (2), 1–19.

Donnermann, M., Lein, M., et al. (2021). Social robots and gamification for technology supported learning: An empirical study on engagement and motivation. Computers in Human Behavior, 121 , 106792.

Duggal, K., Gupta, L. R., et al. (2021). Gamification and machine learning inspired approach for classroom engagement and learning. Mathematical Problems in Engineering, 2021 , 9922775.

Enders, B. (2013). GAMIFICATION, GAMES, AND LEARNING: What managers and practitioners need to know . The E-learning Guild.

Facey-Shaw, L., Specht, M., et al. (2020). Do badges affect intrinsic motivation in introductory programming students? Simulation and Gaming, 51 (1), 33–54.

Fajiculay, J. R., Parikh, B. T., et al. (2017). Student perceptions of digital badges in a drug information and literature evaluation course. Currents in Pharmacy Teaching and Learning, 9 (5), 881–886.

Fajri, F. A., R. K. Haribowo P, et al. (2021). Gamification in e-learning: The mitigation role in technostress. International Journal of Evaluation and Research in Education, 10 (2), 606–614.

García, F., Pedreira, O., et al. (2017). A framework for gamification in software engineering. Journal of Systems and Software, 132 , 21–40.

Gari, M. R. N., & Radermacher, A. D. (2018). Gamification in computer science education: A systematic literature review. In ASEE annual conference and exposition, conference proceedings .

Garnett, T., & Button, D. (2018). The use of digital badges by undergraduate nursing students: A three-year study. Nurse Education in Practice, 32 , 1–8.

Garone, P., & Nesteriuk, S. (2019). Gamification and learning: A comparative study of design frameworks . Springer.

Górska, D. (2016). E-learning in Higher Education. The Person and the Challenges. the Journal of Theology, Education, Canon Law and Social Studies Inspired by Pope John Paul II, 6 (2), 35.

Guérard-Poirier, N., Beniey, M., et al. (2020). An educational network for surgical education supported by gamification elements: Protocol for a randomized controlled trial. JMIR Research Protocols, 9 (12), e21273.

Gunawan, F. E., & Jupiter,. (2018). Gamification analysis and implementation in online learning. ICIC Express Letters, 12 (12), 1195–1204.

Gündüz, A. Y., & Akkoyunlu, B. (2020). Effectiveness of gamification in flipped learning. SAGE Open , 10 (4).

Hallifax, S., Serna, A., et al. (2019a). Factors to consider for tailored gamification. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (pp. 559–572). Barcelona, Spain: Association for Computing Machinery.

Hallifax, S., Serna, A., et al. (2019b). Adaptive gamification in education: A literature review of current trends and developments. In Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) (Vol. 11722 LNCS, pp. 294–307).

Hervas, R., Ruiz-Carrasco, D., et al. (2017). Gamification mechanics for behavioral change: A systematic review and proposed taxonomy. In ACM international conference proceeding series .

Hisham, F. B. M. N., & Sulaiman, S. (2017). Adapting gamification approach in massive open online courses to improve user engagement. UTM Computing Proceedings Innovation in Computing Technology and Applications, 2 , 1–6.

Huang, B., & Hew, K. F. (2018). Implementing a theory-driven gamification model in higher education flipped courses: Effects on out-of-class activity completion and quality of artifacts. Computers and Education, 125 , 254–272.

Hunicke, R., Leblanc, M. G., et al. (2004). MDA: A formal approach to game design and game research.

Jianu, E. M., & Vasilateanu, A. (2017). Designing of an e-learning system using adaptivity and gamification. In 2017 IEEE international systems engineering symposium (ISSE) .

Júnior, E., & Farias, K. (2021). ModelGame: A quality model for gamified software modeling learning. In 15th Brazilian symposium on software components, architectures, and reuse (pp. 100–109). Joinville, Brazil: Association for Computing Machinery.

Kalogiannakis, M., Papadakis, S., et al. (2021). Gamification in science education. A systematic review of the literature. Education Sciences, 11 (1), 22.

Kamunya, S., Mirirti, E., et al. (2020). An adaptive gamification model for e-learning. In 2020 IST-Africa conference (IST-Africa) .

Kapp, K. M. (2012). The gamification of learning and instruction: game-based methods and strategies for training and education.

Kapp, K. M. B. L. M. R. (2014). The gamification of learning and instruction fieldbook: Ideas into practice . Wiley.

Kasinathan, V., Mustapha, A., et al. (2018). Questionify: Gamification in education. International Journal of Integrated Engineering, 10 (6), 139–143.

Khaleel, F. L., Ashaari, N. S., et al. (2019). An empirical study on gamification for learning programming language website. Jurnal Teknologi, 81 (2), 151–162.

Khaleel, F. L., Ashaari, N. S., et al. (2020). The impact of gamification on students learning engagement. International Journal of Electrical and Computer Engineering, 10 (5), 4965–4972.

Khalil, M., Wong, J., et al. (2018). Gamification in MOOCs: A review of the state of the art. In IEEE global engineering education conference, EDUCON .

Kim, J. T., & Lee, W.-H. (2015). Dynamical model for gamification of learning (DMGL). Multimedia Tools and Applications, 74 (19), 8483–8493.

Kitchenham, B. (2004). Procedures for performing systematic reviews . Software Engineering Group Department of Computer Science, Keele University.

Knutas, A., van Roy, R., et al. (2019). A process for designing algorithm-based personalized gamification. Multimedia Tools and Applications, 78 (10), 13593–13612.

Kyewski, E., & Krämer, N. C. (2018). To gamify or not to gamify? An experimental field study of the influence of badges on motivation, activity, and performance in an online learning course. Computers and Education, 118 , 25–37.

Landers, R. N. (2014). Developing a theory of gamified learning: Linking serious games and gamification of learning. Simulation & Gaming, 45 (6), 752–768.

Lavoué, E., Monterrat, B., et al. (2019). Adaptive gamification for learning environments. IEEE Transactions on Learning Technologies, 12 (1), 16–28.

Legaki, N. Z., & Hamari, J. (2020). Gamification in statistics education: A literature review. In CEUR workshop proceedings .

Legaki, N. Z., Xi, N., et al. (2020). The effect of challenge-based gamification on learning: An experiment in the context of statistics education. International Journal of Human Computer Studies, 144 , 102496.

Llorens-Largo, F., Gallego-Durán, F. J., et al. (2016). Gamification of the learning process: Lessons learned. IEEE Revista Iberoamericana De Tecnologias Del Aprendizaje, 11 (4), 227–234.

Majuri, J., Koivisto, J., et al. (2018). Gamification of education and learning: A review of empirical literature. In CEUR workshop proceedings .

Marín, B., Frez, J., et al. (2019). An empirical investigation on the benefits of gamification in programming courses. ACM Transactions on Computing Education, 19 (1), 1–22.

Mi, Q., Keung, J., et al. (2018). A gamification technique for motivating students to learn code readability in software engineering. In Proceedings—2018 international symposium on educational technology, ISET 2018 .

Milenković, I., Šošević, U., et al. (2019). Improving student engagement in a biometric classroom: The contribution of gamification. Universal Access in the Information Society, 18 (3), 523–532.

Mora, A., Riera, D., et al. (2017). Gamification: A systematic review of design frameworks. Journal of Computing in Higher Education, 29 (3), 516–548.

Morschheuser, B., Werder, K., et al. (2017). How to gamify? A method for designing gamification.

Morschheuser, B., Hassan, L., et al. (2018). How to design gamification? A method for engineering gamified software. Information and Software Technology, 95 , 219–237.

Naik, V., & Kamat, V. V. (2015). Adaptive and gamified learning environment (AGLE). In 2015 IEEE seventh international conference on technology for education (T4E) (pp. 7–14).

Nicholson, S. (2012). A user-centered theoretical framework for meaningful gamification.

Nielson, B. (2017). Gamification mechanics vs. gamification dynamics . Retrieved from https://www.yourtrainingedge.com/gamification-mechanics-vs-gamification-dynamics/ .

Ozdamli, S. K. A. F. (2018). A review of research on gamification approach in education.

Pakinee, A., & Puritat, K. (2021). Designing a gamified e-learning environment for teaching undergraduate ERP course based on big five personality traits. Education and Information Technologies, 26 (4), 4049–4067.

Park, J., De, L., et al. (2019). GAMESIT: A gamified system for information technology training. Computers and Education, 142 , 103643.

Pérez-López, I. J., Rivera García, E., et al. (2017). “The prophecy of the chosen ones”: An example of gamification applied to university teaching. Revista Internacional De Medicina y Ciencias De La Actividad Fisica y Del Deporte, 17 (66), 243–260.

Pilkington, C. (2018). A playful approach to fostering motivation in a distance education computer programming course: Behaviour change and student perceptions. International Review of Research in Open and Distance Learning, 19 (3), 282–298.

Rivera, E. S., & Garden, C. L. P. (2021). Gamification for student engagement: A framework. Journal of Further and Higher Education, 45 (7), 999–1012.

Rodríguez, I., Puig, A., et al. (2022). Towards adaptive gamification: A method using dynamic player profile and a case study. Applied Sciences, 12 (1), 486.

Romero-Rodriguez, L. M., Ramirez-Montoya, M. S., et al. (2019). Gamification in MOOCs: Engagement application test in energy sustainability courses. IEEE Access, 7 , 32093–32101.

Ropero-Padilla, C., Rodriguez-Arrastia, M., et al. (2021). A gameful blended-learning experience in nursing: A qualitative focus group study. Nurse Education Today, 106 , 105109.

van Roy, R., & Zaman, B. (2017). Why gamification fails in education and how to make it successful: Introducing nine gamification heuristics based on self-determination theory (pp. 485–509).

Ryan, R., & Deci, E. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. The American Psychologist, 55 , 68–78.

Saggah, A., Atkins, A. S., et al. (2020). A review of gamification design frameworks in education. In 2020 Fourth international conference on intelligent computing in data sciences (ICDS) .

Saleem, A. N., Noori, N. M., et al. (2021). Gamification applications in E-learning: A literature review. Technology, Knowledge and Learning, 27 , 139–159.

Sanal, A. (2019). Content gamification vs structured gamification in E-learning . Retrieved from https://playxlpro.com/content-gamification-vs-structured-gamification-in-e-learning/ .

Sanchez, D. R., Langer, M., et al. (2020). Gamification in the classroom: Examining the impact of gamified quizzes on student learning. Computers and Education, 144 , 103666.

Schöbel, S., & Söllner, M. (2016). How to gamify information systems—Adapting gamification to individual user preferences.

Schonfeld, E. (2010). SCVNGR's secret game mechanics playdeck.

Seaborn, K., & Fels, D. I. (2015). Gamification in theory and action: A survey. International Journal of Human-Computer Studies, 74 , 14–31.

da Silva, R. J. R., Rodrigues, R. G., et al. (2019)."Gamification in management education: A systematic literature review. BAR - Brazilian Administration Review , 16 (2).

Simões, J., Redondo, R. P. D., et al. (2013). A social gamification framework for a K-6 learning platform. Computers in Human Behavior, 29 , 345–353.

Smith, T. (2017). Gamified modules for an introductory statistics course and their impact on attitudes and learning. Simulation and Gaming, 48 (6), 832–854.

Sofiadin, A., & Azuddin, M. (2021). An initial sustainable e-learning and gamification framework for higher education. In International conferences on mobile learning 2021 and educational technologies 2021 .

Subhash, S., & Cudney, E. A. (2018). Gamified learning in higher education: A systematic review of the literature. Computers in Human Behavior, 87 , 192–206.

Swacha, J. (2021). State of research on gamification in education: A bibliometric survey. Education Sciences, 11 , 69.

Toda, A. M., Oliveira, W., et al. (2019). A taxonomy of game elements for gamification in educational contexts: Proposal and evaluation. In 2019 IEEE 19th international conference on advanced learning technologies (ICALT) .

Toda, A., Toledo Palomino, P., et al. (2020). How to gamify learning systems? An experience report using the design sprint method and a taxonomy for gamification elements in education. Educational Technology & Society, 22 , 47–60.

Towongpaichayont, W. (2021). A guideline of designing gamification in the classroom and its case study. ICIC Express Letters, 15 (6), 639–647.

Tsay, C. H. H., Kofinas, A., et al. (2018). Enhancing student learning experience with technology-mediated gamification: An empirical study. Computers and Education, 121 , 1–17.

Urh, M., Vukovic, G., et al. (2015). The model for introduction of gamification into E-learning in higher education. Procedia - Social and Behavioral Sciences, 197 , 388–397.

Uz Bilgin, C., & Gul, A. (2020). Investigating the effectiveness of gamification on group cohesion, attitude, and academic achievement in collaborative learning environments. TechTrends, 64 (1), 124–136.

van Gaalen, A. E. J., Brouwer, J., et al. (2021). Gamification of health professions education: A systematic review. Advances in Health Sciences Education, 26 (2), 683–711.

van Roy, R., & Zaman, B. (2019). Unravelling the ambivalent motivational power of gamification: A basic psychological needs perspective. International Journal of Human Computer Studies, 127 , 38–50.

Wendy Hsin-Yuan Huang, D. S. (2013). Gamification of education . Rotman School of Management, University of Toronto.

Werbach, K., & Hunter, D. (2012). For the win: How game thinking can revolutionize your business.

Winanti, W., Abbas, B. S., et al. (2020). Gamification framework for programming course in higher education. Journal of Games, Game Art, and Gamification, 5 (2), 54–57.

Wongso, O., Rosmansyah, Y., et al. (2014). Gamification framework model, based on social engagement in e-learning 2.0. In 2014 2nd international conference on technology, informatics, management, engineering & environment (pp. 10–14).

Yamani, H. (2021). A conceptual framework for integrating gamification in elearning systems based on instructional design model. International Journal of Emerging Technologies in Learning, 16 , 14–33.

Yildirim, I. (2017). The effects of gamification-based teaching practices on student achievement and students’ attitudes toward lessons. Internet and Higher Education, 33 , 86–92.

Zainuddin, Z., Chu, S. K. W., et al. (2020). The impact of gamification on learning and instruction: A systematic review of empirical evidence. Educational Research Review, 30 , 100326.

Zaric, N., Lukarov, V., et al. (2020). A fundamental study for gamification design: Exploring learning tendencies’ effects. International Journal of Serious Games, 7 (4), 3–25.

Zhao, D., Playfoot, J., et al. (2022). An innovative multi-layer gamification framework for improved STEM learning experience. IEEE Access, 10 , 3879–3889.

Zichermann, G., & Cunningham, C. (2011). Gamification by design: Implementing game mechanics in web and mobile apps . O’Reilly Media, Inc.

Download references

Acknowledgements

Not applicable.

Author information

Authors and affiliations.

Ecole Nationale Supérieure d’Informatique ESI, Ex INI, Algiers, Algeria

Amina Khaldi, Rokia Bouzidi & Fahima Nader

You can also search for this author in PubMed   Google Scholar

Contributions

The authors worked together on the manuscript. All authors have read and approved the final manuscript.

Corresponding author

Correspondence to Amina Khaldi .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Khaldi, A., Bouzidi, R. & Nader, F. Gamification of e-learning in higher education: a systematic literature review. Smart Learn. Environ. 10 , 10 (2023). https://doi.org/10.1186/s40561-023-00227-z

Download citation

Received : 07 November 2022

Accepted : 10 January 2023

Published : 31 January 2023

DOI : https://doi.org/10.1186/s40561-023-00227-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Gamification
• Higher education
• Tertiary education
• Digital learning environments
• Systematic review

• Open access
• Published: 02 September 2024

Building a learning health care community in rural and remote areas: a systematic review

• Shabnam Asghari 1 ,
• Jennifer Bent 1 ,
• Ali Modir 1 ,
• Alison MacDonald 1 ,
• Alison Farrell 2 ,
• Cheri Bethune 1 &
• Wendy Graham 1  

BMC Health Services Research volume  24 , Article number:  1013 ( 2024 ) Cite this article

41 Accesses

Metrics details

A Learning Health Care Community (LHCC) is a framework to enhance health care through mutual accountability between the health care system and the community. LHCC components include infrastructure for health-related data capture, care improvement targets, a supportive policy environment, and community engagement. The LHCC involves health care providers, researchers, decision-makers, and community members who work to identify health care needs and address them with evidence-based solutions. The objective of this study was to summarize the barriers and enablers to building an LHCC in rural areas.

A systematic review was conducted by searching electronic databases. Eligibility criteria was determined by the research team. Published literature on LHCCs in rural areas was systematically collected and organized. Screening was completed independently by two authors. Detailed information about rural health care, activities, and barriers and enablers to building an LHCC in rural areas was extracted. Qualitative analysis was used to identify core themes.

Among 8169 identified articles, 25 were eligible. LHCCs aimed to increase collaboration and co-learning between community members and health care providers, integrate community feedback in health care services, and to share information. Main barriers included obtaining adequate funding and participant recruitment. Enablers included meaningful engagement of stakeholders and stakeholder collaboration.

Conclusions

The LHCC is built on a foundation of meaningful use of health data and empowers health care practitioners and community members in informed decision-making. By reducing the gap between knowledge generation and its application to practice, the LHCC has the potential to transform health care delivery in rural areas.

Peer Review reports

Community participation  is known as the collective involvement of people in assessing their needs [ 1 ]. Prior research suggests that community participation in community services and programming is integral to the health of the community and its sustainability, and such participation can yield positive long-term health outcomes [ 2 ]. Community participation in primary and rural health care services has promoted more accessible and relevant services [ 2 , 3 ], and it can result in higher community-member satisfaction with health services [ 4 ]. There is a long tradition of community-member contributions to various health services and preventative health programs [ 5 , 6 , 7 ].

A Learning Health care System (LHS) is a model that ‘draws from the best scientific evidence while tailoring optimal care to a local health care setting to each patient’ [ 8 , 9 ]. As technology advances, so does access to clinical and person-specific data that can inform health care decision making [ 8 ]. An LHS has three core components: (1) foundational elements; (2) care improvement targets; and (3) a supportive policy environment [ 9 ]. Foundational elements include upgrading digital technology to collect data and facilitate data sharing within the health care system. Care improvement targets assist learning and health through clinical decision support tools, patient-centered care, and clinician-community links. A supportive policy environment includes financial incentives that reward high-value care, encourages transparency within the health care system and commitment from leaders. An LHS can lead to many optimal health care characteristics, including engaged patients, appropriate decision supports, aligned governance, and sharing of necessary data; however, adopting an LHS approach has come with an obstacle, to effectively engaging community members to achieve optimum health outcomes.

A Learning Health Care Community (LHCC) model expands on the LHS model by combining the core elements of an LHS with a fourth component: active and continuous stakeholder and community engagement to improve the quality and value of health care within a community [ 9 ]. An LHCC focuses on health care beyond a health care system and extends beyond an LHS, which focuses on collaboration in health care. The LHCC uses the best practices of the LHS model with evidence-based approaches to engage community members.

The aims of an LHCC include increasing collaboration between public health and traditional health care providers and intending to address all areas required to achieve optimal impact on health for the community. Collaboration from multiple community sectors and effort from the community is vital to the success of the LHCC. Previous literature has found that engaging community members in meaningful conversations regarding their health care is associated with improved health outcomes, quality of time, and better health care experiences [ 9 ].

The LHCC should be trusted and valued by all stakeholders and consist of an economically governable system while being responsive to community needs. Past involvement in discriminatory practices committed against vulnerable and minority populations has impacted community-institution relationships [ 10 ]. However, ongoing commitment of investigators and research teams, treating community members as partners, working collaboratively, increasing skills in relationship building, and listening to diverse voices can overcome mistrust by health care providers and researchers [ 10 , 11 ]. Co-development with communities can assure trustworthy and targeted implementation to address community needs [ 9 ].

An LHS model lacks community involvement and fails to consider community-based health care problems [ 9 ]. As a result, these communities lack control and power over their health care. An LHCC places the voices of communities at the center of their health care, empowers community members, and increases opportunities for health-related learning [ 8 , 9 ]. The LHCC framework is relatively new, and, there is limited research on the impact of LHCCs in rural areas. The majority of existing research on LHCCs is completed in urban areas, or it is outcome-focused with less emphasis on the enablers and barriers that enhanced or hindered the implementation process. We sought to address this gap in existing knowledge by summarizing the barriers and enablers across all LHCCs implemented in rural areas. The current review can be used as a guide to develop more targeted and seamless approaches to develop an LHCC and achieve community health outcomes.

Research aim

The current paper aimed to summarize evidence on the facilitators and barriers that are involved in the LHCC implementation process for rural health care providers, researchers, decision-makers, and community members who wish to implement an LHCC.

Question of interest

What are the barriers and enablers to building a learning health care community in rural areas?

Research design and information sources

We conducted a mixed-methods systematic review. The JBI methodology for mixed-methods systematic reviews was used to inform the entire systematic review. Thematic coding was completed according to the  JBI convergent-integrated approach [ 12 ].

Eligibility criteria

A search of English peer-reviewed published articles that were rural-based, health-related, and involved the implementation of an LHCC was undertaken. Inclusion criteria included the following:

Studies must have been conducted in rural areas. For the purpose of this study, areas were considered rural if they were outside cities [ 1 ]. The research team also checked the author’s definition of a rural area used in the included studies.

Community participation took place and it was related to improving the health services, health knowledge, or well-being of community members.

Studies must have included at least one of the following stakeholders (researcher, health care providers, health system leaders, etc.), as well as a community or patient population.

Co-learning took place by community members and stakeholders. Co-learning refers to a bi-directional learning process, where knowledge exchange occurs between those involved [ 13 ]. Health care providers, leaders, researchers or investigators learned from the community (e.g., their feedback) and community members learned about a health-related topic or health service from health care providers, leaders, researchers or investigators.

Articles were excluded if they were the format of a letter to the editor, a systematic review, an audit, or an editorial. Studies focused on an urban area or simply on an LHS rather than an LHCC were also excluded. All included studies in the review and articles deemed eligible were assessed and included in the review.

Search strategy

Rural and primary health care experts were consulted to identify keywords and studies on LHCCs, community engagement and health care practitioners. Then, in consultation with a librarian, the search terms were tested and refined. Next, a Public Services Librarian from Memorial University of Newfoundland searched databases including Medline (Ovid), Embase (Embase.com) and The Cumulative Index to Nursing and Allied Health Literature (CINAHL; Ebsco) to identify potentially relevant articles up until May 2, 2023. See Additional File 1 for the search history for Ovid Medline. Results were imported into Endnote X9 for deduplication and then into Covidence for screening.

Study selection and data collection process

Relevant titles and abstracts by database searches resulted in the identification of 8169 articles. After duplicates were removed, 8096 articles remained. Articles were screened independently by two trained authors using Covidence, an online software that streamlines the production of systematic reviews. If an abstract was deemed relevant, a vote of ‘Yes’ was assigned to the article, and if an abstract was deemed irrelevant a vote of ‘No’ was assigned to the article. If both reviewers assigned a vote of ‘Yes’ the article moved into the full-text phase, and if both reviewers voted ‘No’, the article was excluded. Any abstracts that received conflicting votes or were assigned a vote of ‘Maybe’ were discussed and resolved between authors. After a consensus was reached, full texts of the remaining articles were obtained and this process was repeated. A calibration exercise was administered to portray the validity of the research on 10% of the articles in each stage and our approach was adjusted if it was required. For the abstract, 816 articles (10%) were randomly selected, and a third reviewer examined these abstracts. Reference lists and citations within full text articles were checked for eligibility. No additional articles were added using this approach. The same process was used for screening and data extraction. Next, the research team confirmed the included articles following review and conflict resolution. Weekly meetings were executed to discuss the articles. If there were any conflicts in article decisions, a third reviewer was involved in the resolution.

Data synthesis methods

Included articles were moved to data extraction by both independent reviewers. A data extraction tool was developed using Microsoft Excel and pilot tested on 5 articles. Both reviewers independently entered the data to the tool. Columns were included that summarized key information pertaining to the article (i.e. Geographical Setting, Study Objectives, Community Group, Enablers and Barriers, etc.).

Data from each article was compiled into the Microsoft Excel sheet and reviewed by the research team. The compiled articles ( n  = 25) were reviewed by several team members and discussed during meetings until we reached consensus. Similar text fragments were grouped and categories were identified. Consistent with the JBI mixed-methods systematic review convergent-integrated approach [ 12 ], categories were identified based on assembled data from both qualitative and quantitative studies. Next, categories were aggregated to form themes relating to the barriers and enablers to building an LHCC. A consensus on the final list of themes was reached through group discussions to reduce bias and ensure consensus was reached.

Enablers and barriers were discussed in relation to the four main components of the LHCC. The Stakeholder and Community Engagement component highlights studies that identified effectively engaging community members to be pivotal to an LHCC success. Infrastructure for health related data-capture was relevant to studies that found harnessing contemporary technology, information sharing through in-person and online interventions, development of educational materials, and health-related data sharing to be vital to patient health. Care Improvement Targets were included as various stakeholders aimed to improve health, disease management, and increase awareness and uptake of prevention interventions within the community. Care Improvement Targets were organized according to the Institute of Medicine’s report: “Crossing the Quality Chasm: A New Health System for the 21st Century” six aims for success (1) safe; (2) effective; (3) timely; (4) patient-centered; (5) efficient; and (6) equitable [ 14 ]. A Supportive Policy Environment was included as acquiring proper funding, resources and support was deemed essential to the LHCC.

Quality appraisal

The quality of each study was assessed independently by two authors using the Mixed Methods Appraisal Tool (MMAT) scoring system [ 15 ]. The average of each reviewer's assigned score determined the quality of each article. Studies of 75% and above were considered good quality, 50–74% were considered fair, and 50% and below were considered low quality. All studies that met inclusion criteria were included, despite the quality appraisal score.

A literature search identified 8169 articles to screen and assess for eligibility. Following screening and the application of inclusion and exclusion criteria, 25 articles were included (Fig.  1 ).

Prisma flow diagram

Studies were conducted in the following countries: the United States ( n  = 15), India ( n  = 3), Canada ( n  = 2), Australia ( n  = 2), Guatemala ( n  = 1), Thailand ( n  = 1), and the Republic of Congo ( n  = 1). The characteristics of included studies are summarized in Table 1 . All studies included stakeholder populations (as defined above) that experienced a form of learning health outcome in their community. There were no age-specific studies, and children were not exclusively excluded. Six articles (24% of all articles) were deemed fair, and 19 studies (76% of all articles) were deemed good quality. The studies were a mix of qualitative ( n  = 14), quantitative ( n  = 3), quality improvement ( n  = 2), and mixed methods ( n  = 6).

Defining a learning health care community

Descriptions and definitions of LHCCs were extracted from all included articles (Table  2 ). Most articles did not directly use ‘learning health care community’ terminology; however, all included key LHCC components.

All LHCC projects aimed to improve health outcomes, such as disease monitoring, management and prevention, and knowledge sharing with community members. All involved collaboration between stakeholders, co-learning, integration of community feedback, and discussed the community participation process. Additionally, most articles discussed developing rural health services such as online learning or research health networks, community-based intervention workshops, learning health systems, and evidence-synthesis programs.

Enablers and barriers to building an LHCC

Common themes emerged across articles. Theme descriptions were created based on the content of included articles and were reviewed by the research team. A consensus on theme names was reached at team meetings. Meaningfully Engaging Stakeholders referred to the use of tools, activities, or community-specific or creative strategies to grasp the interest of diverse stakeholders to be involved in the LHCC implementation process or to be active members within the LHCC. Stakeholder Collaboration referred to bringing together diverse stakeholders to work together to implement the LHCC or to achieve a common goal as members of the LHCC. Using a Participatory Approach was used to describe recruitment and participation of people who are impacted by the LHCC. Strong Stakeholder Relationships referred to the creation of strong interpersonal and mutually beneficial partnerships between stakeholders. Knowledge Sharing referred to learning about a health topic from in-person or online interventions, through the use or creation of knowledge-translation tools, health-related activities or training sessions, or through the use of technology. A Multidisciplinary Team referred to teams that included at least two different stakeholders with different levels of expertise, skills or experience. For example, certain teams included community members, clinicians, nurses, specific patient groups, or community health workers.

The barrier Obtaining Adequate Funding or Research Support referred to challenges to obtain sufficient funding, technology, or personnel to support the development of the LHCC or to sustain the LHCC. Participant Recruitment was used to describe challenges to recruit community members, family members, patients, or other stakeholders that could assist with the development of the LHCC or who serve as active members within the LHCC. A Lack of Knowledge referred to challenges regarding a lack of knowledge about a health topic or illiteracy within the community. This barrier also described a lack of knowledge of how to effectively engage or recruit stakeholders. Maintaining Stakeholder Commitment was used to describe challenges relating to sustaining involvement of LHCC stakeholders, or underestimating the ongoing effort required to promote long-term commitment from stakeholders.

Competing Demands of Stakeholders was used to describe situations when stakeholders within the LHCC had conflicting priorities and therefore, hindered the progress of the LHCC. This barrier was also used to describe when stakeholders struggled to determine a shared mission of the LHCC. Time Constraints were identified by stakeholders that were either unable to or struggled to meet specific deadlines during the LHCC development process, or those who underestimated the time required to build relationships and effectively engage stakeholders.

See Table  3 for the most frequently noted enablers and barriers and the corresponding number of articles. See Table  2 for more detailed information on the enablers and barriers reported in each article.

The most frequently noted enabler that facilitated LHCC implementation included meaningfully engaging stakeholders ( n  = 15). This finding ‘Stakeholder and Community Engagement’ is one of the four main components of the LHCC framework. One LHCC described by Curtis et al. [ 26 ] integrated patient engagement at all stages of their program by ensuring participant voices were heard and incorporated into strategy development. They developed a multidisciplinary team of specialists, Indigenous care providers, patient partners, and policymakers who collectively contributed in the development and implementation process of the LHCC. The LHCC involved shared decision making at all stages and community-member knowledge was leveraged to enhance culturally-safe care.

Other enablers mentioned in four or less articles included having a shared mission, incorporating incremental improvements, demonstrating the value of stakeholder involvement, having strong leadership, prolonging engagement throughout the project, using culturally appropriate research methods, conducting a needs assessment, getting continuous feedback from the community, and providing stakeholders access to their health data.

Allowing patients to engage with their own health data to facilitate participation relates to the ‘Infrastructure for Health-Related Data Capture and Knowledge Sharing’ component of the LHCC model. For example, Arcia et al. [ 16 ] used a participatory-design approach where participants completed health surveys on their self-reported health outcomes and anthropometric measures. Data was returned to participants through clinical infographics and all participants effectively provided helpful feedback during design sessions. As a result, the LHCC led to tailored infographic designs that were more engaging, informative, and comprehensible according to participants. See Fig. 2  for quotes from included articles mapped to each enabler.

Mapping article quotes to LHCC enablers

The most frequently mentioned barrier that hindered the success of LHCC implementation included obtaining adequate funding and resources ( n  = 12). This barrier relates to the ‘Supportive Policy Environment’ component of the LHCC framework. Having adequate funding (i.e. compensation for participants or stakeholders), or needed resources (i.e. technology, supplies, tools, or personnel) to meet LHCC deadlines was deemed essential for specific LHCCs.

Other barriers mentioned in one or two articles included mistrust from the community, achieving effective stakeholder collaboration, a need for flexibility in the research plan, resistance to change, stakeholder burnout, maintaining stakeholder connections, knowledge, and resource dissemination, having limited staff, and time commitment. See Fig.  3  for quotes from included articles mapped to each barrier.

Mapping article quotes to LHCC barriers

Care Improvement Targets involve increased learning through clinician-community links, the use of clinical decision support, patient-centered care and optimizing delivery of health care and community-based resources [ 9 ]. Twenty studies ( n  = 20) reported on care improvement targets. Murray et al. [ 24 ] reported on the ‘Autism Speaks Autism Treatment Network’. This network leveraged community-partnerships to transition to an LHCC. The LHCC incorporated a model of co-production of knowledge between families and clinicians that promoted learning of each other’s expertise and values. They provided opportunities to take part in remote and in-person meetings, and participation in workgroups and monthly webinars. The LHCC resulted in growth of enrollment by almost 700 members. Among these studies ( n = 20), enablers and barriers were described in alignment with the Institute of Medicine’s six aims of quality care [ 14 ].

Patient-centered care: Seventeen studies ( n  = 17) involved the patient or family member involvement to improve patient health or patient-centered care (i.e. by consulting patients and implementing their feedback).

Safety: Five studies ( n = 5) reported safety of health care or research practices (i.e. the Kidney Check LHCC could increase the use of culturally safe practices in Indigenous communities) [ 26 ].

Timely: Five studies ( n  = 5) reported that the LHCC has potential to led to more timely access to interventions, services, and resulted in faster decision-making by health care providers, and decreased wait times for patients.

Efficient: It was reported in eleven studies ( n = 11) that the LHCC could lead to greater efficiency (i.e. one study connected individuals who frequently use emergency services for non-urgent conditions to more appropriate care, resulting in fewer 911 calls) [ 18 ].

Effective: Twenty studies ( n = 20) reported on the effectiveness of the LHCC (i.e. one study developed a program that led to increased health knowledge and disease management strategies amongst affected populations) [ 34 ].

Equitable: Seventeen studies ( n = 17) reported on the equity of the LHCC. For example, one LHCC offered free health knowledge or exercise classes to vulnerable populations [ 33 ].

The current review aimed to summarize available evidence on the enablers and barriers of rural LHCC implementation. The review of LHCC studies suggests that LHCC implementation can increase community participation in planning health services and health-related activities. Health-related organizations implemented the LHCC to increase collaboration between community members and stakeholders, integrate community feedback in planning, and increase co-learning to expand knowledge sharing and improve health and services within the community.

Stakeholder engagement and collaboration were crucial enablers to LHCC success. It is unsurprising that stakeholder engagement was identified by multiple authors as active and continuous stakeholder and community engagement is a core component of an LHCC. Osborn & Squires [ 40 ] investigated perspectives on patient engagement using a survey in 11 countries: Australia, Canada, France, Germany, the Netherlands, New Zealand, Norway, Sweden, Switzerland, the United Kingdom, and the United States of America. They found that across countries patients who were more engaged with their health services, received greater quality care, and reported more positive views on the health care system. When stakeholders understand the value of the project, their involvement are provided with clear direction, and can interact with their data, they are more likely to be engaged. Also, the importance of stakeholder collaboration in health care is reiterated by Mullins et al [ 9 ] and supported by the Robert Wood Johnson Culture Health Framework [ 41 ]. According to this framework, achieving healthy equitable communities involves collaboration from multiple sectors to have the greatest impact on public and community health [ 9 , 41 ]. Methods noted to facilitate collaboration include staying connected through various platforms (i.e. online networks, email or text-message), maintaining this contact on a regular basis, and meeting regularly as a group.

Although funding agencies have recognized the value of the LHS and LHCC models, obtaining adequate funding and resources was the greatest barrier identified in included studies [ 9 ]. Authors emphasized the challenge to receive robust investments from partner organizations and funding agencies to meet project staffing requirements, compensate participants for their time, and to purchase equipment and materials needed for the LHCC. In order to attract investors, it is important to document the benefits of the LHCC and present this information to organizational leaders and stakeholders. Furthermore, developing a feasible plan and demonstrating how you can sustain innovation can be influential for investors. Participant recruitment was also commonly documented as an obstacle. Particular solutions to address this issue have been noted in the included studies, such as participants being invited by a trusted and recognized expert in the field, putting effort into building relationships with participants, collaborating with them to determine clear goals, determine meeting frequency in advance, and have a clear decision-making process.

LHCCs aimed to either increase learning by community members about health topics (i.e. increase health literacy, interest in preventative health care) from experts within the field, researchers or health care providers, or they aimed to increase learning by stakeholders from the community or a particular community group (i.e. consulting midwives on how to retain them).

The current review outlines some overlap in enablers and barriers to urban or larger scale LHCCs. Similar to rural LHCCs, urban LHCCs also identify stakeholder collaboration, inclusion of multi-disciplinary teams, and knowledge sharing as enablers [ 42 , 43 , 44 ]. A lack of knowledge, and competing demands of stakeholders were also identified as barriers to implementation of urban LHCC implementation [ 42 , 43 ]. Enablers specific to urban LHCCs include having access to population health data, access to technology that can analyze large-scale data, and conducting research to make comparisons and pinpoint trends in data [ 42 , 44 , 45 ].

Barriers identified in urban LHCCs included concerns around legal, ethical and policy challenges regarding patient health information and challenges relating to developing technological safeguards to protect the safety and security of patient health information. In addition, language barriers were also identified as a barriers to urban LHCCs [ 43 ]. Differences in enablers and barriers between urban–rural areas are related to the differences between rural–urban communities. For example, urban communities consist of much larger more diverse populations with greater access to patient health data and advanced technology.

Additionally, although having a multidisciplinary team is identified in both rural and urban LHCCs as an enabler, this term is defined differently between the communities. Rural LHCCs face challenges to recruit and retain their workforce. For example, only 8% of physicians in Canada work in rural areas [ 46 , 47 ]. Therefore, rural ‘multidisciplinary teams’ are less likely to include specialists, epidemiologists, data-analysts, health-system leaders or policy-makers compared to urban LHCCs. Further, urban LHCCs place less importance on sustained community engagement. One reason for this may be due to challenges relating to the effort and time required to effectively engage larger populations. The small population size of rural communities may be more practical to engage for longer-periods of time.

Rural populations experience unique challenges compared to their urban counterparts. Rural-areas differ in terms of their geographic location, population size, weather, size of their workforce, and access to financial resources and health care services [ 48 , 49 ]. Rural communities often have a greater proportion of elderly residents with chronic conditions [ 50 ], have limited access to health-information [ 51 ], and health care providers [ 52 ]. Despite these challenges, rural LHCCs provide an opportunity to address rural health needs. Driven by evidence, rural LHCCs place considerable effort into engaging the community in the LHCC implementation process. Shared-decision-making and co-production of knowledge are priorities of rural LHCCs. Due to workforce shortages, health-related education and training for community members appear to be pivotal in LHCC success. As evidenced by the current review, acquiring adequate funding to support LHCC implementation, meaningfully engaging stakeholders, and fostering collaboration are key components of successful rural LHCCs. Furthermore, LHCCs strive to increase health system transparency and accountability of participating entities to make improvements to health care [ 9 ]. By increasing awareness of community health problems, and clinical and financial support, the LHCC has potential to have a positive impact on community health.

The role of the LHCC is to improve the quality and efficiency of health care, therefore the LHCC must not overlook the impact of the social determinants of health (SDOH). Extensive research has emphasized the role of the SDOH on patient and community wellness, despite the quality of health care available [ 53 ]. For example, the United States spends an extensive amount of money on health care but is ranked last compared to other developed nations for a multitude of health outcomes, likely due to a lack of public health and social programs [ 54 ]. Despite the significance of the SDOH on patient health outcomes, the majority of included articles did not report on the SDOH. Future research should investigate the role an LHCC can play in addressing community and patient social risk factors. In addition, there was a lack of studies that focused on specific patient outcomes following LHCC implementation, and the impact of patient outcomes on the success of the LHCC. There is a need for greater research on LHCCs related to patient groups facing certain health diseases and evaluation of the impact of the LHCC on these health outcomes. Also, included studies did not report on the use of pragmatic clinical trials. Future research should investigate the role of clinical trials on LHCCs in rural areas.

Overall, there was repetition of several enablers and barriers mentioned in the included studies (i.e. effectively engaging stakeholders was mentioned in 15 studies and acquiring adequate funding was mentioned in 12 studies). However, there was also variability in barriers and enablers noted by authors mentioned in only one or two articles. For example, having strong leadership was mentioned as an enabler in two articles, and having limited staff was mentioned as a barrier in one article. One explanation for this could be that certain enablers and barriers were very specific to the LHCC being implemented. Therefore, enablers and barriers can vary based on the approach that is taken by implementers. Further, certain LHCCs may not have had one strong leader helping with implementation, and instead used a committee that struggled to make decisions. In this case, a strong leader would not be noted as an enabler. Nevertheless, this variation indicates that there is a lack of knowledge on the enablers and barriers that facilitate LHCC implementation in rural areas and this is an area for future research.

There were several strengths to the current study. First, this study expands on prior research by collecting the most common enablers and barriers that result from LHCC development and informs implementers of the most common obstacles that may be faced. Second, the approach taken to conduct the current review was closely aligned with recommendations [ 55 ]. Third, given the current circumstances of rural medicine and patient engagement, our research team deemed it imperative to conduct a deeper review of how to encourage stakeholders and patients to be involved in a collaborative partnership. This study is the fundamental step to a more thorough environmental scan.

The present study must be considered with limitations in mind. Although a minimum of two researchers and a librarian completed an exhaustive search for articles that met the inclusion criteria, there is a chance that some studies were missed or potentially miscategorized. Additionally, included studies differed across cultural, geographic, sociological, and geo-political boundaries across countries and this may impact the generalizability of our findings in different settings. Searches were limited to those in the English language and published articles. Articles published following the completion of the literature search were not included in this review. Therefore, this review may be subject to publication bias. Further, two studies noted that their findings may have been impacted by selection bias, as despite participation being encouraged in their respective LHCC, some participants did not choose to attend [ 23 , 28 ]. Additionally, Golden et al. [ 28 ] recruited participants based on convenience, subjecting their findings to sampling bias and Gierisch et al. [ 23 ] opted to not record their interviews, acknowledging the potential for introducing social desirability bias and confirmatory bias. Although some articles assessed the effect of the LHCC following using pre- and post-implementation assessments, the chosen metrics varied across the studies and approximately half of included studies did not evaluate the impact of the LHCC. Therefore, the current study lacked a systemic approach and analysis of LHCC outcomes.

Ample evidence has shown that rural areas face poorer health outcomes than urban areas, due to a lack of infrastructure, health care providers, screening and medical equipments, and accessibility. The current study can be particularly useful to health care providers, researchers, health-related organizations, policymakers and leaders living in rural areas who want to reduce this health outcome gap, improve health-knowledge, patient engagement in health services planning, and research in their community. The current review summarizes the available evidence on the enablers and barriers to implementing an LHCC in rural communities and it is useful for anyone looking to implement an LHCC. Our findings highlight a clear lack of studies on rural LHCC implementation, as well as a lack of studies that evaluate the effect of the LHCC post-implementation, preventing the opportunity to conduct a meta-analysis. The LHCC model is relatively new, therefore, this review can increase awareness of the many benefits of this model and help to inform approaches to transition to this model.

The LHCC is built collaboratively on a foundation of meaningful use of health data and empowers health care practitioners and patients in informed decision-making. Despite the number of barriers to implementing an LHCC, all studies that did implement LHCCs reported potential for positive outcomes. Although there are a variety of commentary, and perspective papers on LHCC development, this study adds to existing literature by summarizing the essential enablers and experienced barriers to facilitate LHCC implementation. Taken as a whole, an LHCC can be a potential solution to increase community engagement and collaboration between health care providers, researchers, decision makers and community members, and to mobilize resources in rural areas, and thereby lead to improved health services, health-knowledge, and health outcomes in these regions. Further research is needed on evidence-based approaches to effectively engage communities to be involved with their health care as well as the long-term outcomes of LHCC implementation in rural areas.

Availability of data and materials

Not applicable.

Abbreviations

• Learning health system
• Learning health care community

Preston R, Waugh H, Larkins S, Taylor J. Community participation in rural primary health care: intervention or approach? Aust J Prim Health. 2010;16(1):4. https://doi.org/10.1071/py09053 .

Article   PubMed   Google Scholar  

Taylor J, Wilkinson D, Cheers B. Working with Communities in Health and Human Services. South Melbourne, Australia: Oxford University Press; 2008. https://researchonline.jcu.edu.au/29462/ . Accessed 10 Apr 2023.

National Rural Health Alliance. Healthy Horizons Outlook 2003–2007. 2003. https://www.ruralhealth.org.au/content/healthy-horizons-outlook-2003-2007 . Accessed 3 Apr 2023.

‌4. Kilpatrick S. Multi-level rural community engagement in health. Australian Journal of Rural Health. 2009;17(1):39–44. https://doi.org/10.1111/j.1440-1584.2008.01035.x .

Collins Y. The more things change, the more they stay the same: health care in regional Victoria. In: Lockie S, Bourke L, editors. Rurality bites, the social and environmental transformation of rural Australia. Annandale, Australia: Pluto Press; 2001.

Google Scholar  

Brady M, Byrne J, Henderson G. “Which bloke would stand up for Yalata?” The struggle of an Aboriginal community to control the availability of alcohol. Australian Aboriginal Studies. 2003;2003(2):62–62. https://go.gale.com/ps/i.do?id=GALE%7CA123582158&sid=googleScholar&v=2.1&it=r&linkaccess=abs&issn=07294352&p=AONE&sw=w&userGroupName=anon%7E6fa8ff7e .

Taylor J, Wilkinson D, Cheers B. Community participation in organising rural general practice: Is it sustainable? Aust J Rural Health. 2006;14(4):144–7. https://doi.org/10.1111/j.1440-1584.2006.00790.x .

Olsen L, Aisner D, McGinnis JM, eds. Roundtable on Evidence-Based Medicine. The Learning Healthcare System: Workshop Summary. National Academies Press, US: Institute of Medicine; 2007. https://www.ncbi.nlm.nih.gov/books/NBK53494/ . Accessed 15 Apr 2023.

‌Mullins CD, Wingate LT, Edwards HA, Tofade T, Wutoh A. Transitioning from learning healthcare systems to learning health care communities. J Comp Eff Res. 2018;7(6):603–614. https://doi.org/10.2217/cer-2017-0105 .

Irby MB, Moore KR, Mann-Jackson L, Hamlin D, Randall I, Summers P, et al. Community-engaged research: common themes and needs identified by investigators and research teams at an emerging academic learning health system. Int J Environ Res Public Health. 2021;18(8):3893. https://doi.org/10.3390/ijerph18083893 .

Article   PubMed   PubMed Central   Google Scholar  

Rhodes SD. Authentic engagement and community-based participatory research for public health and medicine. In: Rhodes SD, Aronson RL, editors. Innovations in HIV prevention research and practice through community engagement. New York: Springer; 2014. p. 1–10.

Chapter   Google Scholar  

JBI Manual for Evidence Synthesis. JBI eBooks. https://synthesismanual.jbi.global/ . Accessed 15 July 2022.

Redko C, Bessong P, Burt D, Luna M, Maling S, Moore C, et al. Exploring the significance of bidirectional learning for global health education. Ann Glob Health. 2017;82(6):955. https://doi.org/10.1016/j.aogh.2016.11.008 .

Article   Google Scholar  

Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, D.C: National Academy Press; 2001. https://www.med.unc.edu/neurosurgery/wp-content/uploads/sites/460/2018/10/Crossing-the-Quality-Chasm.pdf . Accessed Jan 2023.

Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, et al. The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Educ Inf. 2018;34(4):285–91. https://doi.org/10.3233/EFI-180221 .

Arcia A, Suero-Tejeda N, Bales ME, Merrill JA, Yoon S, Woollen J, et al. Sometimes more is more: iterative participatory design of infographics for engagement of community members with varying levels of health literacy. J Am Med Inform Assoc. 2016;23(1):174–83. https://doi.org/10.1093/jamia/ocv079 .

Kunz S, Ingram M, Piper R, Wu T, Litton N, Brady J, et al. Rural collaborative model for diabetes prevention and management: a case study. Health Promot Pract. 2017;18(6):798–805. https://doi.org/10.1177/1524839917712730 .

Carpenter D, Hassell S, Mardon R, Fair S, Johnson M, Siegel S, et al. Using Learning Communities to Support Adoption of Health Care Innovations. The Joint Commission Journal on Quality and Patient Safety. 2018;44(10):566–73. https://doi.org/10.1016/j.jcjq.2018.03.010 .

Chhabra R, Rivera A, Sharma N, Ghosh S, Bauman L. Engaging community members as health advocates in a peer driven intervention—a cervical cancer prevention pilot in Punjab, India. Glob J Health Educ Promot. 2018;18(1):s37–53.

Fung-Kee-Fung M, Maziak DE, Pantarotto JR, Smylie J, Taylor L, Timlin T, et al. Regional process redesign of lung cancer care: a learning health system pilot project. Curr Oncol. 2018;25(1):59–66. https://doi.org/10.3747/co.25.3719 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Key KD, Lewis EY. Sustainable community engagement in a constantly changing health system. Learn Health Syst. 2018;2(3):e10053. https://doi.org/10.1002/lrh2.10053 .

Myers RE, DiCarlo M, Romney M, Fleisher L, Sifri R, Soleiman J, et al. Using a health system learning community strategy to address cancer disparities. Learn Health Syst. 2018;2(4):e10067. https://doi.org/10.1002/lrh2.10067 .

‌Gierisch JM, Hughes JM, Williams JW, Gordon AM, Goldstein KM. Qualitative Exploration of Engaging Patients as Advisors in a Program of Evidence Synthesis. Medical Care. 2019;57(10 Suppl 3):S246–52. https://doi.org/10.1097/MLR.0000000000001174 .

Murray DS, Anixt JS, Coury DL, Kuhlthau K, Seide J, Kelly A, et al. Transforming an Autism Pediatric Research Network into a Learning Health System: Lessons Learned. Pediatric Quality & Safety [Internet]. 2019;4(2):e152. Available from:  https://doi.org/10.1097/pq9.0000000000000152

Baba A, Martineau T, Theobald S, Sabuni P, Nobabo MM, Alitimango A, et al. Developing strategies to attract, retain and support midwives in rural fragile settings: participatory workshops with health system stakeholders in Ituri Province, Democratic Republic of Congo. Health Res Policy Syst. 2020;18(1):133. https://doi.org/10.1186/s12961-020-00631-8 .

Curtis S, Martin H, DiNella M, Lavallee B, Chartrand C, McLeod L, et al. Kidney Check Point-of-Care Testing—Furthering Patient Engagement and Patient-Centered Care in Canada’s Rural and Remote Indigenous Communities: Program Report. Can J Kidney Health Dis. 2021;8:205435812110037. https://doi.org/10.1177/20543581211003744 .

Donahue MA, Herman ST, Dass D, Farrell K, Kukla A, Abend NS, et al. Establishing a learning healthcare system to improve health outcomes for people with epilepsy. Epilepsy Behav. 2021;117:107805. https://doi.org/10.1016/j.yebeh.2021.107805 .

Golden RE, Klap R, Carney DV, Yano EM, Hamilton AB, Taylor SL, et al. Promoting learning health system feedback loops: Experience with a VA practice-based research network card study. Healthcare. 2021;8:100484. https://doi.org/10.1016/j.hjdsi.2020.100484 .

Keck C, Hartley DM, Havens M, Margolis PA, Seid M. Getting what is needed, when it's needed: sharing information, knowledge, and know-how in a Collaborative Learning Health System. Learn Health Syst. 2021;5(3):e10268. https://doi.org/10.1002/lrh2.10268 .

Beks H, Amos T, Bell J, Ryan J, Hayward J, Brown A, et al. Participatory research with a rural Aboriginal Community Controlled Health Organisation: lessons learned using the CONSIDER statement. Rural Remote Health. 2022;22(1):6740. https://doi.org/10.22605/RRH6740 .

Lindeman MA, Togni SJ. Improving services for Aboriginal women experiencing sexual violence: working at the knowledge interface. Aust Soc Work. 2022;75(3):372–84.

Marsh Z, Teegala Y, Cotter V. Improving diabetes care of community-dwelling underserved older adults. J Am Assoc Nurse Pract. 2022;34(10):1156–66. https://doi.org/10.1097/JXX.0000000000000773 .

Mishra A, Singh AK, Parida SP, Pradhan S, Nair J. Understanding Community Participation in Rural Health Care: A Participatory learning and Action approach. Frontiers in Public Health. 2022;10. Available from:  https://doi.org/10.3389/fpubh.2022.860792

Oser TK, Zittleman L, Curcija K, Kwan B, Burke S, Gonzalez S, Huss K, Johnson M, Sanchez N, Neuberger J, Iacob E, Simonetti J, Litchman M. Informing a Randomized Control Trial in Rural Populations: Adaptation of a Diabetes Self-Management Education and Support Intervention. JMIR Diabetes. 2022;7(2): e35664. https://doi.org/10.2196/35664 .

Pitchalard K, Moonpanane K, Wimolphan P, Singkhorn O, Wongsuraprakit S. Implementation and evaluation of the peer-training program for village health volunteers to improve chronic disease management among older adults in rural Thailand. Int J Nurs Sci. 2022;9(3):328–33. https://doi.org/10.1016/j.ijnss.2022.06.011 .

Pullyblank K, Brunner W, Wyckoff L, Krupa N, Scribani M, Strogatz D. Implementation of evidence-based disease self-management programs in a rural region: leveraging and linking community and health care system assets. Health Educ Behav. 2022:10901981221078516. https://doi.org/10.1177/10901981221078516 . Epub ahead of print.

Quraishi S, Quraishi H, Yadav H, Singh A, Fasih I, Vasquez NA, et al. Digital storytelling and community engagement to find missing TB cases in rural Nuh, India. Trop Med Infect Dis. 2022;7(3):49. https://doi.org/10.3390/tropicalmed7030049 .

Gregg C, Valdez M, Stollak I, Martin S, Story WT, Perry HB. Reducing inequities in maternal and child health in rural Guatemala through the CBIO+ Approach of Curamericas: 7. The empowering effect of Care Groups. Int J Equity Health. 2023;21(Suppl 2):199. https://doi.org/10.1186/s12939-022-01759-5 .

Niranjan SJ, Opoku-Agyeman W, Hardy CM, Bowman T, Vedre-Kyanam A, Hearld KR, et al. Using community health advisors to increase lung cancer screening awareness in the black belt: a pilot study. J Cancer Educ. 2023;38(4):1286–95. https://doi.org/10.1007/s13187-022-02261-w .

Osborn R, Squires D. International Perspectives on Patient Engagement. J Ambul Care Manage. 2012;35(2):118–28. https://doi.org/10.1097/jac.0b013e31824a579b .

‌24. The Robert Wood Johnson Foundation. Building a culture of health. https://www.cultureofhealth.org/en/taking-action.html (Date unknown). Accessed 20 Apr 2023.

Bernstein JA, Friedman C, Jacobson P, Rubin JC. Ensuring public health’s future in a national-scale learning health system. Am J Prev Med. 2015;48(4):480–7. https://doi.org/10.1016/j.amepre.2014.11.013 .

McCall DS, Rootman I, Bayley D. International School Health Network: an informal network for advocacy and knowledge exchange. Promot Educ. 2005;12(3–4):173–7. https://doi.org/10.1177/10253823050120030121 .

Edwards EM, Ehret DEY, Soll RF, Horbar JD. Vermont Oxford Network: a worldwide learning community. Transl Pediatr. 2019;8(3):182–92. https://doi.org/10.21037/tp.2019.07.01 .

Corrigan D, Munnelly G, Kazienko P, Kajdanowicz T, Soler JK, Mahmoud S, Porat T, Kostopoulou O, Curcin V, Delaney B. Requirements and validation of a prototype learning health system for clinical diagnosis. Learn Health Syst. 2017;1(4): e10026. https://doi.org/10.1002/lrh2.10026 .

Canadian Institute for Health Information. Supply, distribution and migration of physicians in Canada 2015—data tables. Ottawa, ON: Canadian Institute for Health Information; 2016.

Bosco C, Oandasan I. Review of family medicine within rural and remote Canada: education, practice, and policy. Mississauga, ON: College of Family Physicians of Canada; 2016.

Chen X, Orom H, Hay JL, Waters EA, Schofield E, Li Y, Kiviniemi MT. Differences in Rural and Urban Health Information Access and Use. J Rural Health. 2019;35(3):405–17. https://doi.org/10.1111/jrh.12335 .

McIlhenny C, Guzic B, Knee D, Demuth B, Roberts J. Using technology to deliver healthcare education to rural patients. Rural & Remote Health . 2011;11(4). https://www.rrh.org.au/journal/article/1798

Rural Health Information Hub. (2024). Rural Public Health Agencies. https://www.ruralhealthinfo.org/topics/public-health . Accessed 10 Feb 2024.

Lustria ML, Smith SA, Hinnant CC. Exploring digital divides: an examination of eHealth technology use in health information seeking, communication and personal health information management in the USA. Health Informatics J. 2011;17(3):224–243. https://pubmed.ncbi.nlm.nih.gov/21937464

Learmonth YC, Adamson BC, Balto JM, Chiu CY, Molina-Guzman I, Finlayson M, Riskin BJ, Motl RW. Multiple sclerosis patients need and want information on exercise promotion from healthcare providers: a qualitative study. Health Expect. 2017;20(4):574–83. https://doi.org/10.1111/hex.12482 .

Palakshappa D, Miller DP, Rosenthal GE. Advancing the learning health system by incorporating social determinants. Am J Manag Care. 2020;26(1):e4–6. https://doi.org/10.37765/ajmc.2020.42146 .

Papanicolas I, Woskie LR, Jha AK. Health Care Spending in the United States and Other High-Income Countries. JAMA. 2018;319(10):1024–39. https://doi.org/10.1001/jama.2018.1150 .

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Rev. 2021;10(89). https://doi.org/10.1186/s13643-021-01626-4 .

Download references

Acknowledgements

The Rural360 team is supported by the use of Memorial University of Newfoundland facilities.

The Rural360 team is funded by the International Grenfell Association and the Mitacs Accelerate Program.

Author information

Authors and affiliations.

Department of Family Medicine, Faculty of Medicine, Newfoundland and Labrador, Centre for Rural Health Studies, Memorial University of Newfoundland, 300 Prince Philip Dr, St. John’s, NL, A1B 3V6, Canada

Shabnam Asghari, Jennifer Bent, Ali Modir, Alison MacDonald, Cheri Bethune & Wendy Graham

Faculty of Medicine, Memorial University of Newfoundland, Newfoundland and Labrador, St. John’s, Canada

Alison Farrell

You can also search for this author in PubMed   Google Scholar

Contributions

S.A. helped to determine the research question, objectives, and inclusion and exclusion criteria, provided guidance on article screening, the study design, and data analysis, and contributed to article development and the quality appraisal. J.B. and A.M. completed abstract, and article screening, the quality appraisal, data analysis and interpretation, article writing and table and figure development. J.B. implemented all feedback from the research team to the article. A.M. helped develop the research question and objectives, assisted with the development of the inclusion and exclusion criteria, abstract and article screening, and table, and figure development. A.M. and S.A. provided feedback throughout the course of the project. A.F. conducted the literature search and provided feedback on article screening. C.B. and W.G. provided constructive feedback to the research team, assisted in the development of the research question, provided guidance on the research methods, and feedback on the article and supporting documentation. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Shabnam Asghari .

Ethics declarations

Ethics approval and consent to participate.

The need for ethics approval was waived for the current study, as it is a systematic review.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1. search strategy, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Asghari, S., Bent, J., Modir, A. et al. Building a learning health care community in rural and remote areas: a systematic review. BMC Health Serv Res 24 , 1013 (2024). https://doi.org/10.1186/s12913-024-11194-7

Download citation

Received : 02 November 2023

Accepted : 11 June 2024

Published : 02 September 2024

DOI : https://doi.org/10.1186/s12913-024-11194-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Rural health
• Community engagement
• Community participation

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

Systematic review and recommendations for re-irradiation for intraprostatic radiorecurrent prostate cancer after definitive radiation therapy

• Topic Paper
• Published: 12 September 2024
• Volume 42 , article number  520 , ( 2024 )

Cite this article

• Anjali L. Saripalli 1 ,
• Bhanu Prasad Venkatesulu 1 ,
• Nicholas G. Nickols 2 , 3 ,
• Luca F. Valle 2 , 3 ,
• Matthew M. Harkenrider 1 ,
• Amar U. Kishan 2 &
• Abhishek A. Solanki   ORCID: orcid.org/0009-0007-2186-2987 1  

Intraprostatic recurrence (IRR) of prostate cancer after radiation therapy is increasingly identified. Our objective was to review the literature to determine the optimal workup for identifying IRR, the management options, and practical considerations for the delivery of re-irradiation as salvage local therapy.

We performed a systematic review of available publications and ongoing studies on the topics of IRR, with a focus on salvage re-irradiation.

Work up of biochemically recurrent prostate cancer includes PSMA PET/CT and multiparametric MRI, followed by biopsy to confirm IRR. Management options include continued surveillance, palliative hormonal therapy, and salvage local therapy. Salvage local therapy can be delivered using re-irradiation with low dose rate brachytherapy, high dose rate (HDR) brachytherapy, and stereotactic body radiotherapy (SBRT), as well as non-radiation modalities, such as cryotherapy, high-intensity focused ultrasound, irreversible electroporation and radical prostatectomy. Data demonstrate that HDR brachytherapy and SBRT have similar efficacy compared to the other salvage local therapy modalities, while having more favorable side effect profiles. Recommendations for radiation therapy planning and delivery using HDR and SBRT based on the available literature are discussed.

Salvage re-irradiation is safe and effective and should be considered in patients with IRR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price excludes VAT (USA) Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Salvage treatment for radio-recurrent prostate cancer: a review of literature with focus on recent advancements in image-guided focal salvage therapies

Salvage therapy options for local prostate cancer recurrence after primary radiotherapy: a literature review.

Prostate Cancer

Explore related subjects.

• Medical Imaging

Data availability

This study was a systematic review and analysis of existing clinical studies and thus no new data are available for sharing.

National Comprehensive Cancer Network (2011) NCCN Clinical practice guidelines in oncology prostate cancer. Version I. 2023:2023

Google Scholar  

Mahal BA, Butler S, Franco I, Spratt DE, Rebbeck TR, D’Amico AV et al (2019) Use of active surveillance or watchful waiting for low-risk prostate cancer and management trends across risk groups in the United States, 2010–2015. JAMA 321:704–706. https://doi.org/10.1001/jama.2018.19941

Article   PubMed   PubMed Central   Google Scholar  

Jones CU, Hunt D, McGowan DG, Amin MB, Chetner MP, Bruner DW et al (2011) Radiotherapy and short-term androgen deprivation for localized prostate cancer. N Engl J Med 365:107–118. https://doi.org/10.1056/NEJMoa1012348

Article   CAS   PubMed   Google Scholar  

Zumsteg ZS, Spratt DE, Romesser PB, Pei X, Zhang Z, Kollmeier M et al (2015) Anatomical patterns of recurrence following biochemical relapse in the dose escalation era of external beam radiotherapy for prostate cancer. J Urol 194:1624–1630. https://doi.org/10.1016/j.juro.2015.06.100

Ma TM, Chu F-I, Sandler H, Feng FY, Efstathiou JA, Jones CU et al (2022) Local failure events in prostate cancer treated with radiotherapy: a pooled analysis of 18 randomized trials from the meta-analysis of randomized trials in cancer of the prostate consortium (LEVIATHAN). Eur Urol 82:487–498. https://doi.org/10.1016/j.eururo.2022.07.011

Article   PubMed   Google Scholar  

Smith CP, Xiang M, Armstrong WR, Nickols NG, Steinberg ML, Reiter RE et al (2023) Patterns of failure in men with radiorecurrent prostate cancer: a post hoc analysis of 3 prospective gallium 68 prostate-specific membrane antigen positron emission tomography/computed tomography imaging trials. Int J Radiat Oncol Biol Phys S0360–3016(23):00181–00185. https://doi.org/10.1016/j.ijrobp.2023.02.039

Article   Google Scholar  

Maitre P, Sood S, Pathare P, Krishnatry R, Agrawal A, Rangarajan V et al (2022) Timing of Ga68-PSMA PETCT and patterns of recurrence after prostate radiotherapy: Implications for potential salvage. Radiother Oncol J Eur Soc Ther Radiol Oncol 169:71–76. https://doi.org/10.1016/j.radonc.2022.02.014

Article   CAS   Google Scholar  

Kishan AU, Chu F-I, King CR, Seiferheld W, Spratt DE, Tran P et al (2020) Local failure and survival after definitive radiotherapy for aggressive prostate cancer: an individual patient-level meta-analysis of six randomized trials. Eur Urol 77:201–208. https://doi.org/10.1016/j.eururo.2019.10.008

Xie W, Regan MM, Buyse M, Halabi S, Kantoff PW, Sartor O et al (2017) Metastasis-free survival is a strong surrogate of overall survival in localized prostate cancer. J Clin Oncol Off J Am Soc Clin Oncol 35:3097–3104. https://doi.org/10.1200/JCO.2017.73.9987

Özgür BC, Gültekin S, Ekici M, Yılmazer D, Alper M (2015) A narrowing range of bone scan in newly diagnosed prostate cancer patients: a retrospective comparative study. Urol Ann 7:193–198. https://doi.org/10.4103/0974-7796.150479

Valle LF, Greer MD, Shih JH, Barrett T, Law YM, Rosenkrantz AB et al (2018) Multiparametric MRI for the detection of local recurrence of prostate cancer in the setting of biochemical recurrence after low dose rate brachytherapy. Diagn Interv Radiol Ank Turk 24:46–53. https://doi.org/10.5152/dir.2018.17285

Morris MJ, Rowe SP, Gorin MA, Saperstein L, Pouliot F, Josephson D et al (2021) Diagnostic performance of 18F-DCFPyL-PET/CT in men with biochemically recurrent prostate cancer: results from the CONDOR phase III, multicenter study. Clin Cancer Res Off J Am Assoc Cancer Res 27:3674–3682. https://doi.org/10.1158/1078-0432.CCR-20-4573

Pienta KJ, Gorin MA, Rowe SP, Carroll PR, Pouliot F, Probst S et al (2021) A Phase 2/3 prospective multicenter study of the diagnostic accuracy of prostate specific membrane antigen PET/CT with 18F-DCFPyL in prostate cancer patients (OSPREY). J Urol 206:52–61. https://doi.org/10.1097/JU.0000000000001698

Fendler WP, Calais J, Eiber M, Flavell RR, Mishoe A, Feng FY et al (2019) Assessment of 68Ga-PSMA-11 PET accuracy in localizing recurrent prostate cancer: a prospective single-arm clinical trial. JAMA Oncol 5:856–863. https://doi.org/10.1001/jamaoncol.2019.0096

Raveenthiran S, Yaxley J, Gianduzzo T, Kua B, McEwan L, Wong D et al (2019) The use of 68Ga-PET/CT PSMA to determine patterns of disease for biochemically recurrent prostate cancer following primary radiotherapy. Prostate Cancer Prostatic Dis 22:385–390. https://doi.org/10.1038/s41391-019-0163-0

Jansen BHE, van Leeuwen PJ, Wondergem M, van der Sluis TM, Nieuwenhuijzen JA, Knol RJJ et al (2021) Detection of recurrent prostate cancer using prostate-specific membrane antigen positron emission tomography in patients not meeting the phoenix criteria for biochemical recurrence after curative radiotherapy. Eur Urol Oncol 4:821–825. https://doi.org/10.1016/j.euo.2020.01.002

Einspieler I, Rauscher I, Düwel C, Krönke M, Rischpler C, Habl G et al (2017) Detection efficacy of hybrid 68Ga-PSMA ligand PET/CT in prostate cancer patients with biochemical recurrence after primary radiation therapy defined by phoenix criteria. J Nucl Med Off Publ Soc Nucl Med 58:1081–1087. https://doi.org/10.2967/jnumed.116.184457

Meredith G, Wong D, Yaxley J, Coughlin G, Thompson L, Kua B et al (2016) The use of 68 Ga-PSMA PET CT in men with biochemical recurrence after definitive treatment of acinar prostate cancer. BJU Int 118(Suppl 3):49–55. https://doi.org/10.1111/bju.13616

Valle LF, Calais J, Marks LS, Raman S, Reiter RE, Rettig M et al (2023) Radiographic-pathologic concordance in the workup of locally radiorecurrent prostate cancer. J Clin Oncol 41:313–313. https://doi.org/10.1200/JCO.2023.41.6_suppl.313

Fendler WP, Calais J, Eiber M, Simko JP, Kurhanewicz J, Santos RD et al (2021) False positive PSMA PET for tumor remnants in the irradiated prostate and other interpretation pitfalls in a prospective multi-center trial. Eur J Nucl Med Mol Imaging 48:501–508. https://doi.org/10.1007/s00259-020-04945-1

Venkatesulu B, Yoo R, Block AM, Welsh JS, Farooq A, Gupta G et al (2022) Performance of Multiparametric MRI and PET imaging in identifying and localizing locally radiorecurrent prostate cancer in a phase I/II trial of focal salvage HDR. Int J Radiat Oncol 114:e249–e250. https://doi.org/10.1016/j.ijrobp.2022.07.1228

Trost LW, Serefoglu E, Gokce A, Linder BJ, Sartor AO, Hellstrom WJG (2013) Androgen deprivation therapy impact on quality of life and cardiovascular health, monitoring therapeutic replacement. J Sex Med 10(Suppl 1):84–101. https://doi.org/10.1111/jsm.12036

Valle LF, Lehrer EJ, Markovic D, Elashoff D, Levin-Epstein R, Karnes RJ et al (2021) A systematic review and meta-analysis of local salvage therapies after radiotherapy for prostate cancer (MASTER). Eur Urol 80:280–292. https://doi.org/10.1016/j.eururo.2020.11.010

Duchesne GM, Woo HH, Bassett JK, Bowe SJ, D’Este C, Frydenberg M et al (2016) Timing of androgen-deprivation therapy in patients with prostate cancer with a rising PSA (TROG 03.06 and VCOG PR 01–03 [TOAD]): a randomised, multicentre, non-blinded, phase 3 trial. Lancet Oncol 17:727–737. https://doi.org/10.1016/S1470-2045(16)00107-8

Crook JM, O’Callaghan CJ, Duncan G, Dearnaley DP, Higano CS, Horwitz EM et al (2012) Intermittent androgen suppression for rising PSA level after radiotherapy. N Engl J Med 367:895–903. https://doi.org/10.1056/NEJMoa1201546

Article   CAS   PubMed   PubMed Central   Google Scholar  

Baumann BC, Baumann JC, Christodouleas JP, Soffen E (2017) Salvage of locally recurrent prostate cancer after external beam radiation using reduced-dose brachytherapy with neoadjuvant plus adjuvant androgen deprivation. Brachytherapy 16:291–298. https://doi.org/10.1016/j.brachy.2016.12.011

Burri RJ, Stone NN, Unger P, Stock RG (2010) Long-term outcome and toxicity of salvage brachytherapy for local failure after initial radiotherapy for prostate cancer. Int J Radiat Oncol Biol Phys 77:1338–1344. https://doi.org/10.1016/j.ijrobp.2009.06.061

Crook JM, Zhang P, Pisansky TM, Trabulsi EJ, Amin MB, Bice W et al (2019) A prospective phase 2 trial of transperineal ultrasound-guided brachytherapy for locally recurrent prostate cancer after external beam radiation therapy (NRG Oncology/RTOG-0526). Int J Radiat Oncol Biol Phys 103:335–343. https://doi.org/10.1016/j.ijrobp.2018.09.039

Crook J, Rodgers JP, Pisansky TM, Trabulsi EJ, Amin MB, Bice W et al (2022) Salvage low-dose-rate prostate brachytherapy: clinical outcomes of a Phase 2 trial for local recurrence after external beam radiation therapy (NRG Oncology/RTOG 0526). Int J Radiat Oncol Biol Phys 112:1115–1122. https://doi.org/10.1016/j.ijrobp.2021.10.138

Yamada Y, Kollmeier MA, Pei X, Kan CC, Cohen GN, Donat SM et al (2014) A Phase II study of salvage high-dose-rate brachytherapy for the treatment of locally recurrent prostate cancer after definitive external beam radiotherapy. Brachytherapy 13:111–116. https://doi.org/10.1016/j.brachy.2013.11.005

Corkum MT, Morton G, Loblaw DA, Tseng C-L, Murgic J, Ravi A et al (2022) A prospective study of magnetic resonance imaging-guided focal salvage high-dose-rate brachytherapy for radiorecurrent prostate cancer: updated results of 30 patients. Pract Radiat Oncol 12:e531–e537. https://doi.org/10.1016/j.prro.2022.05.018

Bergamin S, Eade T, Kneebone A, Booth J, Hsiao E, Schembri GP et al (2020) Interim results of a prospective prostate-specific membrane antigen-directed focal stereotactic reirradiation trial for locally recurrent prostate cancer. Int J Radiat Oncol Biol Phys 108:1172–1178. https://doi.org/10.1016/j.ijrobp.2020.07.014

Pasquier D, Lacornerie T, Supiot S, Pommier P, Quivrin M, Simon J-M et al (2023) The safety and efficacy of salvage stereotactic radiation therapy in patients with intraprostatic tumor recurrence after previous external radiation therapy: phase 1 results from the GETUG-AFU 31 study. Eur Urol Oncol 6:399–405. https://doi.org/10.1016/j.euo.2023.01.009

Ekanger C, Helle SI, Reisæter L, Hysing LB, Kvåle R, Honoré A et al (2024) Salvage reirradiation for locally recurrent prostate cancer: results from a prospective study with 7.2 years of follow-up. J Clin Oncol Off J Am Soc Clin Oncol. 42:1934–1942. https://doi.org/10.1200/JCO.23.01391

Henríquez López I, González-San Segundo C, Vegas JO, Gutierrez C, Hervas A, Cabeza Rodriguez MÁ et al (2019) Salvage brachytherapy for locally-recurrent prostate cancer after radiation therapy: a comparison of efficacy and toxicity outcomes with high-dose rate and low-dose rate brachytherapy. Radiother Oncol J Eur Soc Ther Radiol Oncol 141:156–163. https://doi.org/10.1016/j.radonc.2019.09.006

Kollmeier MA, McBride S, Taggar A, Anderson E, Lin M, Pei X et al (2017) Salvage brachytherapy for recurrent prostate cancer after definitive radiation therapy: a comparison of low-dose-rate and high-dose-rate brachytherapy and the importance of prostate-specific antigen doubling time. Brachytherapy 16:1091–1098. https://doi.org/10.1016/j.brachy.2017.07.013

Wu SY, Wong AC, Shinohara K, Roach M, Cunha JAM, Valdes G et al (2021) Salvage high-dose-rate brachytherapy for recurrent prostate cancer after definitive radiation. Pract Radiat Oncol 11:515–526. https://doi.org/10.1016/j.prro.2021.04.007

Fuller D, Wurzer J, Shirazi R, Bridge S, Law J, Crabtree T et al (2020) Retreatment for local recurrence of prostatic carcinoma after prior therapeutic irradiation: efficacy and toxicity of HDR-like SBRT. Int J Radiat Oncol Biol Phys 106:291–299. https://doi.org/10.1016/j.ijrobp.2019.10.014

Pasquier D, Martinage G, Janoray G, Rojas DP, Zerini D, Goupy F et al (2019) Salvage stereotactic body radiation therapy for local prostate cancer recurrence after radiation therapy: a retrospective multicenter study of the GETUG. Int J Radiat Oncol Biol Phys 105:727–734. https://doi.org/10.1016/j.ijrobp.2019.07.012

van Son MJ, Peters M, Moerland MA, Lagendijk JJW, Eppinga WSC, Shah TT et al (2020) MRI-guided ultrafocal salvage high-dose-rate brachytherapy for localized radiorecurrent prostate cancer: updated results of 50 patients. Int J Radiat Oncol Biol Phys 107:126–135. https://doi.org/10.1016/j.ijrobp.2020.01.023

Kissel M, Pounou A, Ka K, Alexis A, Irani J, Jereczek-Fossa BA et al (2022) Efficacy and toxicity following salvage high-dose-rate brachytherapy for locally recurrent prostate cancer after radiotherapy. Brachytherapy 21:424–434. https://doi.org/10.1016/j.brachy.2022.01.005

Lewin R, Amit U, Laufer M, Berger R, Dotan Z, Domachevsky L et al (2021) Salvage re-irradiation using stereotactic body radiation therapy for locally recurrent prostate cancer: the impact of castration sensitivity on treatment outcomes. Radiat Oncol 16:114

Van den Broeck T, van den Bergh RCN, Briers E, Cornford P, Cumberbatch M, Tilki D et al (2020) Biochemical recurrence in prostate cancer: the european association of urology prostate cancer guidelines panel recommendations. Eur Urol Focus 6:231–234. https://doi.org/10.1016/j.euf.2019.06.004

Palma DA, Olson R, Harrow S, Gaede S, Louie AV, Haasbeek C et al (2019) Stereotactic ablative radiotherapy versus standard of care palliative treatment in patients with oligometastatic cancers (SABR-COMET): a randomised, phase 2, open-label trial. Lancet Lond Engl 393:2051–2058. https://doi.org/10.1016/S0140-6736(18)32487-5

Phillips R, Shi WY, Deek M, Radwan N, Lim SJ, Antonarakis ES et al (2020) Outcomes of observation vs stereotactic ablative radiation for oligometastatic prostate cancer: the oriole phase 2 randomized clinical trial. JAMA Oncol 6:650. https://doi.org/10.1001/jamaoncol.2020.0147

Ost P, Reynders D, Decaestecker K, Fonteyne V, Lumen N, De Bruycker A et al (2018) Surveillance or metastasis-directed therapy for oligometastatic prostate cancer recurrence: a prospective, randomized, multicenter phase II trial. J Clin Oncol Off J Am Soc Clin Oncol 36:446–453. https://doi.org/10.1200/JCO.2017.75.4853

Corkum MT, Buyyounouski MK, Chang AJ, Chung HT, Chung P, Cox BW et al (2023) Salvage prostate brachytherapy in radiorecurrent prostate cancer: an international Delphi consensus study. Radiother Oncol J Eur Soc Ther Radiol Oncol 184:109672. https://doi.org/10.1016/j.radonc.2023.109672

Jereczek-Fossa BA, Marvaso G, Zaffaroni M, Gugliandolo SG, Zerini D, Corso F et al (2021) Salvage stereotactic body radiotherapy (SBRT) for intraprostatic relapse after prostate cancer radiotherapy: An ESTRO ACROP Delphi consensus. Cancer Treat Rev 98:102206. https://doi.org/10.1016/j.ctrv.2021.102206

Zhong J, Brown S, Serra M, Shuttleworth P, Bownes P, Thompson C et al (2022) Reirradiation options for previously irradiated prostate cancer (RO-PIP): feasibility study investigating toxicity outcomes following reirradiation with stereotactic body radiotherapy (SBRT) versus high-dose-rate brachytherapy (HDR-BT). BMJ Open 12:e068580. https://doi.org/10.1136/bmjopen-2022-068580

Ménard C, Navarro-Domenech I, Liu ZA, Joseph L, Barkati M, Berlin A et al (2022) MRI-guided focal or integrated boost high dose rate brachytherapy for recurrent prostate cancer. Front Oncol 12:971344. https://doi.org/10.3389/fonc.2022.971344

Wojcieszek P, Szlag M, Głowacki G, Cholewka A, Gawkowska-Suwińska M, Kellas-Ślęczka S et al (2016) Salvage high-dose-rate brachytherapy for locally recurrent prostate cancer after primary radiotherapy failure. Radiother Oncol J Eur Soc Ther Radiol Oncol 119:405–410. https://doi.org/10.1016/j.radonc.2016.04.032

Morton G, McGuffin M, Chung HT, Tseng C-L, Helou J, Ravi A et al (2020) Prostate high dose-rate brachytherapy as monotherapy for low and intermediate risk prostate cancer: efficacy results from a randomized phase II clinical trial of one fraction of 19 Gy or two fractions of 13.5 Gy. Radiother Oncol J Eur Soc Ther Radiol Oncol. 146:90–96. https://doi.org/10.1016/j.radonc.2020.02.009

Rasing MJA, Peters M, van Son M, Moerland MA, Eppinga W, van de Pol SMG et al (2023) Recurrence characteristics after focal salvage HDR brachytherapy in prostate cancer. Radiother Oncol J Eur Soc Ther Radiol Oncol 180:109495. https://doi.org/10.1016/j.radonc.2023.109495

Chitmanee P, Tsang Y, Tharmalingam H, Hamada M, Alonzi R, Ostler P et al (2020) Single-dose focal salvage high dose rate brachytherapy for locally recurrent prostate cancer. Clin Oncol R Coll Radiol G B 32:259–265. https://doi.org/10.1016/j.clon.2019.10.008

Slevin F, Hodgson S, Rodda SL, Bownes P, Bottomley D, Adiotomre E et al (2020) Efficacy and toxicity outcomes for patients treated with focal salvage high dose rate brachytherapy for locally recurrent prostate cancer. Clin Transl Radiat Oncol 23:20–26. https://doi.org/10.1016/j.ctro.2020.03.010

Solanki AA, Yoo RK, Adams W, Davicioni E, Mysz ML, Shea S et al (2023) F-SHARP: a Phase I/II trial of focal salvage high-dose-rate brachytherapy for Radiorecurrent prostate cancer. BJU Int. https://doi.org/10.1111/bju.16150

Patel KR, Rydzewski NR, Schott E, Cooley-Zgela T, Ning H, Cheng J et al (2023) A phase 1 trial of focal salvage stereotactic body radiation therapy for radiorecurrent prostate cancer. Pract Radiat Oncol S1879–8500(23):00171–00176. https://doi.org/10.1016/j.prro.2023.05.012

Chen CP, Weinberg V, Shinohara K, Roach M, Nash M, Gottschalk A et al (2013) Salvage HDR brachytherapy for recurrent prostate cancer after previous definitive radiation therapy: 5 year outcomes. Int J Radiat Oncol Biol Phys 86:324–329. https://doi.org/10.1016/j.ijrobp.2013.01.027

Nguyen PL, Chen M-H, D’Amico AV, Tempany CM, Steele GS, Albert M et al (2007) Magnetic resonance image-guided salvage brachytherapy after radiation in select men who initially presented with favorable-risk prostate cancer: a prospective phase 2 study. Cancer 110:1485–1492. https://doi.org/10.1002/cncr.22934

Loening SA, Turner JW (1993) Use of percutaneous transperineal 198Au seeds to treat recurrent prostate adenocarcinoma after failure of definitive radiotherapy. Prostate 23:283–290. https://doi.org/10.1002/pros.2990230403

Peters M, van der Voort van Zyp JRN, Moerland MA, Hoekstra CJ, van de Pol S, Westendorp H, et al (2016) Multivariable model development and internal validation for prostate cancer specific survival and overall survival after whole-gland salvage Iodine-125 prostate brachytherapy. Radiother Oncol J Eur Soc Ther Radiol Oncol. 119:104–110. https://doi.org/10.1016/j.radonc.2016.02.002

Vargas C, Swartz D, Vashi A, Blasser M, Kasraeian A, Cesaretti J et al (2014) Salvage brachytherapy for recurrent prostate cancer. Brachytherapy 13:53–58. https://doi.org/10.1016/j.brachy.2013.10.012

Jereczek-Fossa BA, Rojas DP, Zerini D, Fodor C, Viola A, Fanetti G et al (2019) Reirradiation for isolated local recurrence of prostate cancer: mono-institutional series of 64 patients treated with salvage stereotactic body radiotherapy (SBRT). Br J Radiol 92:20180494. https://doi.org/10.1259/bjr.20180494

Francolini G, Loi M, Di Cataldo V, Detti B, Stocchi G, Masi L et al (2022) Stereotactic re-irradiation in recurrent prostate cancer after previous postoperative or definitive radiotherapy: long-term results after a median follow-up of 4 Years. Clin Oncol R Coll Radiol G B 34:50–56. https://doi.org/10.1016/j.clon.2021.11.002

D’Agostino GR, Di Brina L, Mancosu P, Franzese C, Iftode C, Franceschini D et al (2019) Reirradiation of locally recurrent prostate cancer with volumetric modulated arc therapy. Int J Radiat Oncol Biol Phys 104:614–621. https://doi.org/10.1016/j.ijrobp.2019.02.041

Perennec T, Vaugier L, Toledano A, Scher N, Thomin A, Pointreau Y et al (2021) Stereotactic re-irradiation for local recurrence after radical prostatectomy and radiation therapy: a retrospective multicenter study. Cancers 13:4339. https://doi.org/10.3390/cancers13174339

Archer P, Marvaso G, Detti B, Colombo F, Francolini G, Vandendorpe B et al (2023) Salvage stereotactic reirradiation for local recurrence in the prostatic bed after prostatectomy: a retrospective multicenter study. Eur Urol Oncol 6:303–310. https://doi.org/10.1016/j.euo.2023.03.005

Miszczyk L, Stąpór-Fudzińska M, Miszczyk M, Maciejewski B, Tukiendorf A (2018) Salvage cyberknife-based reirradiation of patients with recurrent prostate cancer: the single-center experience. Technol Cancer Res Treat 17:1533033818785496. https://doi.org/10.1177/1533033818785496

Download references

The authors did not receive support from any organization for the submitted work.

Author information

Authors and affiliations.

Department of Radiation Oncology, Stritch School of Medicine, Cardinal Bernardin Cancer Center, Loyola University Chicago, Maywood, IL, USA

Anjali L. Saripalli, Bhanu Prasad Venkatesulu, Matthew M. Harkenrider & Abhishek A. Solanki

Department of Radiation Oncology, University of California Los Angeles, Los Angeles, CA, USA

Nicholas G. Nickols, Luca F. Valle & Amar U. Kishan

Department of Radiation Oncology, Greater Los Angeles Veterans Affairs Healthcare System, Los Angeles, CA, USA

Nicholas G. Nickols & Luca F. Valle

You can also search for this author in PubMed   Google Scholar

Contributions

AL Saripalli: Data collection, Data management, Manuscript writing/editing, BP Venkatesulu: contributed to systematic review, NG Nickols: Manuscript writing/editing, LF Valle: Manuscript writing/editing, MM Harkenrider: Manuscript writing/editing, AU Kishan: Project development, Manuscript writing/editing, AA Solanki: Project development, Manuscript writing/editing

Corresponding author

Correspondence to Abhishek A. Solanki .

Ethics declarations

Conflict of interest.

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 67 KB)

Rights and permissions.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Saripalli, A.L., Venkatesulu, B.P., Nickols, N.G. et al. Systematic review and recommendations for re-irradiation for intraprostatic radiorecurrent prostate cancer after definitive radiation therapy. World J Urol 42 , 520 (2024). https://doi.org/10.1007/s00345-024-05205-9

Download citation

Received : 09 October 2023

Accepted : 17 July 2024

Published : 12 September 2024

DOI : https://doi.org/10.1007/s00345-024-05205-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Prostate cancer
• Radiorecurrence
• Salvage local therapy
• Re-irradiation
• High dose rate brachytherapy
• Stereotactic body radiotherapy
• Find a journal
• Publish with us
• Track your research