critically reflect on the research process in qualitative research

• - Google Chrome

Intended for healthcare professionals

• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Critically appraising...

Critically appraising qualitative research

• Related content
• Peer review
• Ayelet Kuper , assistant professor 1 ,
• Lorelei Lingard , associate professor 2 ,
• Wendy Levinson , Sir John and Lady Eaton professor and chair 3
• 1 Department of Medicine, Sunnybrook Health Sciences Centre, and Wilson Centre for Research in Education, University of Toronto, 2075 Bayview Avenue, Room HG 08, Toronto, ON, Canada M4N 3M5
• 2 Department of Paediatrics and Wilson Centre for Research in Education, University of Toronto and SickKids Learning Institute; BMO Financial Group Professor in Health Professions Education Research, University Health Network, 200 Elizabeth Street, Eaton South 1-565, Toronto
• 3 Department of Medicine, Sunnybrook Health Sciences Centre
• Correspondence to: A Kuper ayelet94{at}post.harvard.edu

Six key questions will help readers to assess qualitative research

Summary points

Appraising qualitative research is different from appraising quantitative research

Qualitative research papers should show appropriate sampling, data collection, and data analysis

Transferability of qualitative research depends on context and may be enhanced by using theory

Ethics in qualitative research goes beyond review boards’ requirements to involve complex issues of confidentiality, reflexivity, and power

Over the past decade, readers of medical journals have gained skills in critically appraising studies to determine whether the results can be trusted and applied to their own practice settings. Criteria have been designed to assess studies that use quantitative methods, and these are now in common use.

In this article we offer guidance for readers on how to assess a study that uses qualitative research methods by providing six key questions to ask when reading qualitative research (box 1). However, the thorough assessment of qualitative research is an interpretive act and requires informed reflective thought rather than the simple application of a scoring system.

Box 1 Key questions to ask when reading qualitative research studies

Was the sample used in the study appropriate to its research question.

Were the data collected appropriately?

Were the data analysed appropriately?

Can I transfer the results of this study to my own setting?

Does the study adequately address potential ethical issues, including reflexivity?

Overall: is what the researchers did clear?

One of the critical decisions in a qualitative study is whom or what to include in the sample—whom to interview, whom to observe, what texts to analyse. An understanding that qualitative research is based in experience and in the construction of meaning, combined with the specific research question, should guide the sampling process. For example, a study of the experience of survivors of domestic violence that examined their reasons for not seeking help from healthcare providers might focus on interviewing a …

Log in using your username and password

BMA Member Log In

If you have a subscription to The BMJ, log in:

• Need to activate
• Log in via institution
• Log in via OpenAthens

Log in through your institution

Subscribe from £184 *.

Subscribe and get access to all BMJ articles, and much more.

* For online subscription

Access this article for 1 day for: £50 / $60/ €56 ( excludes VAT )

You can download a PDF version for your personal record.

Buy this article

Developing Critical Reflection as a Research Method

Cite this chapter.

• Jan Fook PhD 6  

Part of the book series: Practice, Education, Work and Society ((PEWS,volume 5))

2726 Accesses

48 Citations

Critical reflection is normally used in professional learning settings to assist practitioners to improve practice. I have worked for some time using critical reflection in this way with many different types of professionals. Over time, however, I have been impressed by the deeper and more complex understanding of practice experience which the process enables, and which practitioners themselves often cannot initially express. And so I have begun to speculate about the research potential of the critical reflection process, and whether it might be developed as a research method to allow better formulations of practice experience, and therefore, ultimately, better practice.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Unable to display preview.  Download preview PDF.

Similar content being viewed by others

Critical Reflection

Is reflection like soap? a critical narrative umbrella review of approaches to reflection in medical education research

Developing Reflective Practice

Alvesson, M., & Skoldberg, K. (2000). Reflexive methodology . London: Sage.

Google Scholar  

Bradley, B. (2005). Psychology and experience . Cambridge: Cambridge University Press.

Brookfield, S. (2009). The concept of critical reflection: Promises and contradictions. European Journal of Social Work , 12 (3), 293–304.

Article   Google Scholar  

Brown, W. (1995). States of injury. Princeton: Princeton University Press.

Denzin, N., & Lincoln, Y. (Eds.) (1994). Handbook of qualitative research. London: Sage.

Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the educative process . New York: D. C. Heath & Co.

Dilthey, W. (1961). Pattern and meaning in history . New York: Harper Torchbooks.

Elliot, J. (2005). Using narrative in social research . London: Sage.

Fook, J. (2002). Social work: Critical theory and practice . London: Sage.

Fook, J., & Askeland, G.A. (2006). The “critical” in critical reflection. In S. White, J. Fook & F. Gardner (Eds.), C ritical reflection in health and social care (pp. 40–53). Maidenhead: Open University Press.

Fook, J., & Gardner, F. (2007). Practising critical reflection . Maidenhead: Open University Press.

Green, B. (Ed.) (2009). Understanding and researching professional practice. Rotterdam: Sense.

Haug, F. (1987). Female sexualisation: A collective work of memory. London: Verso.

Heaney, T.W., & Horton, A.I. (1991). Reflective engagement for social change. In J. Mezirow (Ed.), Fostering critical reflection in adulthood (pp. 74–98). San Francisco: Jossey-Bass.

Merrill, B., & West, L. (2009). Using biographical methods in social research . London: Sage.

Reason, P., & Bradbury, H. (Eds.) (2001). Handbook of action research: Participative inquiry and practice . London: Sage.

Schön, D.A. (1983). The reflective practitioner: How professionals think in action . New York: Basic Books.

Steier, F. (Ed.) (1991). Research and reflexivity. London: Sage.

Stephenson, N., & Papadopoulos, D. (2006). Analysing everyday experience . Basingstoke: Palgrave Macmillan.

Taylor, C., & White, S. (2000). Practising reflexivity in health and welfare . Maidenhead: Open University Press.

Turner, V.W., & Bruner, E.M. (1986). The anthropology of experience . Chicago: Uni. of Illinois Press.

White, S. (2001). Autoethnography as reflexive inquiry: The research act as self-surveillance. In I. Shaw & N. Gould (Eds.), Qualitative research in social work (pp. 100–115). London: Sage.

Download references

Author information

Authors and affiliations.

Interprofessional Institute, South West London Academic Network, UK

Jan Fook PhD

You can also search for this author in PubMed   Google Scholar

Editor information

Editors and affiliations.

Charles Sturt University, Australia

Independent Research & Practice Development Consultant, UK

Angie Titchen

University of Western Sydney, Australia

Debbie Horsfall

Donna Bridges

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Sense Publishers

About this chapter

Fook, J. (2011). Developing Critical Reflection as a Research Method. In: Higgs, J., Titchen, A., Horsfall, D., Bridges, D. (eds) Creative Spaces for Qualitative Researching. Practice, Education, Work and Society, vol 5. SensePublishers. https://doi.org/10.1007/978-94-6091-761-5_6

Download citation

DOI : https://doi.org/10.1007/978-94-6091-761-5_6

Publisher Name : SensePublishers

Online ISBN : 978-94-6091-761-5

eBook Packages : Humanities, Social Sciences and Law Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

Societies and partnerships

• Find a journal
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Am J Pharm Educ
• v.84(1); 2020 Jan

A Review of the Quality Indicators of Rigor in Qualitative Research

Jessica l. johnson.

a William Carey University School of Pharmacy, Biloxi, Mississippi

Donna Adkins

Sheila chauvin.

b Louisiana State University, School of Medicine, New Orleans, Louisiana

Attributes of rigor and quality and suggested best practices for qualitative research design as they relate to the steps of designing, conducting, and reporting qualitative research in health professions educational scholarship are presented. A research question must be clear and focused and supported by a strong conceptual framework, both of which contribute to the selection of appropriate research methods that enhance trustworthiness and minimize researcher bias inherent in qualitative methodologies. Qualitative data collection and analyses are often modified through an iterative approach to answering the research question. Researcher reflexivity, essentially a researcher’s insight into their own biases and rationale for decision-making as the study progresses, is critical to rigor. This article reviews common standards of rigor, quality scholarship criteria, and best practices for qualitative research from design through dissemination.

INTRODUCTION

Within the past 20 years, qualitative research in health professions education has increased significantly, both in practice and publication. Today, one can pick up most any issue of a wide variety of health professions education journals and find at least one article that includes some type of qualitative research, whether a full study or the inclusion of a qualitative component within a quantitative or mixed methods study. Simultaneously, there have been recurrent calls for enhancing rigor and quality in qualitative research.

As members of the academic community, we share responsibility for ensuring rigor in qualitative research, whether as researchers who design and implement, manuscript reviewers who critique, colleagues who discuss and learn from each other, or scholarly teachers who draw upon results to enhance and innovate education. Therefore, the purpose of this article is to summarize standards of rigor and suggested best practices for designing, conducting, and reporting high-quality qualitative research. To begin, Denzin and Lincoln’s definition of qualitative research, a long-standing cornerstone in the field, provides a useful foundation for summarizing quality standards and best practices:

Qualitative research involves the studied use and collection of a variety of empirical materials – case study; personal experience; introspection; life story; interview; artifacts; cultural texts and productions; observational, historical, interactional, and visual texts – that describe the routine and problematic moments and meanings in individual lives. Accordingly, qualitative researchers deploy a wide range of interconnected interpretative practices, hoping always to get a better understanding of the subject matter at hand. It is understood, however, that each practice makes the world visible in a different way. Hence there is frequently a commitment to using more than one interpretative practice in any study. 1

In recent years, multiple publications have synthesized quality criteria and recommendations for use by researchers and peer reviewers alike, often in the form of checklists. 2-6 Some authors have raised concerns about the use of such checklists and adherence to strict, universal criteria because they do not afford sufficient flexibility to accommodate the diverse approaches and multiple interpretive practices often represented in qualitative studies. 7-11 They argue that a strict focus on using checklists of specific technical criteria may stifle the diversity and multiplicity of practices that are so much a part of achieving quality and rigor within the qualitative paradigm. As an alternative, some of these authors have published best practice guidelines for use by researchers and peer reviewers to achieve and assess methodological rigor and research quality. 12,13

Some journals within the field of health professions education have also established best practice guidance, as opposed to strict criteria or a checklist, for qualitative research. These have been disseminated as guiding questions or evaluation categories. In 2015, Academic Medicine produced an expanded second edition of a researcher/author manual that includes specific criteria with extensive explanations and examples. 14 Still others have disseminated best practice guidelines through a series of methodological articles within journal publications. 2

In this article, attributes of rigor and quality and suggested best practices are presented as they relate to the steps of designing, conducting, and reporting qualitative research in a step-wise approach.

BEST PRACTICES: STEP-WISE APPROACH

Step 1: identifying a research topic.

Identifying and developing a research topic is comprised of two major tasks: formulating a research question, and developing a conceptual framework to support the study. Formulating a research question is often stimulated by real-life observations, experiences, or events in the researcher’s local setting that reflect a perplexing problem begging for systematic inquiry. The research question begins as a problem statement or set of propositions that describe the relationship among certain concepts, behaviors, or experiences. Agee 15 and others 16,17 note that initial questions are usually too broad in focus and too vague regarding the specific context of the study to be answerable and researchable. Creswell reminds us that initial qualitative research questions guide inquiry, but they often change as the author’s understanding of the issue develops throughout the study. 16 Developing and refining a primary research question focused on both the phenomena of interest and the context in which it is situated is essential to research rigor and quality.

Glassick, Huber, and Maeroff identified six criteria applicable to assessing the quality of scholarship. 18,19 Now commonly referred to as the Glassick Criteria ( Table 1 ), these critical attributes outline the essential elements of any scholarly approach and serve as a general research framework for developing research questions and designing studies. The first two criteria, clear purpose and adequate preparation, are directly related to formulating effective research questions and a strong conceptual framework.

Glassick’s Criteria for Assessing the Quality of Scholarship of a Research Study 18

Generating and refining a qualitative research question requires thorough, systematic, and iterative review of the literature, and the use of those results to establish a clear context and foundation for the question and study design. Using an iterative approach, relevant concepts, principles, theories or models, and prior evidence are identified to establish what is known, and more importantly, what is not known. The iterative process contributes to forming a better research question, the criteria for which can be abbreviated by the acronym FINER, ie, f easible, i nteresting, n ovel, e thical, and r elevant, that is answerable and researchable, in terms of research focus, context specificity, and the availability of time, logistics, and resources to carry out the study. Developing a FINER research question is critical to study rigor and quality and should not be rushed, as all other aspects of research design depend on the focus and clarity of the research question(s) guiding the study. 15 Agee provides clear and worthwhile additional guidance for developing qualitative research questions. 15

Reflexivity, the idea that a researcher’s preconceptions and biases can influence decisions and actions throughout qualitative research activities, is a critical aspect of rigor even at the earliest stages of the study. A researcher’s background, beliefs, and experiences may affect any aspect of the research from choosing which specific question to investigate through determining how to present the results. Therefore, even at this early stage, the potential effect of researcher bias and any ethical considerations should be acknowledged and addressed. That is, how will the question’s influence on study design affect participants’ lives, position the researcher in relationship with others, or require specific methods for addressing potential areas of research bias and ethical considerations?

A conceptual framework is then actively constructed to provide a logical and convincing argument for the research. The framework defines and justifies the research question, the methodology selected to answer that question, and the perspectives from which interpretation of results and conclusions will be made. 5,6,20 Developing a well-integrated conceptual framework is essential to establishing a research topic based upon a thorough and integrated review of relevant literature (addressing Glassick criteria #1 and #2: clear purpose and adequate preparation). Key concepts, principles, assumptions, best practices, and theories are identified, defined, and integrated in ways that clearly demonstrate the problem statement and corresponding research question are answerable, researchable, and important to advancing thinking and practice.

Ringsted, Hodges, and Sherpbier describe three essential parts to an effective conceptual framework: theories and/or concepts and principles relevant to the phenomenon of interest; what is known and unknown from prior work, observations, and examples; and the researcher’s observations, ideas, and suppositions regarding the research problem statement and question. 21 Lingard describes four types of unknowns to pursue during literature review: what no one knows; what is not yet well understood; what controversy or conflicting results, understandings, or perspectives exist; and what are unproven assumptions. 22 In qualitative research, these unknowns are critical to achieving a well-developed conceptual framework and a corresponding rigorous study design.

Recent contributions from Ravitch and colleagues present best practices in developing frameworks for conceptual and methodological coherence within a study design, regardless of the research approach. 23,24 Their recommendations and arguments are highly relevant to qualitative research. Figure 1 reflects the primary components of a conceptual framework adapted from Ravitch and Carl 23 and how all components contribute to decisions regarding research design, implementation, and applications of results to future thinking, study, and practice. Notice that each element of the framework interacts with and influences other elements in a dynamic and interactive process from the beginning to the end of a research project. The intersecting bidirectional arrows represent direct relationships between elements as they relate to specific aspects of a qualitative research study.

Adaptation of Ravitch and Carl’s Components of a Conceptual Framework 23

Maxwell also provides useful guidance for developing an effective conceptual framework specific to the qualitative research paradigm. 17 The 2015 second edition of the Review Criteria for Research Manuscripts 14 and work by Ravitch and colleagues 23,24 provide specific guidance for applying the conceptual framework to each stage of the research process to enhance rigor and quality. Quality criteria for assessing a study’s problem statement, conceptual framework, and research question include the following: introduction builds a logical case and provides context for the problem statement; problem statement is clear and well-articulated; conceptual framework is explicit and justified; research purpose and/or question is clearly stated; and constructs being investigated are clearly identified and presented. 14,24,25 As best practice guidelines, these criteria facilitate quality and rigor while providing sufficient flexibility in how each is achieved and demonstrated.

While a conceptual framework is important to rigor in qualitative research, Huberman and Miles caution qualitative researchers about developing and using a framework to the extent that it influences qualitative design deductively because this would violate the very principles of induction that define the qualitative research paradigm. 25 Our profession’s recent emphasis on a holistic admissions process for pharmacy students provides a reasonable example of inductive and deductive reasoning and their respective applications in qualitative and quantitative research studies. Principles of inductive reasoning are applied when a qualitative research study examines a representative group of competent pharmacy professionals to generate a theory about essential cognitive and affective skills for patient-centered care. Deductive reasoning could then be applied to design a hypothesis-driven prospective study that compares the outcomes of two cohorts of students, one group admitted using traditional criteria and one admitted based on a holistic admissions process revised to value the affective skills of applicants. Essentially, the qualitative researcher must carefully generate a conceptual framework that guides the research question and study design without allowing the conceptual framework to become so rigid as to dictate a testable hypothesis, which is the founding principle of deductive reasoning. 26

Step 2: Qualitative Study Design

The development of a strong conceptual framework facilitates selection of appropriate study methods to minimize the bias inherent in qualitative studies and help readers to trust the research and the researcher (see Glassick criteria #3 in Table 1 ). Although researchers can employ great flexibility in the selection of study methods, inclusion of best practice methods for assuring the rigor and trustworthiness of results is critical to study design. Lincoln and Guba outline four criteria for establishing the overall trustworthiness of qualitative research results: credibility, the researcher ensures and imparts to the reader supporting evidence that the results accurately represent what was studied; transferability, the researcher provides detailed contextual information such that readers can determine whether the results are applicable to their or other situations; dependability, the researcher describes the study process in sufficient detail that the work could be repeated; confirmability, the researcher ensures and communicates to the reader that the results are based on and reflective of the information gathered from the participants and not the interpretations or bias of the researcher. 27

Specific best practice methods used in the sampling and data collection processes to increase the rigor and trustworthiness of qualitative research include: clear rationale for sampling design decisions, determination of data saturation, ethics in research design, member checking, prolonged engagement with and persistent observation of study participants, and triangulation of data sources. 28

Qualitative research is focused on making sense of lived, observed phenomenon in a specific context with specifically selected individuals, rather than attempting to generalize from sample to population. Therefore, sampling design in qualitative research is not random but defined purposively to include the most appropriate participants in the most appropriate context for answering the research question. Qualitative researchers recognize that certain participants are more likely to be “rich” with data or insight than others, and therefore, more relevant and useful in achieving the research purpose and answering the question at hand. The conceptual framework contributes directly to determining sample definitions, size, and recruitment of participants. A typical best practice is purposive sampling methods, and when appropriate, convenience sampling may be justified. 29

Purposive sampling reflects intentional selection of research participants to optimize data sources for answering the research question. For example, the research question may be best answered by persons who have particular experience (critical case sampling) or certain expertise (key informant sampling). Similarly, additional participants may be referred for participation by active participants (snowball sampling) or may be selected to represent either similar or opposing viewpoints (confirming or disconfirming samples). Again, the process of developing and using a strong conceptual framework to guide and justify methodological decisions, in this case defining and establishing the study sample, is critical to rigor and quality. 30 Convenience sampling, using the most accessible research participants, is the least rigorous approach to defining a study sample and may result in low accuracy, poor representativeness, low credibility, and lack of transferability of study results.

Qualitative studies typically reflect designs in which data collection and analysis are done concurrently, with results of ongoing analysis informing continuing data collection. Determination of a final sample size is largely based on having sufficient opportunity to collect relevant data until new information is no longer emerging from data collection, new coding is not feasible, and/or no new themes are emerging; that is, reaching data saturation , a common standard of rigor for data collection in qualitative studies . Thus, accurately predicting a sample size during the planning phases of qualitative research can be challenging. 30 Care should be taken that sufficient quantity (think thick description) and quality (think rich description) of data have been collected prior to concluding that data saturation has been achieved. A poor decision regarding sample size is a direct consequence of sampling strategy and quality of data generated, which leaves the researcher unable to fully answer the research question in sufficient depth. 30

Though data saturation is probably the most common terminology used to describe the achievement of sufficient sample size, it does not apply to all study designs. For example, one could argue that in some approaches to qualitative research, data collection could continue infinitely if the event continues infinitely. In education, we often anecdotally observe variations in the personality and structure of a class of students, and as generations of students continue to evolve with time, so too would the data generated from observing each successive class. In such situations, data saturation might never be achieved. Conversely, the number of participants available for inclusion in a sample may be small and some risk of not reaching data saturation may be unavoidable. Thus, the idea of fully achieving data saturation may be unrealistic when applied to some populations or research questions. In other instances, attrition and factors related to time and resources may contribute to not reaching data saturation within the limits of the study. By being transparent in the process and reporting of results when saturation may not have been possible, the resulting data may still contribute to the field and to further inquiry. Replication of the study using other samples and conducting additional types of follow-up studies are other options for better understanding the research phenomenon at hand. 31

In addition to defining the sample and selecting participants, other considerations related to sampling bias may impact the quantity and quality of data generated and therefore the quality of the study result. These include: methods of recruiting, procedures for informed consent, timing of the interviews in relation to experience or emotion, procedures for ensuring participant anonymity/confidentiality, interview setting, and methods of recording/transcribing the data. Any of these factors could potentially change the nature of the relationship between the researcher and the study participants and influence the trustworthiness of data collected or the study result. Thus, ongoing application of previously mentioned researcher reflexivity is critical to the rigor of the study and quality of sampling. 29,30

Common qualitative data collection methods used in health professions education include interview, direct observation methods, and textual/document analysis. Given the unique and often highly sensitive nature of data being collected by the researcher, trustworthiness is an essential component of the researcher-participant relationship. Ethical conduct refers to how moral principles and values are part of the research process. Participants’ perceptions of ethical conduct are fundamental to a relationship likely to generate high quality data. During each step of the research process, care must be taken to protect the confidentiality of participants and shield them from harm relating to issues of respect and dignity. Researchers must be respectful of the participants’ contributions and quotes, and results must be reported truthfully and honestly. 8

Interview methods range from highly structured to increase dependability or completely open-ended to allow for interviewers to clarify a participant’s response for increased credibility and confirmability. Regardless, interview protocols and structure are often modified or refined, based on concurrent data collection and analysis processes to support or refute preliminary interpretations and refine focus and continuing inquiry. Researcher reflexivity, or acknowledgement of researcher bias, is absolutely critical to the credibility and trustworthiness of data collection and analysis in such study designs. 32

Interviews should be recorded and transcribed verbatim prior to coding and analysis. 28 Member checking, a common standard of rigor, is a practice to increase study credibility and confirmability that involves asking a research subject to verify the transcription of an interview. 1,16,28 The research subject is asked to verify the completeness and accuracy of an interview transcript to ensure the transcript truthfully reflects the meaning and intent of the subject’s contribution.

Prolonged engagement involves the researcher gaining familiarity and understanding of the culture and context surrounding the persons or situations being studied. This strategy supports reflexivity, allowing the researcher to determine how they themselves may be a source of bias during the data collection process by altering the nature of how individuals behave or interact with others in the presence of the researcher. Facial expressions, spoken language, body language, style of dress, age, race, gender, social status, culture, and the researcher’s relationship with the participants may potentially influence either participants’ responses or how the researcher interprets those responses. 33 “Fitting in” by demonstrating an appreciation and understanding of the cultural norms of the population being studied potentially allows the researcher to obtain more open and honest responses from participants. However, if the research participants or topic are too familiar or personal, this may also influence data collection or analysis and interpretation of the results. 33 The possible applications of this section to faculty research with student participants in the context of pharmacy education are obvious, and researcher reflexivity is critical to rigor.

Some researchers using observational methods adopt a strategy of direct field observation, while others play partial or full participant roles in the activity being observed. In both observation scenarios, it is impossible to separate the researcher from the environment, and researcher reflexivity is essential. The pros and cons of observation approach, relative to the research question and study purpose, should be evaluated by the researcher, and the justification for the observational strategy selected should be made clear. 34 Regardless of the researcher’s degree of visibility to the study participants, persistent observation of the targeted sample is critical to the confirmability standard and to achieving data saturation. That is, study conclusions must be clearly grounded in persistent phenomena witnessed during the study, rather than on a fluke event. 28

Researchers acknowledge that observational methodologies are limited by the reality that the researcher carries a bias in determining what is observed, what is recorded, how it is recorded, and how it is transcribed for analysis. A study’s conceptual framework is critical to achieving rigor and quality and provides guidance in developing predetermined notions or plans for what to observe, how to record, and how to minimize the influence of potential bias. 34 Researcher notes should be recorded as soon as possible after the observation event to optimize accuracy. The more detailed and complete the notes, the more accurate and useful they can be in data analysis or in auditing processes for enhancing rigor in the interpretation phase of the study. 34

Triangulation is among the common standards of rigor applied within the qualitative research paradigm. Data triangulation is used to identify convergence of data obtained through multiple data sources and methods (eg, observation field notes and interview transcripts) to avoid or minimize error or bias and optimize accuracy in data collection and analysis processes. 33,35,36

Again, researcher practice in reflexivity throughout research processes is integral to rigor in study design and implementation. Researchers must demonstrate attention to appropriate methods and reflective critique, which are represented in both core elements of the conceptual framework ( Figure 1 ) and Glassick criteria ( Table 1 ). In so doing, the researcher will be well-prepared to justify sampling design and data collection decisions to manuscript reviewers and, ultimately, readers.

Step 3: Data Analysis

In many qualitative studies, data collection runs concurrently with data analysis. Specific standards of rigor are commonly used to ensure trustworthiness and integrity within the data analysis process, including use of computer software, peer review, audit trail, triangulation, and negative case analysis.

Management and analyses of qualitative data from written text, observational field notes, and interview transcriptions may be accomplished using manual methods or the assistance of computer software applications for coding and analysis. When managing very large data sets or complex study designs, computer software can be very helpful to assist researchers in coding, sorting, organizing, and weighting data elements. Software applications can facilitate ease in calculating semi-quantitative descriptive statistics, such as counts of specific events, that can be used as evidence that the researcher’s analysis is based on a representative majority of data collected ( inclusivism ) rather than focusing on selected rarities ( anecdotalism ). Using software to code data can also make it easier to identify deviant cases, detect coding errors, and estimate interrater reliability among multiple coders. 37 While such software helps to manage data, the actual analyses and interpretation still reside with the researcher.

Peer review, another common standard of rigor, is a process by which researchers invite an independent third-party researcher to analyze a detailed audit trail maintained by the study author. The audit trail methodically describes the step-by-step processes and decision-making throughout the study. Review of this audit trail occurs prior to manuscript development and enhances study confirmability. 1,16 The peer reviewer offers a critique of the study methods and validation of the conclusions drawn by the author as a thorough check on researcher bias.

Triangulation also plays a role in data analysis, as the term can also be used to describe how multiple sources of data can be used to confirm or refute interpretation, assertions, themes, and study conclusions. If a theme or theory can be arrived at and validated using multiple sources of data, the result of the study has greater credibility and confirmability. 16,33,36 Should any competing or controversial theories emerge during data collection or analysis, it is vital to the credibility and trustworthiness of the study that the author disclose and explore those negative cases. Negative case analysis refers to actively seeking out and scrutinizing data that do not fit or support the researcher’s interpretation of the data. 16

The use of best practices applying to data collection and data analysis facilitates the full examination of data relative to the study purpose and research question and helps to prevent premature closure of the study. Rather than stopping at the initial identification of literal, first-level assertion statements and themes, authors must progress to interpreting how results relate to, revise, or expand the conceptual framework, or offer an improved theory or model for explaining the study phenomenon of interest. Closing the loop on data collection is critical and is achieved when thorough and valid analysis can be linked back to the conceptual framework, as addressed in the next section.

Step 4: Drawing Valid Conclusions

Lingard and Kennedy 38 succinctly state that the purpose of qualitative research is to deepen one’s understanding of specific perspectives, observations, experiences, or events evidenced through the behaviors or products of individuals and groups as they are situated in specific contexts or circumstances. Conclusions generated from study results should enhance the conceptual framework, or contribute to a new theory or model development, and are most often situated within the discussion and conclusion sections of a manuscript.

The discussion section should include interpretation of the results and recommendations for practice. Interpretations should go beyond first-level results or literal description of observed behaviors, patterns, and themes from analysis. The author’s challenge is to provide a complete and thorough examination and explanation of how specific results relate to each other, contribute to answering the research question, and achieve the primary purpose of the research endeavor. The discussion should “close the loop” by integrating study results and analysis with the original conceptual framework. The discussion section should also provide a parsimonious narrative or graphical explanation and interpretation of study results that enhances understanding of the targeted phenomena.

The conclusion section should provide an overall picture or synopsis of the study, including its important and unique contributions to the field from the perspective of both conceptual and practical significance. The conclusion should also include personal and theoretical perspectives and future directions for research. Together, the discussion and conclusion should include responses to the larger questions of the study’s contributions, such as: So what? Why do these results matter? What next?

The strength of conclusions is dependent upon the extent to which standards of rigor and best practices were demonstrated in design, data collection, data analysis, and interpretation, as described in previous sections of this article. 4,12,17,23,24 Quality and rigor expectations for drawing valid conclusions and generating new theories are reflected in the following essential features of rigor and quality, which include: “Close the loop” to clearly link research questions, study design, data collection and analysis, and interpretation of results. Reflect effective integration of the study results with the conceptual framework and explain results in ways that relate, support, elaborate, and/or challenge conclusions of prior scholarship. Descriptions of new or enhanced frameworks or models are clear and effectively grounded in the study results and conclusions. Practical or theoretical implications are effectively discussed, including guidance for future studies. Limitations and issues of reflexivity and ethics are clearly and explicitly described, including references to actions taken to address these areas. 3,4,12,14

Step 5: Reporting Research Results

Key to quality reporting of qualitative research results are clarity, organization, completeness, accuracy, and conciseness in communicating the results to the reader of the research manuscript. O’Brien and others 4 proposed a standardized framework specifically for reporting qualitative studies known as the Standards for Reporting Qualitative Research (SRQR, Table 2 ). This framework provides detailed explanations of what should be reported in each of 21 sections of a qualitative research manuscript. While the SRQR does not explicitly mention a conceptual framework, the descriptions and table footnote clarification for the introduction and problem statement reflect the essential elements and focus of a conceptual framework. Ultimately, readers of published work determine levels of credibility, trustworthiness, and the like. A manuscript reviewer, the first reader of a study report, has the responsibility and privilege of providing critique and guidance to authors regarding achievement of quality criteria, execution and reporting of standards of rigor, and the extent to which meaningful contributions to thinking and practice in the field are presented. 13,39

An Adaptation of the 21 Elements of O’Brien and Colleagues’ Standards for Reporting Qualitative Research (SRQR) 4

Authors must avoid language heavy with connotations or adjectives that insert the researcher’s opinion into the database or manuscript. 14,40 The researcher should be as neutral and objective as possible in interpreting data and in presenting results. Thick and rich descriptions, where robust descriptive language is used to provide sufficient contextual information, enable the reader to determine credibility, transferability, dependability, and confirmability .

The process of demonstrating the credibility of research is rooted in honest and transparent reporting of how biases and other possible confounders were identified and addressed throughout study processes. Such reporting, first described within the study’s conceptual framework, should be revisited in reporting the work. Confounders may include the researcher’s training and previous experiences, personal connections to the background theory, access to the study population, and funding sources. These elements and processes are best represented in Glassick’s criteria for effective presentation and reflective critique ( Table 1 , criteria 5 and 6). Transferability is communicated, in part, through description of sampling factors such as: geographical location of the study, number and characteristics of participants, and the timeframe of data collection and analysis. 40 Such descriptions also contribute to the credibility of the results and readers’ determination of transfer to their and other contexts. To ensure dependability, the research method must be reported in detail such that the reader can determine proper research practices have been followed and that future researchers can repeat the study. 40 The confirmability of the results is influenced by reducing or at a minimum explaining any researcher influence on the result by applying and meeting standards of rigor such as member checking, triangulation, and peer review. 29,33

In qualitative studies, the researcher is often the primary instrument for data collection. Any researcher biases not adequately addressed or errors in judgement can affect the quality of data and subsequent research results. 33 Thus, due to the creative interpretative and contextually bound nature of qualitative studies, the application of standards of rigor and adherence to systematic processes well-documented in an audit trail are essential. The application of rigor and quality criteria extend beyond the researcher and are also important to effective peer review processes within a study and for scholarly dissemination. The goal of rigor in qualitative research can be described as ensuring that the research design, method, and conclusions are explicit, public, replicable, open to critique, and free of bias. 41 Rigor in the research process and results are achieved when each element of study methodology is systematic and transparent through complete, methodical, and accurate reporting. 33 Beginning the study with a well-developed conceptual framework and active use of both researcher reflexivity and rigorous peer review during study implementation can drive both study rigor and quality.

As the number of published qualitative studies in health professions educational research increases, it is important for our community of health care educators to keep in mind the unique aspects of rigor in qualitative studies presented here. Qualitative researchers should select and apply any of the above referenced study methods and research practices, as appropriate to the research question, to achieve rigor and quality. As in any research paradigm, the goal of quality and rigor in qualitative research is to minimize the risk of bias and maximize the accuracy and credibility of research results. Rigor is best achieved through thoughtful and deliberate planning, diligent and ongoing application of researcher reflexivity, and honest communication between the researcher and the audience regarding the study and its results.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Part 4: Using qualitative methods

18. Qualitative data collection

Chapter outline.

• Ethical responsibility and cultural respect (5 minute read)
• Critical considerations (3 minute read)
• Preparations for the data gathering process (6 minute read)
• Interviews (20 minute read)
• Focus groups (15 minute read)
• Observations (6 minute read)
• Documents and other artifacts (13 minute read)

Content warning: examples in this chapter contain references to multiple demands on students’ time, loss of employment, sexual assault, trauma-informed care, inpatient psychiatric services, immigration, and the Holocaust.

In this chapter we will explore information to help you plan for and organize your strategy to gather your qualitative data. You will face a number of decisions as you plan this section of your proposal. Gathering qualitative data comes with important ethical and cultural responsibilities. Furthermore, qualitative research can be a powerful tool, but we need to be thoughtful as to how it will be used, as it can as easily become a tool of oppression as one of empowerment. Below are some considerations to help you reflect on some of these dynamics as you plan your study. The first sections apply to every type of qualitative research. Then, we discuss specific strategies to choose from as you plan your qualitative study.

18.1 Ethical responsibility and cultural respect

Learning objectives.

Learners will be able to…

• Explain the special considerations researchers should keep in mind as they design qualitative studies and collect qualitative data
• Determine steps that can be taken to protect participants and exhibit cultural respect during qualitative data collection

Because qualitative data collection so often involves direct contact with human participants and requesting them to share detailed and potentially personally sensitive information with us as researchers, we need to be especially sensitive to ethical considerations. It is a process that requires forethought, planning, and mindful attention throughout. Below are some ethical considerations to help guide you in this activity.

Special limitations to anonymity, confidentiality and ability to remove or withdraw data

Because with qualitative research we are often meeting with people in person to gather data, either from interviews , focus groups , or observations , we clearly can’t guarantee them anonymity . This makes it all the more important to consider what you will do to protect the confidentiality of your participants. This may involve using steps like:

• Using pseudonyms or assigned study identification codes rather than names on study materials
• Stripping all potentially identifying information from transcripts
• Keeping signed informed consent forms separate from other data so the two can’t be linked
• Ensuring that when data is not being used it is appropriately stored and locked so that others outside the research team don’t have access to it
• Ensuring that when data is being used it is not in a space (in person or virtual) where people outside the research team can view it
• Making sure that all members of your research team have been approved by your IRB
• Being very clear in your informed consent who will have access to data and for what purposes

Additionally, at times we will write into our informed consent that participants may withdraw from a study at any time. When a person expresses a desire to withdraw, we remove their data from the study. However, let’s say we conducted interviews and identified a theme that was present in their interview, but was also in a number of other interviews. Their ideas would still be represented in our findings, but we would make sure not to use any quotes or unique contributions from that individual. Also, if a person participates in a focus group, they are part of an interactive dialogue and the discussion is often connected to ideas shared by others as the conversation evolves, making it very hard to completely remove their data. Again, we would respect their wishes by not using any of their direct words, but their presence and contributions shaped the discussion in ways that we won’t be able to excise. It is best to be upfront about this as you are seeking informed consent.

• What steps will you be taking to protect the qualitative data that is shared with you?

Prepare with competence, enter with humility

When we ask people to share their thoughts, feelings, and experiences with us, we need to do so in a way that demonstrates respect and authenticity . This means that we approach participants in a professional manner that reflects both competence as a researcher and that illustrates we have done some preparation to learn about the population ahead of time (that we are not “coming in cold”). Activities that can help to demonstrate this are:

• Speaking with knowledgeable community members regarding the topic, our research design, and important aspects of the community (contemporary and historical) before beginning our data collection)
• Examining previous research and other sources of information regarding the group/community we are interested in work with, or if not available, groups/communities that may be similar
• Using data from the first two bullet points, we design our data collection in a way that is culturally sensitive (e.g. where we ask people to provide data, what tools we use, our wording)
• Preparing research materials (e.g. informed consent forms, recruitment materials, informational sheets) that are accessible and understandable for participants
• Providing information and education about research in general and our research topic specifically

This needs to be tempered with humility. Participants grant us the privilege of allowing us to witness some piece of their life. We need to have humility in knowing that we can never fully understand their experiences because we are not them. In a real sense, we are the learners and they are the teachers. Despite us doing the pre-work discussed above to become more competent in our approach, humility means we will ask the participant directly what is acceptable in respect to our data collection. I believe that when taking a culturally humble approach that we should take at least a little bit of time to understand what research means to the participant and what this particular topic means to the them, again, by asking them directly.

Key Takeaways

• Qualitative data collection involves special considerations to help ensure the privacy, confidentiality, or anonymity of participants because of the the often intimate and detailed information that we are collecting as qualitative researchers.
• Preparing for qualitative data collection requires that we educate ourselves as researchers in advance about the population we will be working with to guide and develop our data collection plan. Furthermore, from the standpoint of cultural humility, we don’t assume that these preparations are adequate. We need to verify with participants what is culturally acceptable to them as individuals.
• As you prepare for data collection planning, what actions do you plan to take to demonstrate preparations for cultural sensitivity and cultural humility?

18.2 Critical considerations

• Assess factors that may impact community members’ perceptions of researchers and their intentions
• Identify opportunities to support greater reciprocity in researcher-participant relationships (especially as it relates to your proposal)

What/whose interests are represented?

Data is a resource that participants own that they choose to share with us. Think about it: When a smartphone app or computer program wants your personal data, you’re usually asked to read a privacy statement and agree to certain terms. Companies are legally required to notify you about their intentions to use the data you may share. And many companies certainly recognize that your data is a valuable resource and seek it out. As researchers, we have similar responsibilities, but with higher ethical standards.

If we are going to ask participants to share this resource, we need to consider why we need it. Clearly, we are invested in this research for some reason, otherwise we wouldn’t be spending our time doing it. Being upfront and genuine with our participants about why this topic is important to us and what we hope comes out of this research is a good first step. We also need to describe to other stakeholders (such as funders or sponsors) who might be involved why we are interested in it. In addition, it is helpful to consider what this research might represent to our participants.

• They may be unsure what to think about the research—This especially may be true if they have had limited exposure to research and/or academia.
• They might be nervous or apprehensive that it could have consequences, either for them individually or for their community
• They might be excited to share their story and may feel as though they are contributing to something larger or some beneficial change

Considering these factors can help us to be more sensitive as we prepare to enter the field for data collection.

Think about your study. Put yourself in the role of research participant.

What information would you want to know?

• About research in general
• About the researcher
• About the research topic

How reciprocal is the arrangement?

Building off the preceding discussion about what research might mean to participants, it is also important to consider the reciprocity in the researcher – participant relationship. We know that we are benefiting from the exchange – we are getting data, research findings, research products and any other advantages or opportunities that might be attached to these. However, the benefits are not always as clear on the participant side of this relationship. Sometimes we are able to provide incentives to honor a participant’s time and contribution to a project, but these are often relatively limited. Participants may also intrinsically value making a contribution to a research project that can eventually help to change or build awareness around something that is important to them, but these are often distant and intangible benefits. While we may not be able to change the fact that we may benefit more from this exchange than our participants, it is important for us to acknowledge this and to consider how this can affect the power differential. We may be asking for a lot, with relatively little to offer in return. This is in contrast to participatory research approaches (which have been discussed elsewhere), in which there is much more of an intentional effort to more equally distribute the benefits of these relationships.

• As a means of developing empathy as a researcher, it is worth considering what the significance or meaning of research is to the populations we are interested in working with. What do we (as researchers) and our projects represent to community members?
• As critical researchers, we need to be considered with the power differences that often exist as we conduct research, especially in the act of asking for data from participants. The request is often lop-sided, with us benefiting considerably more than the participant.

18.3 Preparations for the data gathering process

• Explain important influences to account for in qualitative data gathering
• Organize and document preparatory steps to plan data gathering activities for your qualitative proposal

As you may have guessed from our discussion regarding qualitative research planning and sampling, you have a number of options available for qualitative data gathering, and consequently, a number of choices to make. Your decisions should be driven by your research question and research design, including the resources that are at your disposal for conducting your study. Remember, qualitative research is a labor-intensive venture. While it may not require lots of fancy equipment, it requires a significant investment of people’s time and potentially other resources (e.g. space, incentives for participants, transportation). Each source of data (interviews, focus groups, observations, other artifacts), will require separate planning as you approach data gathering.

Our impact on the data gathering process

In the last chapter, you were introduced to the tool of reflexive journaling as a means of encouraging you to reflect on and document your role in the research process. Since qualitative researchers generally play a very active and involved role in the data gathering process (e.g. conducting interviews, facilitating focus groups, selecting artifacts), we need to consider ways to capture our influence on this part of the qualitative process. Let’s say you are conducting interviews. As you head into the interview, you might be bringing in thoughts about a previous interview, a conversation you just had with your research professor, or worries about finishing all your assignments by the end of the semester! During the interview, you are likely to be surprised by some things that are said or some parts may evoke strong emotions. These responses may lead you to consider pursuing a slightly different line of questioning, and potentially highlighting or de-emphasizing certain aspects. Understanding and being aware of your personal reactions during the data collection process is very important. As part of your design and planning, you may specify that you will reflexively journal before and after each interview in an attempt to capture pre- and post-interview thoughts and feelings. This can help us to consider how we influence and are influenced by the research process. Towards the end of this chapter, after we have had a chance to talk about some of these data gathering strategies, there is a reflexive journal prompt to help you consider how to begin to reflect on the way you as a researcher might impact your work and how you work might impact you.

Decision Point

How will you account for your role in the research process?

• This may be your reflexive journal or you may have other thoughts about how you can account for this.
• Whatever you choose, how will you develop a routine/habit around this to ensure that you are regularly implementing this?

Reflexive Journal Entry Prompt

This is going to be a bit meta, but for this prompt, I want you reflect on the reflecting you are doing for your reflexive journaling.

• Do you see this as a potentially helpful tool for tracking your influence and reactions? What appeals to you? What puts you off?
• If so, how did you develop this mindset?
• If not, how can you strengthen this skill?

When are we done

Finally, as you plan for your data collection you need to consider when to stop. As suggested previously in our discussion on sampling, the concept of saturation is important here. As a reminder, saturation is the point at which no new ideas or concepts are being presented as you continue to collect new pieces of data. Again, as qualitative researchers, we are often collecting and analyzing our data simultaneously. This is what enables us to continue screening for the point of saturation. Of course, not all studies utilize the point of saturation as their determining factor for the amount of data they will collect. This may be predetermined by other factors, such as restricted access or other limitations to the scope of the investigation. While there is no hard and fast rule for the quantity of data you gather, the quality is important; you want to be comprehensive, consistent, and systematic in your approach.

Next, we will discuss some of the different approaches to gathering qualitative data. I’m going to start out with Table 18.1 that allows us to compare these different approaches, providing you with a general framework that will allow us to dive a bit deeper into each one. After you finish reading this chapter, it might be helpful to come back to this table as you continue with your proposal planning.

• As you are preparing to initiate data collection, make sure that you have a plan for how you will capture and document your influence on the process. Reflexive journaling can be a useful tool to accomplish this.
• Be sure to take some time to think about when you will end your data collection. Make this an intentional, justified decisions, rather than a haphazard one.

18.4 Interviews

• Identify key considerations when planning to use interviewing as a strategy for qualitative data gathering, including preparations, tools, and skills to support it
• Assess whether interviewing is an effective approach to gather data for your qualitative research proposal

A common form of qualitative data gathering involves conducting interviews . Interviews offer researchers a way to gather data directly from participants by asking them to share their thoughts on a range of questions related to a research topic. Interviews are generally conducted individually, although occasionally couples (or other dyads , which consist of a combination of two people) may be interviewed. Interviews are a particularly good strategy for capturing unique perspectives and exploring experiences in detail. People may have a host of responses to the request to be interviewed, ranging from flat out rejection to excitement at the opportunity to share their story. As you plan to conduct your interviews you will need to decide on your delivery method, how you will capture the data, you will construct your interview guide , and hone your research interviewing skills.

Delivery method

As technology has advanced, so too have our options for conducting interviews. While in-person interviews are generally still the mainstay of the qualitative researcher, phone or video-based interviews have expanded the reach of many studies, allowing us to gain access to participants across vast distances with relatively few resources. Interviewing in-person allows you to capture important non-verbal and contextual information that will likely be limited if you choose to conduct your interview via phone or video. For instance, if we conduct an interview by phone, we miss the opportunity to see how our participant interacts with their surroundings and we can’t see if their arms are crossed or their foot is fidgety. This may indicate that a certain topic might make them particularly uncomfortable. Alternatively, we may pose a question that makes a smile come across their face. If we are interviewing in person, we can ask a follow-up question noting the smile as a change in their expression, however, it’s hard to hear a smile over the phone! Additionally, there is something to be said for the ability to make a personal connection with your interviewee that may help them to engage more easily in the interview process. This personal connection can be challenging over the phone or mediated by technology. As an example, I often offer to my students that we can meet for “virtual” office hours using Zoom if it is hard for them to get to campus. However, they will often prefer to come to campus, despite the inconvenience because they would prefer to avoid the technology.

Regardless of which method you select, make sure you are well prepared. If you are meeting in person, know where you are going and allow plenty of time to get there. Remember, you are asking someone to give up their time to speak with you, and time is precious! When determining where you will meet for your interview, you may choose to meet at your office, their home, or a neutral setting in the community. If meeting somewhere in the community, do consider that you want to choose a place where you can reasonably assure the participant’s privacy and confidentiality as they are speaking with you. In most instances, I try to ask participants where they would feel most comfortable meeting. If you are speaking over phone or video, make sure to test your equipment ahead of time so that you are comfortable using it, and make sure that both you and the participant have access to a private space as you are speaking. If participants have minor children, plan ahead for whether the children should stay in the same space as the interview. If not, you may need to arrange child care or at least discuss child care with participants in advance. We also want to be mindful of how we are situated during an interview, ideally minimizing any power imbalances. This may be especially important when meeting in an office, making sure to sit across from our participants rather than behind a desk.

Capturing the data

You will also need to consider how you plan to physically capture your data. Some researchers record their interviews, using either a smartphone or a digital recording device. Recording the exchange allows you to have a verbatim record, which can allow the researcher to more fully participate in the interview, instead of worrying about capturing everything in writing. However, if there is a problem with recording – either the quality of the recording or some other equipment malfunction, the researcher can be up the proverbial creek without a paddle. Additionally, using a recording device may be perceived as a barrier between the researcher and the participant, as the participant may not feel comfortable being recorded. If you do plan to record, you should always ask permission first and announce clearly when you are starting and stopping the recording. If you will use recording equipment, be sure to test it carefully in advance, and bring backup batteries/phone charger with you.

The alternative to recording is taking field notes. Field notes consist of a written record of the interview, completed during the interview. You may elect to take field notes even if you are recording the interview, and most people do. This allows us to capture main ideas that stand out to us as researchers, nonverbal information that won’t show up in a recording, and some of our own reactions as the interview is being conducted. These field notes become invaluable if you have a problem with your recording. Even if you don’t, they provide helpful information as you interpret the data you do have in your transcript (the typed version of your recording).

If you are not recording and are relying completely on your notes, it is important to know that you are not going to capture every word and that you shouldn’t try. You want to plan in advance how you will structure your notes so that they make sense to you and are easy to follow. Try to capture all main ideas, important quotes that stand out, and whenever possible, use the participant’s own words. We need to recognize that when we paraphrase what the person is stating, we are introducing our ‘spin’ on it – their ideas go through our filter. We likely can’t avoid some of this, but we do want to minimize it as much as possible. Part of how we do this when we are relying on field notes is to take our interview notes and create expanded field notes , ideally within 24 hours of the interview. The longer you wait to expand your field notes, the less reliable they become, as our memory fades quickly! Much like they sound, expanded field notes take our jottings from the interview and expand them, providing more detail regarding the context or meaning of the statements that were captured. Expanded field notes may also contain questions, comments, or reactions that we, as the researcher, may have had to the data, which are usually kept in the margins, rather than in the body of the notes.

Below are a few resources to learn more about taking quality field notes. Along with the reading, practice, practice, practice!

Resources to learn more about capturing your Field Notes:

Deggs, D., & Hernandez, F. (2018). Enhancing the value of qualitative field notes through purposeful reflection .

Robert Wood Johnson Foundation (2008). Qualitative guidelines project: Fieldnotes .

University of Southern California Libraries. (2019). Research guides: Organizing your social sciences research paper, writing field notes .

Wolfinger, N. (2002). On writing fieldnotes: Collection strategies and background expectancies.

Interview guide

The questions that you ask during your interview will be outlined in a tool called an interview guide . Along with your interview questions, your interview guide will also often contain a brief introduction reminding the participant of the topics that will be covered in the interview and any other instructions you want to provide them (note: much of this will simply serve as a reminder of what you already went over in your informed consent, but it is good practice to remind them right before you get started as well). In addition, the guide often ends with a debriefing statement that thanks the participant for their contribution, inquires whether they have any questions or concerns, and provides contact and resource information as appropriate. Below is a brief interview guide for a study that I was involved with, in which we were interviewing alumni regarding their perceptions of advanced educational needs in the field of social work and specifically their thoughts about practice doctorate of social work (DSW) degrees/programs.

Some interviews are prescribed or structured, with a rigid set of questions that are asked consistently each time, with little to no deviation. This is called a structured interview . More often however, we are dealing with semi-structured interviews , which provide a general framework for the questions that will be asked, but- contain more flexibility to pursue related topics that are brought up by participants. This often leads to researchers asking unplanned follow-up questions to help explore new ideas that are introduced by participants. Sometimes we also use unstructured interviews . These interview guides usually just contain a very open-ended talking prompt that we want participants to respond to. If we are using a highly structured interview guide, this suggests we are leaning toward deductive reasoning apporach—we have a pretty good idea based on existing evidence what we are looking for and what questions we want to ask to help us test our existing understanding. If we are using an unstructured guide, this suggests we are leaning toward an inductive reasoning approach—we start by trying to get people to elaborate extensively on open-ended questions to provide us with data that we will use to develop our understanding of this topic.

An important concept related to the contents of your interview guide is the idea of emergent design . With qualitative research we often treat our interview guide as dynamic, meaning that as new ideas are brought up, we may integrate these new questions into our interview guide for future interviews. This reflects emergent design, as our interview guide shifts to accommodate our emerging understanding of the research topic as we are gathering data. If you do plan to use an emergent design approach in your interviews, it is important to acknowledge this in your IRB application. When you submit your application, you will need to provide the IRB with your interview guide so that they have an idea of the questions you will be discussing with participants. While using an emergent approach to some of your questions is generally acceptable (and even expected), these questions still should be clearly relevant and related to what was presented in your IRB application. If you find that you begin diverging into new areas that are substantively different from this, you should consider submitting an IRB addendum that reflects the changes, and it may be a good idea to consult with your IRB to see if this is necessary.

Designing interview questions and probes

Making up questions, it sounds easy right? Little kids are running around asking questions all the time! However, what you quickly find when conducting research is that it takes skills, ingenuity and practice to craft good interview questions. If you are conducting an unstructured interview, you will generally have fewer questions and they will be quite broad. Depending on your topic, you might ask questions like:

• Tell me about a time…
• What was it like to…
• What should people understand about…
• What does it mean to…

If your interview is more structured, your questions will be a bit more focused, but with qualitative interviewing, we are still generally trying to get people to open up about their experiences with something, so you will want to design questions that will help them to do this. Probes can be important tools to help us accomplish this. You can think of probes as brief follow-ups that are attached to a particular question that will help you explore a topic a bit further. We usually develop probes either through existing literature or knowledge on a topic, or we might add probes to our interview guide as we begin data collection based on what previous participants tell us. As an example, I’m very interested in research on the concept of wellness. I know that the Substance Abuse and Mental Health Services Administration (SAMHSA) has adopted a heuristic tool, The Wheel of Wellness , that outlines eight dimensions of wellness based on research by Swarbrick (2006). [1] When interviewing participants with the broad, unstructured question “What does wellness mean in your life?”, I might use these eight dimensions that are spokes of this wheel (i.e. emotional, spiritual, intellectual, physical, environmental, financial, occupational, and social) as probes to explore if/how these dimensions might be relevant in the lives of these participants. Probes suggest that we are anticipating that certain areas may be relevant to our question.

Here are a few general guidelines to consider when crafting your interview questions.

Make them approachable

We are usually relatively unfamiliar with our participants, at least on a personal level. This can make sitting down for an interview where we might be asking some deep questions a bit awkward and uncomfortable, at least at first. Because of this, we want to craft our questions in such a way that they are not off-putting, inadvertently accusatory or judgmental, or culturally insensitive. To accomplish this, we want to make sure we phrase questions in a neutral tone (e.g. “Tell me what that was like”, as opposed to, “That sounds horrible, what was that like”). To accomplish this, we can shift perspectives and think about what it would be like for us to be asked these questions (especially by a stranger). Pilot testing is especially important here. You should plan in time for this, both conducting pilot testing and incorporating feedback on questions. Pilot testing involves you taking your questions on a dry-run with a few people outside of your sample. You might consider testing these out with peers, colleagues, or friends to get their perspective. You might want to get feedback on:

• Did the question make sense to them?
• Did they know what information you were looking for and how to respond?
• What was it like to be asked that question?
• What suggestions do they have for rephrasing the question (if it wasn’t clear)?

Also, if we are conducting interviews on topics that may be particularly hard for people to talk about, we will likely want to start out with some questions that are easier to address prior to getting into the heavier topics.

Make them relatable

Unlike surveys, where researchers may not be able to explain the meaning of a question, with interviews, we are present to help clarify questions if needed. However, ideally, our questions are as clear as possible from the beginning. This means that we avoid jargon or technical terms, we anticipate areas that might be hard to explain and try to provide some examples or a metaphor that might help get the point across, and we do our homework to relay our questions in an appropriate cultural context. Like the discussion above, pilot testing our questions can be very helpful for ensuring the relatability of our questions, especially with community representatives. When pilot testing, do your best to test questions with a person/people from the same culture and educational level as the future participants. What sounds good in our heads might make little sense to our intended audience.

Make them individually distinct, but collectively comprehensive

Just like when we are developing survey questions, you don’t want to ask more than one question at the same time. This is confusing and hard to respond to for the participant, so make sure you are only asking about one idea in each question. However, when you are thinking about your list of questions, or about your interview guide collectively, ensure that you have comprehensively included all the ideas related to your topic. It’s extremely disheartening for a qualitative researcher that has concluded their interviews to realize there was a really important area that was not included in the guide. To avoid this, make sure to know the literature in your area well and talk to other people who study this area to get their perspective on what topics need to be included. Additional topics may come up when you pilot test your interview questions.

Interview skills

As social workers, we receive much training regarding interviewing and related interpersonal skills. Many of these skills certainly transfer to interviewing for research purposes, such as attending to both verbal and non-verbal communication, active listening, and clarification. However, it is also important to understand how a practice-related interview differs from a research interview.

The most important difference has to do with providing clarity around the purpose of the interview. For a practice-related interview, we are gathering information to help understand our client’s situation and better meet their needs. The interview is a means to provide quality services to our clients, and the emphasis is on the client and resources flowing to them. However, the research interview is ideologically much different. The interview is the means and the end. The purpose of the interview is to help answer the research question, but most often, there is little or limited direct benefit to the participant. The researcher is largely the beneficiary of the exchange, as the participant provides us with data. If the participant does become upset or is negatively affected by their participation, we may help facilitate their connection with appropriate support services to address this, such as counseling or crisis numbers (and indeed, this is our ethical obligation as a competent researcher). However, counseling and treatment is not our responsibility when conducting research interviews and we should be very careful not to confuse it as such. If we do act in this way, it creates the potential for a dual relationship with the interviewee (participant and client) and puts them in a vulnerable situation. Make sure you are clear what your role is in this encounter.

Along with recognizing the focus of your role, here is a checklist of general tips for qualitative interviewing skills:

• Approach the interview in a relaxed, but professional manner
• Be observant of verbal, nonverbal, and contextual information
• Exhibit a non-judgmental stance
• Explain information clearly and check for comprehension
• Demonstrate respect for your participants and be polite
• Utilize much more listening and much less talking
• Check for understanding when you are unclear, rather than making assumptions
• Know your materials and technology (e.g. informed consent, interview guide, recording equipment)
• Be concise, clear and organized as you are taking notes
• Have a structured approach for what you need to cover and redirect if the conversation is losing focus
• Be flexible enough so that the interview does not become impersonal and disengaging due to rigidity of your agenda
• Data collection through interviewing requires careful planning for both how we will conduct our interviews (e.g. in person, over the phone, online) and the nature of the interview questions themselves. An interview guide is an important document to develop in planning this.
• Qualitative interviewing uses similar skills to clinical interviewing, but is markedly different. This difference is due in large part to the very different purpose of these two activities.

Let’s get some practice!

Thinking about your topic, if you were to use interviewing as an approach for data collection, identify 4 interview questions that you would consider asking about your topic. Make sure these are open-ended questions so that your participants can elaborate on them.

• Interview question 1:
• Interview question 2:
• Interview question 3:
• Interview question 4:

Now pilot these. Ask a peer to read these questions and think about trying to answer them. You aren’t interested in their actual answers, you want feedback about how these questions were.

• Were they understandable and clear?
• Were they potentially culturally insensitive or offensive in any way?
• Are they something that it seems reasonable that someone could answer (especially with a researcher they likely don’t know previously)?
• Are they asked in a way that are likely to get people to elaborate (rather than just give a one-word answer)?
• What suggestions do they have to address all/any of these areas?

Based on your peer feedback, re-write your four questions incorporating their suggestions.

• Revised interview question 1:
• Revised interview question 2:
• Revised interview question 3:
• Revised interview question 4:

Resources for learning more about conducting Qualitative Interviews.

Baker, S. E., & Edwards, R. (2012) National Centre for Research Methods review paper: How many qualitative interviews is enough?

Clifford, S. Duke University Initiative on Survey Methodology at the Social Science Research Institute (n.d.). Tipsheet: Qualitative interviews.

Harvard University Sociology Dept. (n.d.). Strategies for qualitative interviews .

McGrath et al., (2018). Twelve tips for conducting qualitative research interviews .

Oltmann, S. M. (2016). Qualitative interviews: A methodological discussion of the interviewer and respondent contexts .

A few exemplars of studies employing Interview Data:

Ewart‐Boyle, S., Manktelow, R., & McColgan, M. (2015). Social work and the shadow father: Lessons for engaging fathers in Northern Ireland .

Flashman, S. H. (2015). Exploration into pre-clinicians’ views of the use of role-play games in group therapy with adolescents .

Irvin, K. (2016). Maintaining community roots: understanding gentrification through the eyes of long-standing African American residents in West Oakland .

18.5 Focus groups

• Identify key considerations when planning to use focus groups as a strategy for qualitative data gathering, including preparations, tools, and skills to support it
• Assess whether focus groups are an effective approach to gather data for your qualitative research proposal

Focus groups offer the opportunity to gather data from multiple participants at once. As you have likely learned in some of your practice coursework, groups can help facilitate an environment where people feel (more) comfortable sharing common experiences which can often allow them to delve deeper into topics than they may have individually. As people relate to what others in the group say, they often go on to share their responses to these new ideas – offering a collaborative synergy. Of course, similar to the research vs. clinical interview described above, the purpose of the focus group is much different than that of the therapeutic, psychoeducational, or support group. While other elements (e.g. information sharing, encouragement) may take place, the aim of the focus group must remain anchored in the collection of data and that should be made explicitly clear so participants have accurate expectations. As a cautionary note, the advantages discussed above should be the reason you choose to use a focus group to collect data. You should not choose to conduct a focus group solely out of convenience. Focus groups require a considerable amount of planning and skill to execute well, so it is not reasonable to think that just because a focus group allows you to collect data from multiple participants at once that it is an easier option for data gathering.

Group assembly

Assembling your focus group is an important part of your planning process. Generally speaking, focus groups shouldn’t exceed 10-12 participants. When thinking about size, there are a couple things to consider. On the lower end, you do want enough participants so that they don’t feel pressure to be constantly speaking. I f you only have a couple of focus group members, it loses most of the collective benefit of the focus group approach, as there are few people to generate and share ideas. On the higher end, you want to avoid having so many participants that not everyone gets to be heard and the group conversation becomes unwieldy and hard to manage.

As you are forming your group, you want to strike up a balance between heterogeneity (difference) and homogeneity (sameness) between your group members. If the group is too heterogeneous, then opinions may be so polarized that it is hard to have a productive conversation about the topic. People may not feel comfortable sharing their opinion or it may be difficult to gain a common understanding across the data. If the group is too homogeneous, then it may be hard to get much depth from the data. People may see the topic so similarly that we don’t gain much information about how differing perspectives think about the issue. You generally want your group composition to be different enough to be interesting and produce good conversation, but similar enough that members can relate to each other and have a cohesive conversation. Along these lines, you also need to consider whether or not your participants know each other. Do they have existing relationships? If they do know each other, we need to anticipate that there may be existing group dynamics. This may influence how people engage in discussion with us. On one hand, they may find it easy to share more freely. However, these dynamics may inhibit them from speaking their mind, as they might be concerned about repercussions for sharing within their social network.

As a final note on group composition, sometimes we make decisions on group members’ characteristics based on our topic. For instance, if we are asking questions about help-seeking and common experiences after (heterosexual) sexual assault, it may be challenging to host a mixed-gender group, where participants may feel triggered or guarded having members of the opposite gender present and therefore potentially less open to sharing. It is important to consider the population you are working with and the types of questions you are asking, as this can help you to be sensitive to their perceptions and facilitate the creation of a safe space. Other issues, such as race, age, levels of education, may require consideration as you think about your group composition.

Related to feelings of safety, the setting you select for your focus group is an important decision. Much like with interviews, we want participants to feel as comfortable and at-ease as possible, however, it is perhaps less common to use someone’s home for the purpose of a focus group because we are often bringing together people who may not know one another. As such, try to select a place that feels neutral (e.g. some people may not feel comfortable in a church or a courthouse), accessible, convenient, and that offers privacy for participants. If you are working with a particular group or community, there may be a space that is especially relevant or familiar for people that may work well for this purpose. A c ommunity gatekeeper or other knowledgeable community member can be an excellent resource in helping to identify where a good spa ce might be. Seating in a circle will help participants to share more easily. Focus group organizers often provide refreshments as an incentive and to make participants feel more comfortable. If you decide to provide refreshments, be sensitive to issues like common dietary restrictions and cultural preferences.

Roles of the researcher(s)

Ideally, you are conducting your focus group with a co-researcher. This is important because it allows you to divide up the tasks and makes the process more manageable. Most often, one of you will take on the main facilitator role, with responsibilities for providing information and instructions, introducing topics, asking follow-up questions and generally structuring the encounter. The other person takes on a note-taking/processing role. While not necessarily silent, they likely say very little during the focus group. Instead, they are focused on capturing the context of the encounter. This may include taking notes about what is said, how people respond or react, other details about the space and the overall exchange as a whole. They will also often be especially attentive to group dynamics and capturing these whenever possible. Along with this, if they see that certain group members are dominating or being left out of the conversation, they may help the facilitator to address or shift these dynamics so that the sharing is more equitable. Finally, if something arises where a participant becomes upset or there is an emergency where they need to leave the room, having a co-researcher allows one of you to remain with the group, while the other can attend to the person in distress. For consistency sake, you may want to maintain roles throughout data collection. If you do decide to alternate roles as you conduct multiple focus groups, it is important that you both conduct the respective roles as similarly as possible. Remember, research is about the systematic collection of data, so you want your data collection to follow a consistent process. Below is a chart that offers some tips for each of these roles.

Focus group guide and preparations

As in your preparation for an interview, you will want to spend considerable time developing your focus group guide and the questions it contains. Be sure the language you use in your questions is appropriate for the educational level of your participants; you will need to use vocabulary that is clear and not “jargon”. At the same time, you also want to avoid talking down to your participants. You will probably want to start with some easier, non-threatening questions to help break the ice for the group and help get folks comfortable talking and sharing their input. Be prepared to ask questions in a different way or follow up with probes to help prod the conversation along if a question falls flat or fails to elicit a dialogue. In addition, you will want to plan introductions, both to the study and to one another. Usually we stick to first names, and occasionally during introductions, participants will share how they are connected to the topic of the research. Just like in many practice-related groups, facilitators usually take time to review group norms and expectations before getting started with questions. Some common norms to discuss are:

• Not talking over other participants
• Being respectful of other participants’ contributions
• All people are expected to participate in the conversation
• Not pressuring people to respond to a question if they are uncomfortable
• Using respectful language and avoiding derogatory, discriminatory or accusatory language or tone
• Not using electronic devices and silencing cell-phones during the focus group
• Allowing others ample time to contribute to the conversation and not dominating the discussion

Another expectation to address that is especially important to include is confidentiality . It is important to make clear to participants that what is shared in the group should be kept confidential and not discussed outside the context of the focus group. Additionally, it is important to let participants know that while the researchers ask all participants to protect the confidentiality of what is shared, they can’t guarantee that will be honored. Below figure 18.4 offers an example of a focus group guide template to help you think about how to structure this type of document.

Capturing your data

Finally, as with interviews, you will need to plan how you will capture the data from your focus group(s). Again, you may choose to record the focus groups, take fie ld notes, or use a combination of both. There are some special considerations that apply to these choices when using a focus group, however. First, if recording, anticipate that it may be especially challenging when transcribing the recording to determine who said what. In addition, the quality of the recording can become a challenge. Despite requests for individuals to speak one at a time, inevitably there will be spots where there are multi ple people talking at once, especially with an animated group. Additionally, do test the recording devices, ideally in the space you will be using them. You want to make sure that it can pick up everyone’s voice, even if they are soft-spoken and seated a distance from the device. If you are relying solely on a recordi ng and there is a problem with it, it can be difficult to surmount the barriers this can pose . If this occurs with an interview, while not ideal, you can re-interview a person to replace the information, but re-creating a focus group can be a logistical night mare. When taking field notes , it is a good practice to make a quick seating chart at the beginning so you can make quick references for yourself of who is saying what (see Figure 18.5). Regardl ess of what system you use to stay organize d in taking these notes, make sure to have one that works for you. The conversations will likely happen more rapidly and will include multiple voices, so you will want to be prepared in advance.

• Focus groups offer a valuable tool for qualitative data collection when the topic we are exploring might best be understood through a group discussion that helps participants verbally process and consider their experiences, thoughts, and opinions with others.
• Details like focus group composition, roles of co-facilitators, and anticipation of group norms or guidelines require our attention as we prepare to host a focus group.

Reflexive journal prompt

How do you feel about conducting a focus group?

• What about it is appealing
• What about it seems challenging
• Would you prefer to be the main facilitator or the observer (and why)?
• What might make using a focus group a good choice for your specific research question?
• What might make using a focus group a poor choice for your specific research question?

Resources to learn more about conducting Focus Groups.

Leung, F. H., & Savithiri, R. (2009). Spotlight on focus groups .

Duke, ModU (2016, October 19). Powerful concepts in social science: Preparing for focus groups, qualitative research methods

Onwuegbuzie et al. (2009). A qualitative framework for collecting and analyzing data in focus group research .

Nyumba et al. (2018). The use of focus group discussion methodology: Insights from two decades of application in conservation .

Robert Wood Johnson Foundation (2008). Qualitative Guidelines Project: Focus groups.

A few exemplars of studies employing Focus Groups:

Foote, W. L. (2015). Social work field educators’ views on student specific learning needs .

Hoover, S. M., & Morrow, S. L. (2016). A qualitative study of feminist multicultural trainees’ social justice development .

Kortes-Miller, K., Wilson, K., & Stinchcombe, A. (2019). Care and LGBT aging in Canada: A focus group study on the educational gaps among care workers .

18.6 Observations

• Identify key considerations when planning to use observations as a strategy for qualitative data gathering, including preparations, tools, and skills to support it
• Assess whether observations are an effective approach to gather data for your qualitative research proposal

Observational data can also be very important to the qualitative researcher. As discussed in Chapter 17 , observations can provide important information about context, rea ctions, behaviors, exchanges, and expressions. The focus of observations may be indi viduals, i nteractions between people or within groups, environments or settings, or events like artistic expressions (e.g. plays, poetry readings, art shows), public forums (e.g. town hall meetings, community festivals), private forums (e.g. board meetings, family reunions), and finally, your reactions or responses as the researcher to any and all of these. We will be discussing a variety of different types of qualitative designs in Chapter 22 , including ethnography. Observational data is especially important for ethnographic research designs. 

Researcher engagement

Observational data gathering is a more indirect form of data collection when compared with previous methods we have discussed. With both interviews and focus groups, you are gathering data directly from participants. When making observations, we are relying on our interpretation of what is going on. Even though we are often not directly interacting with people, we generally have an ethical responsibility to disclose that we are gathering data by making observations and gain consent to do so. That being said, there are some instances where we are making observations in public spaces, and in these instances disclosure may not be necessary because we are not gathering any identifiable information about specific people. These instances are rare, but if you are in doubt, consult with your IRB.

Even though I just suggested that making observations is often a more indirect form of data gathering, it does exist on a continuum. If utilizing observational data, you will need to consider where you fall on this continuum. Some research designs situate the researcher as an active participant in the community or group that they are studying, while other designs have the researcher as an independent and detached onlooker. In either case, you need to consider how your presence, either involved or detached, may influence the data you are gathering. This requires us to think of this on a more individual or micro level (how do the individuals we are directly observing perceive us) and a more mezzo or even macro level (how does the community or group of people we are studying collectively feel about our presence and our research)? Are people changing their behavior because of your presence? Are people monitoring or censoring what they say? We can’t always know the answers to these questions, but we can try to reduce these concerns by making repeated observations over time, rather than using a one-time, in-and-out data gathering mission. This means actually spending time within the community that is the focus of your observation. Taking the time to make repeated observations will allow you to develop a reasonable framework of understanding, which in turn will empower you to better interpret what you see and help you determine whether your observations and interpretation are consistent.

Observational skills

When gathering observational data, you are often attending to or taking in many different dimensions. You are potentially observing:

• the context of the environment
• the content of what is being said
• behaviors of people
• affective or emotional aspects of interactions
• sequences of events
• your own reactions to what is being observed

To capture this information, you will need to be keenly aware, focused, and organized. Additionally, you need to make sure you are capturing clear descriptions of what is going on. Remember, notes that seem completely logical and easy to understand at the time you are taking them can become vague and confusing with the passage of time and as you gather more and more data. Part of the clarity of your description often involves taking a non-judgmental approach to documenting your observations. While this may seem easy, judgments or biases frequently slip into our thinking and writing (unbeknownst to us). Along with a non-judgmental stance, researchers making observations also attempt to be as unobtrusive as possible. This means being conscious of your behaviors, your dress and overall appearance. If you show up wearing a suit and tie, and carrying a clipboard while everyone else is wearing jeans and t-shirts, you are likely to stick out like a sore thumb. This is also likely to influence how participants respond and interact with you. Know the environment that you are making your observations in, with a goal of blending in as much as possible.

Observational data is most often captured using field notes. Using recordings for observational data is infrequently used in social work research. This is especially true because of the potential for violations of privacy and threats to confidentiality that recordings (video or audio) may pose to participants. Mirroring our discussion above, when taking field notes, make sure to be organized and have a plan for how you will structure your notes so they are easy to interpret and make sense to you. Creswell (2013) [2] suggests capturing ‘descriptive’ and ‘reflective’ aspects in your observational field notes. Table 18.3 offers some more detailed description of what to include as you capture your data and corresponding examples.

For the purposes of qualitative research, our observations are generally unstructured or more naturalistic . However, you may also see mention of more systematic or structured observations. This is more common for quantitative data collection, where we may be attempting to capture or count the frequency with which a specific behavior or event occurs.

• Observational data collection can be an effective tool for gathering information about settings, interactions, and general human behavior. However, since this is gathered strictly through the researchers own direct observation, it is not a source of data on people’s thoughts, perceptions, values, opinions, beliefs or interpretations.
• There are a range of aspects that we may want to take note of while we are observing (e.g. the setting, interactions, descriptions of people, etc.).
• While we are making our observations, we generally want to do so as inconspicuously and non-judgmentally as possible.

Resources for learning more about conducting Qualitative Observations.

Kawulich, B.B. (2005, May) Participant observation as a data collection method .

Kawulich, B.B. (2012). Collecting data through observation . In C. Wagner, B. Kawulich, & M. Garner (Eds.), Doing social research: A global context ( 150-160). New York: McGraw Hill.

Robert Wood Johnson Foundation (2008). Qualitative Guidelines Project: Observations .

Sliter, M. (2014, June 30). Observational methods: Research methods.

A few exemplars of studies employing qualitative observations:

Avby et al. (2017). Knowledge use and learning in everyday social work practice: A study in child investigation work .

Wilkins et al. (2018). A golden thread? The relationship between supervision, practice, and family engagement in child and family social work .

Wood et al. (2017). The “gray zone” of police work during mental health encounters: Findings from an observational study in Chicago .

18.7 Documents and other artifacts

• Identify key considerations when planning to analyze documents and other artifacts as a strategy for qualitative data gathering, including preparations, tools, and skills to support it
• Assess whether analyzing documents and other artifacts is an effective approach to gather data for your qualitative research proposal

Qualitative researchers may also elect to utilize existing documents (e.g. reports, newspapers, blogs, minutes) or other artifacts (e.g. photos, videos, performances, works of art) as sources of data. Artifact analysis can provide important information on a specific topic, for instance, how same-sex couples are portrayed in the media. They also may provide contextual information regarding the values and popular sentiments of a given time and/or place. When choosing to utilize documents and other artifacts as a source of data for your project, remember that you are approaching these as a researcher, not just as a consumer of media. You need to thoughtfully plan what artifacts you will include, with a clear justification for their selection that is solidly linked to your research question, as well as a plan for systematically approaching these artifacts to identify and obtain relevant information from them.

Obtaining your artifacts

As you begin considering what artifacts you will be using for your research study, there are two points to consider: what will help you to answer your research question and what can you gain access to. In addressing the first of these considerations, you may already have a good idea about what artifacts are needed because you have done a substantial amount of preliminary work and you know this area well. However, if you are unsure, or you need to supplement your existing knowledge, some general sources can include: librarians, historians, community experts, topical experts, organizations or agencies that address the issue or serve the population you will be studying, and other researchers who study this area. In considering access, if the artifacts are public the answer may be a straightforward yes, but if the documents are privately held, you may need to be granted permission – and remember, this is permission to use them for research purposes, not just to view them. When obtaining permission, get something in writing, so that you have this handy to submit with your IRB application. While the types of artifacts you might include are almost endless (given they are relevant to your research question), Table 18.4 offers a list of some ideas for different sources you might consider:

Artifact analysis skills

Consistent with other areas of research, but perhaps especially salient to the use of artifacts, you will require organizational skills. Depending on what sources you choose to include, you may literally have volumes of data. Furthermore, you might not just be dealing with a large amount of data, but also a variety of types of data. Regardless of whether you are using physical or virtual data, you need to have a way to label and catalog (or file) each artifact so that you can easily track it down. As you collect specific information from each piece, make sure it is tagged with the appropriate label so that you can track it back down, as you very well may need to reference it later. This is also very important for honest and transparency in your work as a qualitative researcher – documenting a way to trace your findings back to the raw data .

In addition to staying organized, you also need to think specifically about what you are looking for in the artifacts. This might seem silly, but depending on the amount of data you are dealing with and how broad your research topic is, it might be hard to ‘separate the wheat from the chaff’ and figure out what is important or relevant information. Sometimes this is more clearly defined and we have a prescribed list of things we are looking for. This prescribed list may come from existing literature on the topic. This prescribed list may be based on peer-reviewed literature that is more conceptual, meaning that it focuses on defining concepts, putting together propositions, formulating early stage theories, and laying out professional wisdom, rather than reporting research findings. Drawing on this literature, we can then examine our data to see if there is evidence of these ideas and what this evidence tells us about these concepts. If this is the case, make sure you document this list somewhere, and on this list define each item and provide a code that you can attach when you see it in each document. This document then becomes your codebook .

However, if you aren’t clear ahead of time what this list might be, you may take an emergent approach, meaning that you have some general ideas of what you are seeking. In this event, you will actively create a codebook as you go, like the one described above, as you encounter these ideas in your artifacts. This helps you to gain a better understanding of what items should be included in your list, rather than coming in with preconceived notions about what they should be. There will be more about tracking this in our next chapter on qualitative analysis. Whether you have a prescribed list or use a more emergent design to develop your codebook, you will likely make modifications or corrections to it along the way as your knowledge evolves. When you make these changes, it is very important to have a way to document what changes you made, when, and why. Again, this helps to keep you honest, organized, and transparent. Just as another reminder, if you are using predetermined codes that you are looking for, this is reflective of a more deductive approach, whereas seeking emergent codes is more inductive .

Finally, when using artifacts, you may also need to bring in some creative, out-of-the-box thinking. You may be bringing together many different pieces of data that look and sound nothing alike, yet you are seeking information from them that will allow you tell a cohesive story. You may need to be fluid or flexible in how you are looking at things, and potentially challenge your preconceived notions.

As alluded to above, you may have physical artifacts that you are dealing with, digital artifacts or representations of these artifacts (e.g. videos, photos, recordings), or even field notes about artifacts (for instance, if you take notes of a dramatic performance that can’t be recorded). A large part of what may drive your decisions about how to capture your data may be related to your level of access to those artifacts: can you look at it? Can you touch it, can you take it home with you, can you take a picture of it? Depending on what artifacts we are talking about, some of these may be important questions. Regardless of the answers to these questions, you will need to have a clearly articulated and well-documented plan for how you are obtaining the data and how you will reference it in the future. Table 18.4 provides a list of data gathering activities you might consider, both for documents and for other audiovisual materials.

What types of artifacts might you have access to that might help to answer your research question(s)?

• These could be artifacts available at your field placement, publically available media, through school, or through public institutions
• These can be documents or they can be audiovisual materials
• Think outside the box, how can you gather direct or indirect indications of the thing you are studying

Generate a list of at least 3

Again, drawing on Creswell’s (2013) suggestion of capturing ‘descriptive’ and ‘reflective’ aspects in your field notes, Table 18.5 offers some more detailed description of what to include as your capture your data and corresponding examples when focusing on an artifact.

Resources to learn more about qualitative research with artifacts.

Bowen, G. A. (2009). Document analysis as a qualitative research method .

Rowsell, J. (2011). Carrying my family with me: Artifacts as emic perspectives .

Hammond, J., & McDermott, I. (n.d.). Policy document analysis .

Wang et al. (2017). Arts-based methods in socially engaged research practice: A classification framework .

A few exemplars of studies utilizing documents and other artifacts.

Casey, R. C. (2018). Hard time: A content analysis of incarcerated women’s personal accounts .

Green, K. R. (2018). Exploring the implications of shifting HIV prevention practice Ideologies on the Work of Community-Based Organizations: A Resource dependence perspective . 

Sousa, P., & Almeida, J. L. (2016). Culturally sensitive social work: promoting cultural competence .

Secondary data analysis

I wanted to briefly provide some special attention to secondary data analysis at the end of this chapter. In the past two chapters we have focused our sights most often on what we would call raw data sources . However, you can of course conduct qualitative research with secondary data , which is data that was collected previously for another research project or other purpose; data is not originating from your research process. If you are fortunate enough to have access and permission to use qualitative data that had already been collected, you can pose a new research question that may be answered by analyzing this data. This saves you the time and energy from having to collect the data yourself!

You might procure this data because you know the researcher that collected the original data. For instance, as a student, perhaps there is a faculty member that allows you access to data they had previously collected for another project. Alternatively, maybe you locate a source of qualitative data that is publicly available. Examples of this might include interviews previously conducted with Holocaust survivors. Finally, you might register and join a research data repository . These are sites where contributing researchers can house data that other researchers can view and request permission to use. Syracuse University hosts a repository that is explicitly dedicated to qualitative data . While there are more of these emerging, it may be a challenge to find the specific data you are looking for in a repository. You should also anticipate that data from repositories will have all identifiable information removed. Sharing data you have collected with a repository is a good way to extend the potential usefulness and impact of data, but it also should be anticipated before you collect your data so that you can build it into any informed consent so participants are made aware of the possibility.

Computer Assisted Qualitative Data Analysis Software (CAQDAS)

Some qualitative researchers use software packages known as Computer Assisted Qualitative Data Analysis Software (CAQDAS) in their work. These are tools that can aid researchers in managing, organizing and manipulating/analyzing their data. Some of the more common tools include NVivo, Atlas.ti, and MAXQDA, which have licensing fees attached to them (although many have discounted student rates). However, there are also some free options available if you do some hunting. Taguette Project is the only free and open source CAQDAS project that is currently receiving updates, as previous projects like RQDA which built from the R library are not in active development. Taguette is a young project, and unlike the free alternatives for quantitative data analysis, it lacks the sophisticated analytical tools of commercial CAQDAS programs.

It is unlikely that you will be using a CAQDAS for a student project, mostly because of the additional time investment it will take to become familiar with the software and associated costs (if applicable). In fact the best way to avoid spending money on qualitative data analysis software is to do your analysis by hand or using word processing or spreadsheet software. If you continue on with other qualitative research projects, it may be worth some additional study to learn more about CAQDAS tools. If you do choose to use one of these products, it won’t magically do the analysis for you. You need to be clear about what you are using the software for and how it supports your analysis plan, which will be the focus of our next chapter.

Resources to learn more about CAQDAS.

Maher et al. (2018). Ensuring rigor in qualitative data analysis: A design research approach to coding combining NVivo with traditional material methods .

Woods et al. (2016). Advancing qualitative research using qualitative data analysis software (QDAS)? Reviewing potential versus practice in published studies using ATLAS. ti and NVivo, 1994–2013 .

Zamawe, F. C. (2015). The implication of using NVivo software in qualitative data analysis: Evidence-based reflections .

As you continue to plan your research proposal, make sure to give practical thought to how you will go about collecting your qualitative data. Hopefully this chapter helped you to consider which methods are appropriate and what skills might be required to apply that particular method well. Revisit the table in section 18.3 that summarizes each of these approaches and some of the strengths and challenges associated with each of them. Collecting qualitative data can be a labor-intensive process, to be sure. However, I personally find it very rewarding. In its very forms, we are bearing witness to people’s stories and experiences.

• Artifact analysis can be particularly useful for qualitative research as a means of studying existing data; meaning we aren’t having to collect the data ourselves, but we do have to gather it. As a limitation, we don’t have any control over how the data was created, since we weren’t involved in it.
• There are many sources of existing data that we can consider for artifact analysis. Think of all the things around us that can help to tell some story! Artifact analysis may be especially appealing as a potential time saver for student researchers if you can gain permission to use existing artifacts or use artifacts that are publicly available.
• Artifact analysis still requires a systematic and premeditated approach to how you will go about extract information from your artifacts.

Here are a few questions to get you thinking about the role that you play as you gather qualitative data.

• What are your initial thoughts about qualitative data collection?
• Why might that be?
• What excites you about this process?
• What worries you about this process?
• What aspects of yourself will strengthen or enhance this process?
• What aspects of yourself may hinder or challenge this process?

Decision Point: How will you go about qualitative data collection?

• Justify your choice(s) here in relation to your research question and availability of resources at your disposal
• who will be collecting data
• what will be involved
• how will it be safely stored and organized
• how are you protecting human participants
• if you have a team, how is communication being established so everyone is “on the same page”
• how will you know you are done
• What additional information do you need to know to use this approach?

Media Attributions

• checklist © mohamed_hassan is licensed under a CC0 (Creative Commons Zero) license
• start and finish line © Andrew Hurley is licensed under a CC BY-SA (Attribution ShareAlike) license
• field notes © Tom Carmony is licensed under a CC BY-NC-ND (Attribution NonCommercial NoDerivatives) license
• group talking © Enoz is licensed under a CC BY-NC (Attribution NonCommercial) license
• children watching penguins © Amelia Beamish is licensed under a CC BY-NC (Attribution NonCommercial) license
• Swarbrick, M. (2006). A wellness approach. Psychiatric Rehabilitation Journal, 29 (4), 311. ↵
• Creswell, J. W. (2013). Chapter 7. Data collection. In J. W. Creswell, Qualitative Inquiry & Research Design: Choosing Among Five Approaches (3rd ed.), Los Angeles: Sage ↵
• Harris, M. and Fallot, R. (2001). Using trauma theory to design service systems. New Directions for Mental Health Service s. Jossey Bass; Farragher, B. and Yanosy, S. (2005). Creating a trauma-sensitive culture in residential treatment. Therapeutic Communities, 26 (1), 93-109. ↵

A form of data gathering where researchers ask individual participants to respond to a series of (mostly open-ended) questions.

A form of data gathering where researchers ask a group of participants to respond to a series of (mostly open-ended) questions.

Observation is a tool for data gathering where researchers rely on their own senses (e.g. sight, sound) to gather information on a topic.

The identity of the person providing data cannot be connected to the data provided at any time in the research process, by anyone.

For research purposes, confidentiality means that only members of the research team have access potentially identifiable information that could be associated with participant data. According to confidentiality, it is the research team's responsibility to restrict access to this information by other parties, including the public.

Fake names assigned in research to protect the identity of participants.

Numbers or a series of numbers, symbols and letters assigned in research to both organize data as it is collected, as well as protecting the identity of participants.

A process through which the researcher explains the research process, procedures, risks and benefits to a potential participant, usually through a written document, which the participant than signs, as evidence of their agreement to participate.

an administrative body established to protect the rights and welfare of human research subjects recruited to participate in research activities conducted under the auspices of the institution with which it is affiliated

For the purposes of research, authenticity means that we do not misrepresent ourselves, our interests or our research; we are genuine in our interactions with participants and other colleagues.

An approach to research that more intentionally attempts to involve community members throughout the research process compared to more traditional research methods. In addition, participatory approaches often seek some concrete, tangible change for the benefit of the community (often defined by the community).

A research journal that helps the researcher to reflect on and consider their thoughts and reactions to the research process and how it may be shaping the study

The point where gathering more data doesn't offer any new ideas or perspectives on the issue you are studying.  Reaching saturation is an indication that we can stop qualitative data collection.

A combination of two people or objects

An interview guide is a document that outlines the flow of information during your interview, including a greeting and introduction to orient your participant to the topic, your questions and any probes, and any debriefing statement you might include. If you are part of a research team, your interview guide may also include instructions for the interviewer if certain things are brought up in the interview or as general guidance.

Context is the circumstances surrounding an artifact, event, or experience.

Notes that are taken by the researcher while we are in the field, gathering data.

Expanded field notes represents the field notes that we have taken during data collection after we have had time to sit down and add details to them that we were not able to capture immediately at the point of collection.

A statement at the end of data collection (e.g. at the end of a survey or interview) that generally thanks participants and reminds them what the research was about, what it's purpose is, resources available to them if they need them, and contact information for the researcher if they have questions or concerns.

Interview that uses a very prescribed or structured approach, with a rigid set of questions that are asked very consistently each time, with little to no deviation

An interview that has a general framework for the questions that will be asked, but there is more flexibility to pursue related topics that are brought up by participants than is found in a structured interview approach.

Interviews that contain very open-ended talking prompt that we want participants to respond to, with much flexibility to follow the conversation where it leads.

starts by reading existing theories, then testing hypotheses and revising or confirming the theory

when a researcher starts with a set of observations and then moves from particular experiences to a more general set of propositions about those experiences

Emergent design is the idea that some decision in our research design will be dynamic and change as our understanding of the research question evolves as we go through the research process. This is (often) evident in qualitative research, but rare in quantitative research.

Probes a brief prompts or follow up questions that are used in qualitative interviewing to help draw out additional information on a particular question or idea.

Testing out your research materials in advance on people who are not included as participants in your study.

Someone who has the formal or informal authority to grant permission or access to a particular community.

A document that will outline the instructions for conducting your focus group, including the questions you will ask participants. It often concludes with a debriefing statement for the group, as well.

Ethnography is a qualitative research design that is used when we are attempting to learn about a culture by observing people in their natural environment.

Making qualitative observations that attempt to capture the subjects of the observation as unobtrusively as possible and with limited structure to the observation.

The analysis of documents (or other existing artifacts) as a source of data.

unprocessed data that researchers can analyze using quantitative and qualitative methods (e.g., responses to a survey or interview transcripts)

A code is a label that we place on segment of data that seems to represent the main idea of that segment.

A document that we use to keep track of and define the codes that we have identified (or are using) in our qualitative data analysis.

study publicly available information or data that has been collected by another person

in a literature review, a source that describes primary data collected and analyzed by the author, rather than only reviewing what other researchers have found

Data someone else has collected that you have permission to use in your research.

These are sites where contributing researchers can house data that other researchers can view and request permission to use

These are software tools that can aid qualitative researchers in managing, organizing and manipulating/analyzing their data.

Graduate research methods in social work Copyright © 2021 by Matthew DeCarlo, Cory Cummings, Kate Agnelli is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals

You are here

• Volume 25, Issue 1
• Critical appraisal of qualitative research: necessity, partialities and the issue of bias
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0001-5660-8224 Veronika Williams ,
• Anne-Marie Boylan ,
• http://orcid.org/0000-0003-4597-1276 David Nunan
• Nuffield Department of Primary Care Health Sciences , University of Oxford, Radcliffe Observatory Quarter , Oxford , UK
• Correspondence to Dr Veronika Williams, Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford OX2 6GG, UK; veronika.williams{at}phc.ox.ac.uk

https://doi.org/10.1136/bmjebm-2018-111132

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

• qualitative research

Introduction

Qualitative evidence allows researchers to analyse human experience and provides useful exploratory insights into experiential matters and meaning, often explaining the ‘how’ and ‘why’. As we have argued previously 1 , qualitative research has an important place within evidence-based healthcare, contributing to among other things policy on patient safety, 2 prescribing, 3 4 and understanding chronic illness. 5 Equally, it offers additional insight into quantitative studies, explaining contextual factors surrounding a successful intervention or why an intervention might have ‘failed’ or ‘succeeded’ where effect sizes cannot. It is for these reasons that the MRC strongly recommends including qualitative evaluations when developing and evaluating complex interventions. 6

Critical appraisal of qualitative research

Is it necessary.

Although the importance of qualitative research to improve health services and care is now increasingly widely supported (discussed in paper 1), the role of appraising the quality of qualitative health research is still debated. 8 10 Despite a large body of literature focusing on appraisal and rigour, 9 11–15 often referred to as ‘trustworthiness’ 16 in qualitative research, there remains debate about how to —and even whether to—critically appraise qualitative research. 8–10 17–19 However, if we are to make a case for qualitative research as integral to evidence-based healthcare, then any argument to omit a crucial element of evidence-based practice is difficult to justify. That being said, simply applying the standards of rigour used to appraise studies based on the positivist paradigm (Positivism depends on quantifiable observations to test hypotheses and assumes that the researcher is independent of the study. Research situated within a positivist paradigm isbased purely on facts and consider the world to be external and objective and is concerned with validity, reliability and generalisability as measures of rigour.) would be misplaced given the different epistemological underpinnings of the two types of data.

Given its scope and its place within health research, the robust and systematic appraisal of qualitative research to assess its trustworthiness is as paramount to its implementation in clinical practice as any other type of research. It is important to appraise different qualitative studies in relation to the specific methodology used because the methodological approach is linked to the ‘outcome’ of the research (eg, theory development, phenomenological understandings and credibility of findings). Moreover, appraisal needs to go beyond merely describing the specific details of the methods used (eg, how data were collected and analysed), with additional focus needed on the overarching research design and its appropriateness in accordance with the study remit and objectives.

Poorly conducted qualitative research has been described as ‘worthless, becomes fiction and loses its utility’. 20 However, without a deep understanding of concepts of quality in qualitative research or at least an appropriate means to assess its quality, good qualitative research also risks being dismissed, particularly in the context of evidence-based healthcare where end users may not be well versed in this paradigm.

How is appraisal currently performed?

Appraising the quality of qualitative research is not a new concept—there are a number of published appraisal tools, frameworks and checklists in existence. 21–23  An important and often overlooked point is the confusion between tools designed for appraising methodological quality and reporting guidelines designed to assess the quality of methods reporting. An example is the Consolidate Criteria for Reporting Qualitative Research (COREQ) 24 checklist, which was designed to provide standards for authors when reporting qualitative research but is often mistaken for a methods appraisal tool. 10

Broadly speaking there are two types of critical appraisal approaches for qualitative research: checklists and frameworks. Checklists have often been criticised for confusing quality in qualitative research with ‘technical fixes’ 21 25 , resulting in the erroneous prioritisation of particular aspects of methodological processes over others (eg, multiple coding and triangulation). It could be argued that a checklist approach adopts the positivist paradigm, where the focus is on objectively assessing ‘quality’ where the assumptions is that the researcher is independent of the research conducted. This may result in the application of quantitative understandings of bias in order to judge aspects of recruitment, sampling, data collection and analysis in qualitative research papers. One of the most widely used appraisal tools is the Critical Appraisal Skills Programme (CASP) 26 and along with the JBI QARI (Joanna Briggs Institute Qualitative Assessment and Assessment Instrument) 27 presents examples which tend to mimic the quantitative approach to appraisal. The CASP qualitative tool follows that of other CASP appraisal tools for quantitative research designs developed in the 1990s. The similarities are therefore unsurprising given the status of qualitative research at that time.

Frameworks focus on the overarching concepts of quality in qualitative research, including transparency, reflexivity, dependability and transferability (see box 1 ). 11–13 15 16 20 28 However, unless the reader is familiar with these concepts—their meaning and impact, and how to interpret them—they will have difficulty applying them when critically appraising a paper.

The main issue concerning currently available checklist and framework appraisal methods is that they take a broad brush approach to ‘qualitative’ research as whole, with few, if any, sufficiently differentiating between the different methodological approaches (eg, Grounded Theory, Interpretative Phenomenology, Discourse Analysis) nor different methods of data collection (interviewing, focus groups and observations). In this sense, it is akin to taking the entire field of ‘quantitative’ study designs and applying a single method or tool for their quality appraisal. In the case of qualitative research, checklists, therefore, offer only a blunt and arguably ineffective tool and potentially promote an incomplete understanding of good ‘quality’ in qualitative research. Likewise, current framework methods do not take into account how concepts differ in their application across the variety of qualitative approaches and, like checklists, they also do not differentiate between different qualitative methodologies.

On the need for specific appraisal tools

Current approaches to the appraisal of the methodological rigour of the differing types of qualitative research converge towards checklists or frameworks. More importantly, the current tools do not explicitly acknowledge the prejudices that may be present in the different types of qualitative research.

Concepts of rigour or trustworthiness within qualitative research 31

Transferability: the extent to which the presented study allows readers to make connections between the study’s data and wider community settings, ie, transfer conceptual findings to other contexts.

Credibility: extent to which a research account is believable and appropriate, particularly in relation to the stories told by participants and the interpretations made by the researcher.

Reflexivity: refers to the researchers’ engagement of continuous examination and explanation of how they have influenced a research project from choosing a research question to sampling, data collection, analysis and interpretation of data.

Transparency: making explicit the whole research process from sampling strategies, data collection to analysis. The rationale for decisions made is as important as the decisions themselves.

However, we often talk about these concepts in general terms, and it might be helpful to give some explicit examples of how the ‘technical processes’ affect these, for example, partialities related to:

Selection: recruiting participants via gatekeepers, such as healthcare professionals or clinicians, who may select them based on whether they believe them to be ‘good’ participants for interviews/focus groups.

Data collection: poor interview guide with closed questions which encourage yes/no answers and/leading questions.

Reflexivity and transparency: where researchers may focus their analysis on preconceived ideas rather than ground their analysis in the data and do not reflect on the impact of this in a transparent way.

The lack of tailored, method-specific appraisal tools has potentially contributed to the poor uptake and use of qualitative research for informing evidence-based decision making. To improve this situation, we propose the need for more robust quality appraisal tools that explicitly encompass both the core design aspects of all qualitative research (sampling/data collection/analysis) but also considered the specific partialities that can be presented with different methodological approaches. Such tools might draw on the strengths of current frameworks and checklists while providing users with sufficient understanding of concepts of rigour in relation to the different types of qualitative methods. We provide an outline of such tools in the third and final paper in this series.

As qualitative research becomes ever more embedded in health science research, and in order for that research to have better impact on healthcare decisions, we need to rethink critical appraisal and develop tools that allow differentiated evaluations of the myriad of qualitative methodological approaches rather than continuing to treat qualitative research as a single unified approach.

• Williams V ,
• Boylan AM ,
• Lingard L ,
• Orser B , et al
• Brawn R , et al
• Van Royen P ,
• Vermeire E , et al
• Barker M , et al
• McGannon KR
• Dixon-Woods M ,
• Agarwal S , et al
• Greenhalgh T ,
• Dennison L ,
• Morrison L ,
• Conway G , et al
• Barrett M ,
• Mayan M , et al
• Lockwood C ,
• Santiago-Delefosse M ,
• Bruchez C , et al
• Sainsbury P ,
• ↵ CASP (Critical Appraisal Skills Programme). date unknown . http://www.phru.nhs.uk/Pages/PHD/CASP.htm .
• ↵ The Joanna Briggs Institute . JBI QARI Critical appraisal checklist for interpretive & critical research . Adelaide : The Joanna Briggs Institute , 2014 .
• Stephens J ,

Contributors VW and DN: conceived the idea for this article. VW: wrote the first draft. AMB and DN: contributed to the final draft. All authors approve the submitted article.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Correction notice This article has been updated since its original publication to include a new reference (reference 1.)

Read the full text or download the PDF: