scientific problem solving scientific inquiry answers

What is the Scientific Method: How does it work and why is it important?

The scientific method is a systematic process involving steps like defining questions, forming hypotheses, conducting experiments, and analyzing data. It minimizes biases and enables replicable research, leading to groundbreaking discoveries like Einstein's theory of relativity, penicillin, and the structure of DNA. This ongoing approach promotes reason, evidence, and the pursuit of truth in science.

Updated on November 18, 2023

Beginning in elementary school, we are exposed to the scientific method and taught how to put it into practice. As a tool for learning, it prepares children to think logically and use reasoning when seeking answers to questions.

Rather than jumping to conclusions, the scientific method gives us a recipe for exploring the world through observation and trial and error. We use it regularly, sometimes knowingly in academics or research, and sometimes subconsciously in our daily lives.

In this article we will refresh our memories on the particulars of the scientific method, discussing where it comes from, which elements comprise it, and how it is put into practice. Then, we will consider the importance of the scientific method, who uses it and under what circumstances.

What is the scientific method?

The scientific method is a dynamic process that involves objectively investigating questions through observation and experimentation . Applicable to all scientific disciplines, this systematic approach to answering questions is more accurately described as a flexible set of principles than as a fixed series of steps.

The following representations of the scientific method illustrate how it can be both condensed into broad categories and also expanded to reveal more and more details of the process. These graphics capture the adaptability that makes this concept universally valuable as it is relevant and accessible not only across age groups and educational levels but also within various contexts.

Steps in the scientific method

While the scientific method is versatile in form and function, it encompasses a collection of principles that create a logical progression to the process of problem solving:

• Define a question : Constructing a clear and precise problem statement that identifies the main question or goal of the investigation is the first step. The wording must lend itself to experimentation by posing a question that is both testable and measurable.
• Gather information and resources : Researching the topic in question to find out what is already known and what types of related questions others are asking is the next step in this process. This background information is vital to gaining a full understanding of the subject and in determining the best design for experiments. 
• Form a hypothesis : Composing a concise statement that identifies specific variables and potential results, which can then be tested, is a crucial step that must be completed before any experimentation. An imperfection in the composition of a hypothesis can result in weaknesses to the entire design of an experiment.
• Perform the experiments : Testing the hypothesis by performing replicable experiments and collecting resultant data is another fundamental step of the scientific method. By controlling some elements of an experiment while purposely manipulating others, cause and effect relationships are established.
• Analyze the data : Interpreting the experimental process and results by recognizing trends in the data is a necessary step for comprehending its meaning and supporting the conclusions. Drawing inferences through this systematic process lends substantive evidence for either supporting or rejecting the hypothesis.
• Report the results : Sharing the outcomes of an experiment, through an essay, presentation, graphic, or journal article, is often regarded as a final step in this process. Detailing the project's design, methods, and results not only promotes transparency and replicability but also adds to the body of knowledge for future research.
• Retest the hypothesis : Repeating experiments to see if a hypothesis holds up in all cases is a step that is manifested through varying scenarios. Sometimes a researcher immediately checks their own work or replicates it at a future time, or another researcher will repeat the experiments to further test the hypothesis.

Where did the scientific method come from?

Oftentimes, ancient peoples attempted to answer questions about the unknown by:

• Making simple observations
• Discussing the possibilities with others deemed worthy of a debate
• Drawing conclusions based on dominant opinions and preexisting beliefs

For example, take Greek and Roman mythology. Myths were used to explain everything from the seasons and stars to the sun and death itself.

However, as societies began to grow through advancements in agriculture and language, ancient civilizations like Egypt and Babylonia shifted to a more rational analysis for understanding the natural world. They increasingly employed empirical methods of observation and experimentation that would one day evolve into the scientific method . 

In the 4th century, Aristotle, considered the Father of Science by many, suggested these elements , which closely resemble the contemporary scientific method, as part of his approach for conducting science:

• Study what others have written about the subject.
• Look for the general consensus about the subject.
• Perform a systematic study of everything even partially related to the topic.

By continuing to emphasize systematic observation and controlled experiments, scholars such as Al-Kindi and Ibn al-Haytham helped expand this concept throughout the Islamic Golden Age . 

In his 1620 treatise, Novum Organum , Sir Francis Bacon codified the scientific method, arguing not only that hypotheses must be tested through experiments but also that the results must be replicated to establish a truth. Coming at the height of the Scientific Revolution, this text made the scientific method accessible to European thinkers like Galileo and Isaac Newton who then put the method into practice.

As science modernized in the 19th century, the scientific method became more formalized, leading to significant breakthroughs in fields such as evolution and germ theory. Today, it continues to evolve, underpinning scientific progress in diverse areas like quantum mechanics, genetics, and artificial intelligence.

Why is the scientific method important?

The history of the scientific method illustrates how the concept developed out of a need to find objective answers to scientific questions by overcoming biases based on fear, religion, power, and cultural norms. This still holds true today.

By implementing this standardized approach to conducting experiments, the impacts of researchers’ personal opinions and preconceived notions are minimized. The organized manner of the scientific method prevents these and other mistakes while promoting the replicability and transparency necessary for solid scientific research.

The importance of the scientific method is best observed through its successes, for example: 

• “ Albert Einstein stands out among modern physicists as the scientist who not only formulated a theory of revolutionary significance but also had the genius to reflect in a conscious and technical way on the scientific method he was using.” Devising a hypothesis based on the prevailing understanding of Newtonian physics eventually led Einstein to devise the theory of general relativity .
• Howard Florey “Perhaps the most useful lesson which has come out of the work on penicillin has been the demonstration that success in this field depends on the development and coordinated use of technical methods.” After discovering a mold that prevented the growth of Staphylococcus bacteria, Dr. Alexander Flemimg designed experiments to identify and reproduce it in the lab, thus leading to the development of penicillin .
• James D. Watson “Every time you understand something, religion becomes less likely. Only with the discovery of the double helix and the ensuing genetic revolution have we had grounds for thinking that the powers held traditionally to be the exclusive property of the gods might one day be ours. . . .” By using wire models to conceive a structure for DNA, Watson and Crick crafted a hypothesis for testing combinations of amino acids, X-ray diffraction images, and the current research in atomic physics, resulting in the discovery of DNA’s double helix structure .

Final thoughts

As the cases exemplify, the scientific method is never truly completed, but rather started and restarted. It gave these researchers a structured process that was easily replicated, modified, and built upon. 

While the scientific method may “end” in one context, it never literally ends. When a hypothesis, design, methods, and experiments are revisited, the scientific method simply picks up where it left off. Each time a researcher builds upon previous knowledge, the scientific method is restored with the pieces of past efforts.

By guiding researchers towards objective results based on transparency and reproducibility, the scientific method acts as a defense against bias, superstition, and preconceived notions. As we embrace the scientific method's enduring principles, we ensure that our quest for knowledge remains firmly rooted in reason, evidence, and the pursuit of truth.

The AJE Team

See our "Privacy Policy"

Choose Your Test

• Search Blogs By Category
• College Admissions
• AP and IB Exams
• GPA and Coursework

The 6 Scientific Method Steps and How to Use Them

General Education

When you’re faced with a scientific problem, solving it can seem like an impossible prospect. There are so many possible explanations for everything we see and experience—how can you possibly make sense of them all? Science has a simple answer: the scientific method.

The scientific method is a method of asking and answering questions about the world. These guiding principles give scientists a model to work through when trying to understand the world, but where did that model come from, and how does it work?

In this article, we’ll define the scientific method, discuss its long history, and cover each of the scientific method steps in detail.

What Is the Scientific Method?

At its most basic, the scientific method is a procedure for conducting scientific experiments. It’s a set model that scientists in a variety of fields can follow, going from initial observation to conclusion in a loose but concrete format.

The number of steps varies, but the process begins with an observation, progresses through an experiment, and concludes with analysis and sharing data. One of the most important pieces to the scientific method is skepticism —the goal is to find truth, not to confirm a particular thought. That requires reevaluation and repeated experimentation, as well as examining your thinking through rigorous study.

There are in fact multiple scientific methods, as the basic structure can be easily modified.  The one we typically learn about in school is the basic method, based in logic and problem solving, typically used in “hard” science fields like biology, chemistry, and physics. It may vary in other fields, such as psychology, but the basic premise of making observations, testing, and continuing to improve a theory from the results remain the same.

The History of the Scientific Method

The scientific method as we know it today is based on thousands of years of scientific study. Its development goes all the way back to ancient Mesopotamia, Greece, and India.

The Ancient World

In ancient Greece, Aristotle devised an inductive-deductive process , which weighs broad generalizations from data against conclusions reached by narrowing down possibilities from a general statement. However, he favored deductive reasoning, as it identifies causes, which he saw as more important.

Aristotle wrote a great deal about logic and many of his ideas about reasoning echo those found in the modern scientific method, such as ignoring circular evidence and limiting the number of middle terms between the beginning of an experiment and the end. Though his model isn’t the one that we use today, the reliance on logic and thorough testing are still key parts of science today.

The Middle Ages

The next big step toward the development of the modern scientific method came in the Middle Ages, particularly in the Islamic world. Ibn al-Haytham, a physicist from what we now know as Iraq, developed a method of testing, observing, and deducing for his research on vision. al-Haytham was critical of Aristotle’s lack of inductive reasoning, which played an important role in his own research.

Other scientists, including Abū Rayhān al-Bīrūnī, Ibn Sina, and Robert Grosseteste also developed models of scientific reasoning to test their own theories. Though they frequently disagreed with one another and Aristotle, those disagreements and refinements of their methods led to the scientific method we have today.

Following those major developments, particularly Grosseteste’s work, Roger Bacon developed his own cycle of observation (seeing that something occurs), hypothesis (making a guess about why that thing occurs), experimentation (testing that the thing occurs), and verification (an outside person ensuring that the result of the experiment is consistent).

After joining the Franciscan Order, Bacon was granted a special commission to write about science; typically, Friars were not allowed to write books or pamphlets. With this commission, Bacon outlined important tenets of the scientific method, including causes of error, methods of knowledge, and the differences between speculative and experimental science. He also used his own principles to investigate the causes of a rainbow, demonstrating the method’s effectiveness.

Scientific Revolution

Throughout the Renaissance, more great thinkers became involved in devising a thorough, rigorous method of scientific study. Francis Bacon brought inductive reasoning further into the method, whereas Descartes argued that the laws of the universe meant that deductive reasoning was sufficient. Galileo’s research was also inductive reasoning-heavy, as he believed that researchers could not account for every possible variable; therefore, repetition was necessary to eliminate faulty hypotheses and experiments.

All of this led to the birth of the Scientific Revolution , which took place during the sixteenth and seventeenth centuries. In 1660, a group of philosophers and physicians joined together to work on scientific advancement. After approval from England’s crown , the group became known as the Royal Society, which helped create a thriving scientific community and an early academic journal to help introduce rigorous study and peer review.

Previous generations of scientists had touched on the importance of induction and deduction, but Sir Isaac Newton proposed that both were equally important. This contribution helped establish the importance of multiple kinds of reasoning, leading to more rigorous study.

As science began to splinter into separate areas of study, it became necessary to define different methods for different fields. Karl Popper was a leader in this area—he established that science could be subject to error, sometimes intentionally. This was particularly tricky for “soft” sciences like psychology and social sciences, which require different methods. Popper’s theories furthered the divide between sciences like psychology and “hard” sciences like chemistry or physics.

Paul Feyerabend argued that Popper’s methods were too restrictive for certain fields, and followed a less restrictive method hinged on “anything goes,” as great scientists had made discoveries without the Scientific Method. Feyerabend suggested that throughout history scientists had adapted their methods as necessary, and that sometimes it would be necessary to break the rules. This approach suited social and behavioral scientists particularly well, leading to a more diverse range of models for scientists in multiple fields to use.

The Scientific Method Steps

Though different fields may have variations on the model, the basic scientific method is as follows:

#1: Make Observations 

Notice something, such as the air temperature during the winter, what happens when ice cream melts, or how your plants behave when you forget to water them.

#2: Ask a Question

Turn your observation into a question. Why is the temperature lower during the winter? Why does my ice cream melt? Why does my toast always fall butter-side down?

This step can also include doing some research. You may be able to find answers to these questions already, but you can still test them!

#3: Make a Hypothesis

A hypothesis is an educated guess of the answer to your question. Why does your toast always fall butter-side down? Maybe it’s because the butter makes that side of the bread heavier.

A good hypothesis leads to a prediction that you can test, phrased as an if/then statement. In this case, we can pick something like, “If toast is buttered, then it will hit the ground butter-first.”

#4: Experiment

Your experiment is designed to test whether your predication about what will happen is true. A good experiment will test one variable at a time —for example, we’re trying to test whether butter weighs down one side of toast, making it more likely to hit the ground first.

The unbuttered toast is our control variable. If we determine the chance that a slice of unbuttered toast, marked with a dot, will hit the ground on a particular side, we can compare those results to our buttered toast to see if there’s a correlation between the presence of butter and which way the toast falls.

If we decided not to toast the bread, that would be introducing a new question—whether or not toasting the bread has any impact on how it falls. Since that’s not part of our test, we’ll stick with determining whether the presence of butter has any impact on which side hits the ground first.

#5: Analyze Data

After our experiment, we discover that both buttered toast and unbuttered toast have a 50/50 chance of hitting the ground on the buttered or marked side when dropped from a consistent height, straight down. It looks like our hypothesis was incorrect—it’s not the butter that makes the toast hit the ground in a particular way, so it must be something else.

Since we didn’t get the desired result, it’s back to the drawing board. Our hypothesis wasn’t correct, so we’ll need to start fresh. Now that you think about it, your toast seems to hit the ground butter-first when it slides off your plate, not when you drop it from a consistent height. That can be the basis for your new experiment.

#6: Communicate Your Results

Good science needs verification. Your experiment should be replicable by other people, so you can put together a report about how you ran your experiment to see if other peoples’ findings are consistent with yours.

This may be useful for class or a science fair. Professional scientists may publish their findings in scientific journals, where other scientists can read and attempt their own versions of the same experiments. Being part of a scientific community helps your experiments be stronger because other people can see if there are flaws in your approach—such as if you tested with different kinds of bread, or sometimes used peanut butter instead of butter—that can lead you closer to a good answer.

A Scientific Method Example: Falling Toast

We’ve run through a quick recap of the scientific method steps, but let’s look a little deeper by trying again to figure out why toast so often falls butter side down.

#1: Make Observations

At the end of our last experiment, where we learned that butter doesn’t actually make toast more likely to hit the ground on that side, we remembered that the times when our toast hits the ground butter side first are usually when it’s falling off a plate.

The easiest question we can ask is, “Why is that?”

We can actually search this online and find a pretty detailed answer as to why this is true. But we’re budding scientists—we want to see it in action and verify it for ourselves! After all, good science should be replicable, and we have all the tools we need to test out what’s really going on.

Why do we think that buttered toast hits the ground butter-first? We know it’s not because it’s heavier, so we can strike that out. Maybe it’s because of the shape of our plate?

That’s something we can test. We’ll phrase our hypothesis as, “If my toast slides off my plate, then it will fall butter-side down.”

Just seeing that toast falls off a plate butter-side down isn’t enough for us. We want to know why, so we’re going to take things a step further—we’ll set up a slow-motion camera to capture what happens as the toast slides off the plate.

We’ll run the test ten times, each time tilting the same plate until the toast slides off. We’ll make note of each time the butter side lands first and see what’s happening on the video so we can see what’s going on.

When we review the footage, we’ll likely notice that the bread starts to flip when it slides off the edge, changing how it falls in a way that didn’t happen when we dropped it ourselves.

That answers our question, but it’s not the complete picture —how do other plates affect how often toast hits the ground butter-first? What if the toast is already butter-side down when it falls? These are things we can test in further experiments with new hypotheses!

Now that we have results, we can share them with others who can verify our results. As mentioned above, being part of the scientific community can lead to better results. If your results were wildly different from the established thinking about buttered toast, that might be cause for reevaluation. If they’re the same, they might lead others to make new discoveries about buttered toast. At the very least, you have a cool experiment you can share with your friends!

Key Scientific Method Tips

Though science can be complex, the benefit of the scientific method is that it gives you an easy-to-follow means of thinking about why and how things happen. To use it effectively, keep these things in mind!

Don’t Worry About Proving Your Hypothesis

One of the important things to remember about the scientific method is that it’s not necessarily meant to prove your hypothesis right. It’s great if you do manage to guess the reason for something right the first time, but the ultimate goal of an experiment is to find the true reason for your observation to occur, not to prove your hypothesis right.

Good science sometimes means that you’re wrong. That’s not a bad thing—a well-designed experiment with an unanticipated result can be just as revealing, if not more, than an experiment that confirms your hypothesis.

Be Prepared to Try Again

If the data from your experiment doesn’t match your hypothesis, that’s not a bad thing. You’ve eliminated one possible explanation, which brings you one step closer to discovering the truth.

The scientific method isn’t something you’re meant to do exactly once to prove a point. It’s meant to be repeated and adapted to bring you closer to a solution. Even if you can demonstrate truth in your hypothesis, a good scientist will run an experiment again to be sure that the results are replicable. You can even tweak a successful hypothesis to test another factor, such as if we redid our buttered toast experiment to find out whether different kinds of plates affect whether or not the toast falls butter-first. The more we test our hypothesis, the stronger it becomes!

What’s Next?

Want to learn more about the scientific method? These important high school science classes will no doubt cover it in a variety of different contexts.

Test your ability to follow the scientific method using these at-home science experiments for kids !

Need some proof that science is fun? Try making slime

Trending Now

How to Get Into Harvard and the Ivy League

How to Get a Perfect 4.0 GPA

How to Write an Amazing College Essay

What Exactly Are Colleges Looking For?

ACT vs. SAT: Which Test Should You Take?

When should you take the SAT or ACT?

Get Your Free

Find Your Target SAT Score

Free Complete Official SAT Practice Tests

How to Get a Perfect SAT Score, by an Expert Full Scorer

Score 800 on SAT Math

Score 800 on SAT Reading and Writing

How to Improve Your Low SAT Score

Score 600 on SAT Math

Score 600 on SAT Reading and Writing

Find Your Target ACT Score

Complete Official Free ACT Practice Tests

How to Get a Perfect ACT Score, by a 36 Full Scorer

Get a 36 on ACT English

Get a 36 on ACT Math

Get a 36 on ACT Reading

Get a 36 on ACT Science

How to Improve Your Low ACT Score

Get a 24 on ACT English

Get a 24 on ACT Math

Get a 24 on ACT Reading

Get a 24 on ACT Science

Stay Informed

Get the latest articles and test prep tips!

Melissa Brinks graduated from the University of Washington in 2014 with a Bachelor's in English with a creative writing emphasis. She has spent several years tutoring K-12 students in many subjects, including in SAT prep, to help them prepare for their college education.

Ask a Question Below

Have any questions about this article or other topics? Ask below and we'll reply!

Module 1: Introduction to Biology

Scientific inquiry, learning outcomes.

• Describe “scientific inquiry” and identify its scope of coverage

One thing is common to all forms of science: an ultimate goal “to know.” Curiosity and inquiry are the driving forces for the development of science. Scientists seek to understand the world and the way it operates. Two methods of logical thinking are used: inductive reasoning and deductive reasoning.

Inductive reasoning is a form of logical thinking that analyzes trends or relationships in  data to arrive at a general conclusion. A scientist makes observations and records them. These data can be qualitative (descriptive) or quantitative (consisting of numbers), and the raw data can be supplemented with drawings, pictures, photos, or videos. From many observations, a scientist can draw conclusions based on evidence. In other words, inductive reasoning involves making generalizations from careful observation and the analysis of a large amount of individual data points. Generalizations arrived at through inductive reasoning are not always correct.

Deductive reasoning  is another form of logical thinking that begins from a general principle or law and applies it to a specific circumstance to predict specific results.  From a set of general principles, a scientist can extrapolate and predict specific results that will always be correct as long as the general principles they start from are correct.

Deductive reasoning and inductive reasoning move in opposite directions – inductive reasoning goes from individual observations to broad generalizations while deductive reasoning goes from general principles to specific decisions or predictions.

Both types of logical thinking are related to the two main pathways of scientific study: descriptive science and hypothesis-based science. Descriptive science (or discovery science) aims to observe, explore, and discover, while h ypothesis-based science   begins with a specific question or problem and a potential answer or solution that can be tested. Inductive reasoning is used most often in descriptive science, while deductive reasoning is used most often in hypothesis-based science. The boundary between these two forms of study is often blurred, because most scientific endeavors combine both approaches. Observations lead to questions, questions lead to forming a hypothesis as a possible answer to those questions, and then the hypothesis is tested. Thus, descriptive science and hypothesis-based science are in continuous dialogue.

Hypothesis Testing

Figure 1. Sir Francis Bacon is credited with being the first to document the scientific method.

Biologists study the living world by posing questions about it and seeking science-based responses. This approach is common to other sciences as well and is often referred to as the scientific method. The scientific method was used even in ancient times, but it was first documented by England’s Sir Francis Bacon (1561–1626) (Figure 1), who set up inductive methods for scientific inquiry. The scientific method is not exclusively used by biologists but can be applied to almost anything as a logical problem-solving method.

The scientific process typically starts with an observation (often a problem to be solved) that leads to a question. Let’s think about a simple problem that starts with an observation and apply the scientific method to solve the problem. One Monday morning, a student arrives at class and quickly discovers that the classroom is too warm. That is an observation that also describes a problem: the classroom is too warm. The student then asks a question: “Why is the classroom so warm?”

Recall that a hypothesis is a suggested explanation that can be tested. To solve a problem, several hypotheses may be proposed. For example, one hypothesis might be, “The classroom is warm because no one turned on the air conditioning.” But there could be other responses to the question, and therefore other hypotheses may be proposed. A second hypothesis might be, “The classroom is warm because there is a power failure, and so the air conditioning doesn’t work.”

Once a hypothesis has been selected, a prediction may be made. A prediction is similar to a hypothesis but it typically has the format “If . . . then . . . .” For example, the prediction for the first hypothesis might be, “ If the student turns on the air conditioning, then the classroom will no longer be too warm.”

A hypothesis must be testable to ensure that it is valid. For example, a hypothesis that depends on what a bear thinks is not testable, because it can never be known what a bear thinks. It should also be falsifiable , meaning that it can be disproven by experimental results. An example of an unfalsifiable hypothesis is “Botticelli’s Birth of Venus is beautiful.” There is no experiment that might show this statement to be false. To test a hypothesis, a researcher will conduct one or more experiments designed to eliminate one or more of the hypotheses. This is important. A hypothesis can be disproven, or eliminated, but it can never be proven. Science does not deal in proofs like mathematics. If an experiment fails to disprove a hypothesis, then we find support for that explanation, but this is not to say that down the road a better explanation will not be found, or a more carefully designed experiment will be found to falsify the hypothesis.

Scientific inquiry has not displaced faith, intuition, and dreams. These traditions and ways of knowing have emotional value and provide moral guidance to many people. But hunches, feelings, deep convictions, old traditions, or dreams cannot be accepted directly as scientifically valid. Instead, science limits itself to ideas that can be tested through verifiable observations. Supernatural claims that events are caused by ghosts, devils, God, or other spiritual entities cannot be tested in this way.

Practice Question

Your friend sees this image of a circle of mushrooms and excitedly tells you it was caused by fairies dancing in a circle on the grass the night before. Can your friend’s explanation be studied using the process of science?

Each experiment will have one or more variables and one or more controls. A variable is any part of the experiment that can vary or change during the experiment. A control is a part of the experiment that does not change. Look for the variables and controls in the example that follows. As a simple example, an experiment might be conducted to test the hypothesis that phosphate limits the growth of algae in freshwater ponds. A series of artificial ponds are filled with water and half of them are treated by adding phosphate each week, while the other half are treated by adding a salt that is known not to be used by algae. The variable here is the phosphate (or lack of phosphate), the experimental or treatment cases are the ponds with added phosphate and the control ponds are those with something inert added, such as the salt. Just adding something is also a control against the possibility that adding extra matter to the pond has an effect. If the treated ponds show lesser growth of algae, then we have found support for our hypothesis. If they do not, then we reject our hypothesis. Be aware that rejecting one hypothesis does not determine whether or not the other hypotheses can be accepted; it simply eliminates one hypothesis that is not valid (Figure 2). Using the scientific method, the hypotheses that are inconsistent with experimental data are rejected.

Figure 2. The scientific method is a series of defined steps that include experiments and careful observation. If a hypothesis is not supported by data, a new hypothesis can be proposed.

In the example below, the scientific method is used to solve an everyday problem. Which part in the example below is the hypothesis? Which is the prediction? Based on the results of the experiment, is the hypothesis supported? If it is not supported, propose some alternative hypotheses.

• My toaster doesn’t toast my bread.
• Why doesn’t my toaster work?
• There is something wrong with the electrical outlet.
• If something is wrong with the outlet, my coffeemaker also won’t work when plugged into it.
• I plug my coffeemaker into the outlet.
• My coffeemaker works.

In practice, the scientific method is not as rigid and structured as it might at first appear. Sometimes an experiment leads to conclusions that favor a change in approach; often, an experiment brings entirely new scientific questions to the puzzle. Many times, science does not operate in a linear fashion; instead, scientists continually draw inferences and make generalizations, finding patterns as their research proceeds. Scientific reasoning is more complex than the scientific method alone suggests.

• Scientific Inquiry. Provided by : Open Learning Initiative. Located at : https://oli.cmu.edu/jcourse/workbook/activity/page?context=434a5c2680020ca6017c03488572e0f8 . Project : Introduction to Biology (Open + Free). License : CC BY-NC-SA: Attribution-NonCommercial-ShareAlike
• The Process of Science Ch 1.2 Exercises. Authored by : Charles Molnar and Michelle Nakano. Provided by : BC Campus. Located at : https://opentextbc.ca/biology/h5p-listing/ . License : CC BY: Attribution

• History & Society
• Science & Tech
• Biographies
• Animals & Nature
• Geography & Travel
• Arts & Culture
• Games & Quizzes
• On This Day
• One Good Fact
• New Articles
• Lifestyles & Social Issues
• Philosophy & Religion
• Politics, Law & Government
• World History
• Health & Medicine
• Browse Biographies
• Birds, Reptiles & Other Vertebrates
• Bugs, Mollusks & Other Invertebrates
• Environment
• Fossils & Geologic Time
• Entertainment & Pop Culture
• Sports & Recreation
• Visual Arts
• Demystified
• Image Galleries
• Infographics
• Top Questions
• Britannica Kids
• Saving Earth
• Space Next 50
• Student Center

scientific method

Our editors will review what you’ve submitted and determine whether to revise the article.

• University of Nevada, Reno - College of Agriculture, Biotechnology and Natural Resources Extension - The Scientific Method
• World History Encyclopedia - Scientific Method
• LiveScience - What Is Science?
• Verywell Mind - Scientific Method Steps in Psychology Research
• WebMD - What is the Scientific Method?
• Chemistry LibreTexts - The Scientific Method
• National Center for Biotechnology Information - PubMed Central - Redefining the scientific method: as the use of sophisticated scientific methods that extend our mind
• Khan Academy - The scientific method
• Simply Psychology - What are the steps in the Scientific Method?
• Stanford Encyclopedia of Philosophy - Scientific Method

scientific method , mathematical and experimental technique employed in the sciences . More specifically, it is the technique used in the construction and testing of a scientific hypothesis .

The process of observing, asking questions, and seeking answers through tests and experiments is not unique to any one field of science. In fact, the scientific method is applied broadly in science, across many different fields. Many empirical sciences, especially the social sciences , use mathematical tools borrowed from probability theory and statistics , together with outgrowths of these, such as decision theory , game theory , utility theory, and operations research . Philosophers of science have addressed general methodological problems, such as the nature of scientific explanation and the justification of induction .

The scientific method is critical to the development of scientific theories , which explain empirical (experiential) laws in a scientifically rational manner. In a typical application of the scientific method, a researcher develops a hypothesis , tests it through various means, and then modifies the hypothesis on the basis of the outcome of the tests and experiments. The modified hypothesis is then retested, further modified, and tested again, until it becomes consistent with observed phenomena and testing outcomes. In this way, hypotheses serve as tools by which scientists gather data. From that data and the many different scientific investigations undertaken to explore hypotheses, scientists are able to develop broad general explanations, or scientific theories.

See also Mill’s methods ; hypothetico-deductive method .

Scientific Method

Illustration by J.R. Bee. ThoughtCo. 

• Cell Biology
• Weather & Climate
• B.A., Biology, Emory University
• A.S., Nursing, Chattahoochee Technical College

The scientific method is a series of steps followed by scientific investigators to answer specific questions about the natural world. It involves making observations, formulating a hypothesis , and conducting scientific experiments . Scientific inquiry starts with an observation followed by the formulation of a question about what has been observed. The steps of the scientific method are as follows:

Observation

The first step of the scientific method involves making an observation about something that interests you. This is very important if you are doing a science project because you want your project to be focused on something that will hold your attention. Your observation can be on anything from plant movement to animal behavior, as long as it is something you really want to know more about.​ This is where you come up with the idea for your science project.

Once you've made your observation, you must formulate a question about what you have observed. Your question should tell what it is that you are trying to discover or accomplish in your experiment. When stating your question you should be as specific as possible.​ For example, if you are doing a project on plants , you may want to know how plants interact with microbes. Your question may be: Do plant spices inhibit bacterial growth ?

The hypothesis is a key component of the scientific process. A hypothesis is an idea that is suggested as an explanation for a natural event, a particular experience, or a specific condition that can be tested through definable experimentation. It states the purpose of your experiment, the variables used, and the predicted outcome of your experiment. It is important to note that a hypothesis must be testable. That means that you should be able to test your hypothesis through experimentation .​ Your hypothesis must either be supported or falsified by your experiment. An example of a good hypothesis is: If there is a relation between listening to music and heart rate, then listening to music will cause a person's resting heart rate to either increase or decrease.

Once you've developed a hypothesis, you must design and conduct an experiment that will test it. You should develop a procedure that states very clearly how you plan to conduct your experiment. It is important that you include and identify a controlled variable or dependent variable in your procedure. Controls allow us to test a single variable in an experiment because they are unchanged. We can then make observations and comparisons between our controls and our independent variables (things that change in the experiment) to develop an accurate conclusion.​

The results are where you report what happened in the experiment. That includes detailing all observations and data made during your experiment. Most people find it easier to visualize the data by charting or graphing the information.​

The final step of the scientific method is developing a conclusion. This is where all of the results from the experiment are analyzed and a determination is reached about the hypothesis. Did the experiment support or reject your hypothesis? If your hypothesis was supported, great. If not, repeat the experiment or think of ways to improve your procedure.

• Biology Prefixes and Suffixes: chrom- or chromo-
• Biology Prefixes and Suffixes: proto-
• 6 Things You Should Know About Biological Evolution
• Biology Prefixes and Suffixes: Aer- or Aero-
• Taxonomy and Organism Classification
• Homeostasis
• Biology Prefixes and Suffixes: diplo-
• The Biology Suffix -lysis
• Biology Prefixes and Suffixes Index
• Biology Prefixes and Suffixes: tel- or telo-
• Parasitism: Definition and Examples
• Biology Prefixes and Suffixes: Erythr- or Erythro-
• Biology Prefixes and Suffixes: ana-
• Biology Prefixes and Suffixes: phago- or phag-
• Biology Prefixes and Suffixes: -phyll or -phyl
• Biology Prefixes and Suffixes: haplo-

If you're seeing this message, it means we're having trouble loading external resources on our website.

If you're behind a web filter, please make sure that the domains *.kastatic.org and *.kasandbox.org are unblocked.

To log in and use all the features of Khan Academy, please enable JavaScript in your browser.

Biology archive

Course: biology archive   >   unit 1, the scientific method.

• Controlled experiments
• The scientific method and experimental design

Want to join the conversation?

• Upvote Button navigates to signup page
• Downvote Button navigates to signup page
• Flag Button navigates to signup page

Video transcript

1.2 The Scientific Methods

Section learning objectives.

By the end of this section, you will be able to do the following:

• Explain how the methods of science are used to make scientific discoveries
• Define a scientific model and describe examples of physical and mathematical models used in physics
• Compare and contrast hypothesis, theory, and law

Teacher Support

The learning objectives in this section will help your students master the following standards:

• (A) know the definition of science and understand that it has limitations, as specified in subsection (b)(2) of this section;
• (B) know that scientific hypotheses are tentative and testable statements that must be capable of being supported or not supported by observational evidence. Hypotheses of durable explanatory power which have been tested over a wide variety of conditions are incorporated into theories;
• (C) know that scientific theories are based on natural and physical phenomena and are capable of being tested by multiple independent researchers. Unlike hypotheses, scientific theories are well-established and highly-reliable explanations, but may be subject to change as new areas of science and new technologies are developed;
• (D) distinguish between scientific hypotheses and scientific theories.

Section Key Terms

[OL] Pre-assessment for this section could involve students sharing or writing down an anecdote about when they used the methods of science. Then, students could label their thought processes in their anecdote with the appropriate scientific methods. The class could also discuss their definitions of theory and law, both outside and within the context of science.

[OL] It should be noted and possibly mentioned that a scientist , as mentioned in this section, does not necessarily mean a trained scientist. It could be anyone using methods of science.

Scientific Methods

Scientists often plan and carry out investigations to answer questions about the universe around us. These investigations may lead to natural laws. Such laws are intrinsic to the universe, meaning that humans did not create them and cannot change them. We can only discover and understand them. Their discovery is a very human endeavor, with all the elements of mystery, imagination, struggle, triumph, and disappointment inherent in any creative effort. The cornerstone of discovering natural laws is observation. Science must describe the universe as it is, not as we imagine or wish it to be.

We all are curious to some extent. We look around, make generalizations, and try to understand what we see. For example, we look up and wonder whether one type of cloud signals an oncoming storm. As we become serious about exploring nature, we become more organized and formal in collecting and analyzing data. We attempt greater precision, perform controlled experiments (if we can), and write down ideas about how data may be organized. We then formulate models, theories, and laws based on the data we have collected, and communicate those results with others. This, in a nutshell, describes the scientific method that scientists employ to decide scientific issues on the basis of evidence from observation and experiment.

An investigation often begins with a scientist making an observation . The scientist observes a pattern or trend within the natural world. Observation may generate questions that the scientist wishes to answer. Next, the scientist may perform some research about the topic and devise a hypothesis . A hypothesis is a testable statement that describes how something in the natural world works. In essence, a hypothesis is an educated guess that explains something about an observation.

[OL] An educated guess is used throughout this section in describing a hypothesis to combat the tendency to think of a theory as an educated guess.

Scientists may test the hypothesis by performing an experiment . During an experiment, the scientist collects data that will help them learn about the phenomenon they are studying. Then the scientists analyze the results of the experiment (that is, the data), often using statistical, mathematical, and/or graphical methods. From the data analysis, they draw conclusions. They may conclude that their experiment either supports or rejects their hypothesis. If the hypothesis is supported, the scientist usually goes on to test another hypothesis related to the first. If their hypothesis is rejected, they will often then test a new and different hypothesis in their effort to learn more about whatever they are studying.

Scientific processes can be applied to many situations. Let’s say that you try to turn on your car, but it will not start. You have just made an observation! You ask yourself, "Why won’t my car start?" You can now use scientific processes to answer this question. First, you generate a hypothesis such as, "The car won’t start because it has no gasoline in the gas tank." To test this hypothesis, you put gasoline in the car and try to start it again. If the car starts, then your hypothesis is supported by the experiment. If the car does not start, then your hypothesis is rejected. You will then need to think up a new hypothesis to test such as, "My car won’t start because the fuel pump is broken." Hopefully, your investigations lead you to discover why the car won’t start and enable you to fix it.

A model is a representation of something that is often too difficult (or impossible) to study directly. Models can take the form of physical models, equations, computer programs, or simulations—computer graphics/animations. Models are tools that are especially useful in modern physics because they let us visualize phenomena that we normally cannot observe with our senses, such as very small objects or objects that move at high speeds. For example, we can understand the structure of an atom using models, without seeing an atom with our own eyes. Although images of single atoms are now possible, these images are extremely difficult to achieve and are only possible due to the success of our models. The existence of these images is a consequence rather than a source of our understanding of atoms. Models are always approximate, so they are simpler to consider than the real situation; the more complete a model is, the more complicated it must be. Models put the intangible or the extremely complex into human terms that we can visualize, discuss, and hypothesize about.

Scientific models are constructed based on the results of previous experiments. Even still, models often only describe a phenomenon partially or in a few limited situations. Some phenomena are so complex that they may be impossible to model them in their entirety, even using computers. An example is the electron cloud model of the atom in which electrons are moving around the atom’s center in distinct clouds ( Figure 1.12 ), that represent the likelihood of finding an electron in different places. This model helps us to visualize the structure of an atom. However, it does not show us exactly where an electron will be within its cloud at any one particular time.

As mentioned previously, physicists use a variety of models including equations, physical models, computer simulations, etc. For example, three-dimensional models are often commonly used in chemistry and physics to model molecules. Properties other than appearance or location are usually modelled using mathematics, where functions are used to show how these properties relate to one another. Processes such as the formation of a star or the planets, can also be modelled using computer simulations. Once a simulation is correctly programmed based on actual experimental data, the simulation can allow us to view processes that happened in the past or happen too quickly or slowly for us to observe directly. In addition, scientists can also run virtual experiments using computer-based models. In a model of planet formation, for example, the scientist could alter the amount or type of rocks present in space and see how it affects planet formation.

Scientists use models and experimental results to construct explanations of observations or design solutions to problems. For example, one way to make a car more fuel efficient is to reduce the friction or drag caused by air flowing around the moving car. This can be done by designing the body shape of the car to be more aerodynamic, such as by using rounded corners instead of sharp ones. Engineers can then construct physical models of the car body, place them in a wind tunnel, and examine the flow of air around the model. This can also be done mathematically in a computer simulation. The air flow pattern can be analyzed for regions smooth air flow and for eddies that indicate drag. The model of the car body may have to be altered slightly to produce the smoothest pattern of air flow (i.e., the least drag). The pattern with the least drag may be the solution to increasing fuel efficiency of the car. This solution might then be incorporated into the car design.

Using Models and the Scientific Processes

Be sure to secure loose items before opening the window or door.

In this activity, you will learn about scientific models by making a model of how air flows through your classroom or a room in your house.

• One room with at least one window or door that can be opened
• Work with a group of four, as directed by your teacher. Close all of the windows and doors in the room you are working in. Your teacher may assign you a specific window or door to study.
• Before opening any windows or doors, draw a to-scale diagram of your room. First, measure the length and width of your room using the tape measure. Then, transform the measurement using a scale that could fit on your paper, such as 5 centimeters = 1 meter.
• Your teacher will assign you a specific window or door to study air flow. On your diagram, add arrows showing your hypothesis (before opening any windows or doors) of how air will flow through the room when your assigned window or door is opened. Use pencil so that you can easily make changes to your diagram.
• On your diagram, mark four locations where you would like to test air flow in your room. To test for airflow, hold a strip of single ply tissue paper between the thumb and index finger. Note the direction that the paper moves when exposed to the airflow. Then, for each location, predict which way the paper will move if your air flow diagram is correct.
• Now, each member of your group will stand in one of the four selected areas. Each member will test the airflow Agree upon an approximate height at which everyone will hold their papers.
• When you teacher tells you to, open your assigned window and/or door. Each person should note the direction that their paper points immediately after the window or door was opened. Record your results on your diagram.
• Did the airflow test data support or refute the hypothetical model of air flow shown in your diagram? Why or why not? Correct your model based on your experimental evidence.
• With your group, discuss how accurate your model is. What limitations did it have? Write down the limitations that your group agreed upon.
• Yes, you could use your model to predict air flow through a new window. The earlier experiment of air flow would help you model the system more accurately.
• Yes, you could use your model to predict air flow through a new window. The earlier experiment of air flow is not useful for modeling the new system.
• No, you cannot model a system to predict the air flow through a new window. The earlier experiment of air flow would help you model the system more accurately.
• No, you cannot model a system to predict the air flow through a new window. The earlier experiment of air flow is not useful for modeling the new system.

This Snap Lab! has students construct a model of how air flows in their classroom. Each group of four students will create a model of air flow in their classroom using a scale drawing of the room. Then, the groups will test the validity of their model by placing weathervanes that they have constructed around the room and opening a window or door. By observing the weather vanes, students will see how air actually flows through the room from a specific window or door. Students will then correct their model based on their experimental evidence. The following material list is given per group:

• One room with at least one window or door that can be opened (An optimal configuration would be one window or door per group.)
• Several pieces of construction paper (at least four per group)
• Strips of single ply tissue paper
• One tape measure (long enough to measure the dimensions of the room)
• Group size can vary depending on the number of windows/doors available and the number of students in the class.
• The room dimensions could be provided by the teacher. Also, students may need a brief introduction in how to make a drawing to scale.
• This is another opportunity to discuss controlled experiments in terms of why the students should hold the strips of tissue paper at the same height and in the same way. One student could also serve as a control and stand far away from the window/door or in another area that will not receive air flow from the window/door.
• You will probably need to coordinate this when multiple windows or doors are used. Only one window or door should be opened at a time for best results. Between openings, allow a short period (5 minutes) when all windows and doors are closed, if possible.

Answers to the Grasp Check will vary, but the air flow in the new window or door should be based on what the students observed in their experiment.

Scientific Laws and Theories

A scientific law is a description of a pattern in nature that is true in all circumstances that have been studied. That is, physical laws are meant to be universal , meaning that they apply throughout the known universe. Laws are often also concise, whereas theories are more complicated. A law can be expressed in the form of a single sentence or mathematical equation. For example, Newton’s second law of motion , which relates the motion of an object to the force applied ( F ), the mass of the object ( m ), and the object’s acceleration ( a ), is simply stated using the equation

Scientific ideas and explanations that are true in many, but not all situations in the universe are usually called principles . An example is Pascal’s principle , which explains properties of liquids, but not solids or gases. However, the distinction between laws and principles is sometimes not carefully made in science.

A theory is an explanation for patterns in nature that is supported by much scientific evidence and verified multiple times by multiple researchers. While many people confuse theories with educated guesses or hypotheses, theories have withstood more rigorous testing and verification than hypotheses.

[OL] Explain to students that in informal, everyday English the word theory can be used to describe an idea that is possibly true but that has not been proven to be true. This use of the word theory often leads people to think that scientific theories are nothing more than educated guesses. This is not just a misconception among students, but among the general public as well.

As a closing idea about scientific processes, we want to point out that scientific laws and theories, even those that have been supported by experiments for centuries, can still be changed by new discoveries. This is especially true when new technologies emerge that allow us to observe things that were formerly unobservable. Imagine how viewing previously invisible objects with a microscope or viewing Earth for the first time from space may have instantly changed our scientific theories and laws! What discoveries still await us in the future? The constant retesting and perfecting of our scientific laws and theories allows our knowledge of nature to progress. For this reason, many scientists are reluctant to say that their studies prove anything. By saying support instead of prove , it keeps the door open for future discoveries, even if they won’t occur for centuries or even millennia.

[OL] With regard to scientists avoiding using the word prove , the general public knows that science has proven certain things such as that the heart pumps blood and the Earth is round. However, scientists should shy away from using prove because it is impossible to test every single instance and every set of conditions in a system to absolutely prove anything. Using support or similar terminology leaves the door open for further discovery.

Check Your Understanding

• Models are simpler to analyze.
• Models give more accurate results.
• Models provide more reliable predictions.
• Models do not require any computer calculations.
• They are the same.
• A hypothesis has been thoroughly tested and found to be true.
• A hypothesis is a tentative assumption based on what is already known.
• A hypothesis is a broad explanation firmly supported by evidence.
• A scientific model is a representation of something that can be easily studied directly. It is useful for studying things that can be easily analyzed by humans.
• A scientific model is a representation of something that is often too difficult to study directly. It is useful for studying a complex system or systems that humans cannot observe directly.
• A scientific model is a representation of scientific equipment. It is useful for studying working principles of scientific equipment.
• A scientific model is a representation of a laboratory where experiments are performed. It is useful for studying requirements needed inside the laboratory.
• The hypothesis must be validated by scientific experiments.
• The hypothesis must not include any physical quantity.
• The hypothesis must be a short and concise statement.
• The hypothesis must apply to all the situations in the universe.
• A scientific theory is an explanation of natural phenomena that is supported by evidence.
• A scientific theory is an explanation of natural phenomena without the support of evidence.
• A scientific theory is an educated guess about the natural phenomena occurring in nature.
• A scientific theory is an uneducated guess about natural phenomena occurring in nature.
• A hypothesis is an explanation of the natural world with experimental support, while a scientific theory is an educated guess about a natural phenomenon.
• A hypothesis is an educated guess about natural phenomenon, while a scientific theory is an explanation of natural world with experimental support.
• A hypothesis is experimental evidence of a natural phenomenon, while a scientific theory is an explanation of the natural world with experimental support.
• A hypothesis is an explanation of the natural world with experimental support, while a scientific theory is experimental evidence of a natural phenomenon.

Use the Check Your Understanding questions to assess students’ achievement of the section’s learning objectives. If students are struggling with a specific objective, the Check Your Understanding will help identify which objective and direct students to the relevant content.

This book may not be used in the training of large language models or otherwise be ingested into large language models or generative AI offerings without OpenStax's permission.

Want to cite, share, or modify this book? This book uses the Creative Commons Attribution License and you must attribute Texas Education Agency (TEA). The original material is available at: https://www.texasgateway.org/book/tea-physics . Changes were made to the original material, including updates to art, structure, and other content updates.

Access for free at https://openstax.org/books/physics/pages/1-introduction

• Authors: Paul Peter Urone, Roger Hinrichs
• Publisher/website: OpenStax
• Book title: Physics
• Publication date: Mar 26, 2020
• Location: Houston, Texas
• Book URL: https://openstax.org/books/physics/pages/1-introduction
• Section URL: https://openstax.org/books/physics/pages/1-2-the-scientific-methods

© Jun 7, 2024 Texas Education Agency (TEA). The OpenStax name, OpenStax logo, OpenStax book covers, OpenStax CNX name, and OpenStax CNX logo are not subject to the Creative Commons license and may not be reproduced without the prior and express written consent of Rice University.

Scientific Inquiry & Reasoning Skills - Skill 2: Scientific Reasoning and Problem-solving

New section.

• What's on the MCAT Exam? Home
• Download Exam Content (PDF)

Questions that test scientific reasoning and problem-solving skills differ from questions in the previous category by asking you to use your scientific knowledge to solve problems in the natural, behavioral, and social sciences.

As you work on questions that test this skill, you may be asked to use scientific theories to explain observations or make predictions about natural or social phenomena. Questions may ask you to judge the credibility of scientific explanations or to evaluate arguments about cause and effect. Or they may ask you to use scientific models and observations to draw conclusions. They may ask you to identify scientific findings that call a theory or model into question. Questions in this category may ask you to look at pictures or diagrams and draw conclusions from them. Or they may ask you to determine and then use scientific formulas to solve problems.

For example, you will be asked to show you can use scientific principles to solve problems by:

• Reasoning about scientific principles, theories, and models to make predictions or determine consequences.
• Analyzing and evaluating the validity or credibility of scientific explanations and predictions.
• Evaluating arguments about causes and consequences to determine the most valid argument when using scientific knowledge.
• Bringing together theory, observations, and evidence to draw conclusions.
• Recognizing or identifying scientific findings from a given study that challenge or invalidate a scientific theory or model.
• Determining and using scientific formulas to solve problems.
• Identifying the bond that would form between two structures if they were adjacent to each other.

By way of illustration, questions from the Psychological, Social, and Biological Foundations of Behavior section may ask you demonstrate this skill by:

Using the main premises of symbolic interactionism, use reasoning in an observational study of physician-patient interactions to describe how the premises are connected to perceived patient compliance.

Predicting how an individual will react to cognitive dissonance.

Using reasoning to determine whether a causal explanation is possible when given an example of how someone’s gender or personality predicts his or her behavior.

Explaining how an example, such as when an anorexic teenager restricts eating to satisfy esteem needs, is compatible with the premises of Maslow’s hierarchy of needs.

Drawing a conclusion about which sociological theory would be most consistent with a conceptual diagram that explains how social and environmental factors influence health and why this theory is most consistent.

Identifying the relationship between social institutions that is suggested by an illustration used in a public health campaign.

Recognizing a demographic trend that is represented in a population pyramid.

For more context, let’s consider three Skill 2 questions linked to different foundational concepts in the Psychological, Social, and Biological Foundations of Behavior section; the Biological and Biochemical Foundations of Living Systems section; and the Chemical and Physical Foundations of Biological Systems section.

Skill 2 Example From the Psychological, Social, and Biological Foundations of Behavior Section

Which statement describes what the concept of cultural capital predicts?

Cultural distinctions associated with the young will be more valued within a society.

With improved communication, there will eventually be a convergence of cultural practices of all classes.

Cultural distinctions by class will become less important during a recession because people will have less money to spend.

Cultural distinctions associated with elite classes will be more valued within a society.      

The correct answer is D. It is a Skill 2 question and assesses knowledge of Content Category 10A, Social inequality . It is a Skill 2 question because it requires you to make a prediction based on a particular concept. This question requires you to understand the concept of cultural capital in order to evaluate which prediction about social stratification would be most consistent with the concept.

Skill 2 Example From the Biological and Biochemical Foundations of Living Systems Section

Starting with the translation initiation codon, how many amino acids for this polypeptide does the sequence shown encode?

5'-CUGCCAAUGUGCUAAUCGCGGGGG-3'

The correct answer is A. This is a Skill 2 question, and you must use knowledge from Content Category 1B, Transmission of genetic information from the gene to the protein , to solve this problem. In addition to recalling the sequence for the start codon, this is a Skill 2 question because it requires you to apply the scientific principle of the genetic code to the provided RNA sequence. As a Skill 2 question, reasoning about the role of the stop codon in translation will allow you to arrive at the conclusion that this sequence codes for a polypeptide with two amino acids.

Skill 2 Example From the Chemical and Physical Foundations of Biological Systems Section

The radius of the aorta is about 1.0 cm, and blood passes through it at a velocity of 30 cm/s. A typical capillary has a radius of about 4 × 10 –4 cm, with blood passing through at a velocity of 5 × 10 –2 cm/s. Using these data, what is the approximate number of capillaries in a human body?

The correct answer is C. This Skill 2 question relates to Content Category 4B, Importance of fluids for the circulation of blood, gas movement, and gas exchange. This question asks you to use a mathematical model to make predictions about natural phenomena . To answer this question, you must be able to recognize the principles of flow characteristics of blood in the human body and apply the appropriate mathematical model to an unfamiliar scenario. Answering this question first requires recognition that the volume of blood flowing through the aorta is the same volume of blood flowing through the capillaries. It is a Skill 2 question because you then need to use reasoning skills to find the difference in the volumes that the aorta and capillaries can each carry in order to calculate the total number of capillaries.

Biological and Biochemical Foundations of Living Systems Section

• Foundational Concept 1
• Foundational Concept 2
• Foundational Concept 3

Chemical and Physical Foundations of Biological Systems Section

• Foundational Concept 4
• Foundational Concept 5

Psychological, Social, and Biological Foundations of Behavior Section

• Foundational Concept 6
• Foundational Concept 7
• Foundational Concept 8
• Foundational Concept 9
• Foundational Concept 10

Critical Analysis and Reasoning Skills Section

• Skill 1: Foundations of Comprehension
• Skill 2: Reasoning Within the Text
• Skill 3: Reasoning Beyond the Text
• Passage Types

Scientific Inquiry & Reasoning Skills

• Skill 1: Knowledge of Scientific Principles
• Skill 2: Scientific Reasoning and Problem-solving
• Skill 3: Reasoning about the Design and Execution of Research
• Skill 4: Data-based Statistical Reasoning
• General Mathematical Concepts and Techniques

• Table of Contents
• New in this Archive
• Chronological
• Editorial Information
• About the SEP
• Editorial Board
• How to Cite the SEP
• Special Characters
• Support the SEP
• PDFs for SEP Friends
• Make a Donation
• SEPIA for Libraries
• Entry Contents

Bibliography

Academic tools.

• Friends PDF Preview
• Author and Citation Info
• Back to Top

Scientific Discovery

Scientific discovery is the process or product of successful scientific inquiry. Objects of discovery can be things, events, processes, causes, and properties as well as theories and hypotheses and their features (their explanatory power, for example). Most philosophical discussions of scientific discoveries focus on the generation of new hypotheses that fit or explain given data sets or allow for the derivation of testable consequences. Philosophical discussions of scientific discovery have been intricate and complex because the term “discovery” has been used in many different ways, both to refer to the outcome and to the procedure of inquiry. In the narrowest sense, the term “discovery” refers to the purported “eureka moment” of having a new insight. In the broadest sense, “discovery” is a synonym for “successful scientific endeavor” tout court. Some philosophical disputes about the nature of scientific discovery reflect these terminological variations.

Philosophical issues related to scientific discovery arise about the nature of human creativity, specifically about whether the “eureka moment” can be analyzed and about whether there are rules (algorithms, guidelines, or heuristics) according to which such a novel insight can be brought about. Philosophical issues also arise about rational heuristics, about the characteristics of hypotheses worthy of articulation and testing, and, on the meta-level, about the nature and scope of philosophical reflection itself. This essay describes the emergence and development of the philosophical problem of scientific discovery, surveys different philosophical approaches to understanding scientific discovery, and presents the meta-philosophical problems surrounding the debates.

1. Introduction

2. scientific inquiry as discovery, 3. elements of discovery, 4. logics of discovery, 5. the distinction between the context of discovery and the context of justification, 6.1 discovery as abduction, 6.2 heuristic programming, 7. anomalies and the structure of discovery, 8.1 discoverability, 8.2 preliminary appraisal, 9.1 psychological and social conditions of creativity, 9.2 analogy, 9.3 mental models, other internet resources, related entries.

Philosophical reflection on scientific discovery occurred in different phases. Prior to the 1930s, philosophers were mostly concerned with discoveries in the broadest sense of the term, that is, with the analysis of successful scientific inquiry as a whole. Philosophical discussions focused on the question of whether there were any discernible patterns in the production of new knowledge. Because the concept of discovery did not have a specified meaning and was used in a very broad sense, almost all seventeenth- and eighteenth-century treatises on scientific method could potentially be considered as early contributions to reflections on scientific discovery. In the course of the 19 th century, as philosophy of science and science became two distinct endeavors, the term “discovery” became a technical term in philosophical discussions. Different elements of scientific inquiry were specified. Most importantly, the generation of new knowledge was clearly and explicitly distinguished from its validation, and thus the conditions for the narrower notion of discovery as the act of conceiving new ideas emerged.

The next phase in the discussion about scientific discovery began with the introduction of the so-called “context distinction,” the distinction between the “context of discovery” and the “context of justification”. It was further assumed that the act of conceiving a new idea is a non-rational process, a leap of insight that cannot be regulated. Justification, by contrast, is a systematic process of applying evaluative criteria to knowledge claims. Advocates of the context distinction argued that philosophy of science is exclusively concerned with the context of justification. The assumption underlying this argument is that philosophy is a normative project; it determines norms for scientific practice. Given these assumptions, only the justification of ideas, not their generation, can be the subject of philosophical (normative) analysis. Discovery, by contrast, can only be a topic for empirical study. By definition, the study of discovery is outside the scope of philosophy of science proper.

The introduction of the context distinction and the disciplinary distinction that was tied to it spawned meta-philosophical disputes. For a long time, philosophical debates about discovery were shaped by the notion that philosophical and empirical analyses are mutually exclusive. A number of philosophers insisted, like their predecessors prior to the 1930s, that the philosopher's tasks include the analysis of actual scientific practices and that scientific resources be used to address philosophical problems. They also maintained that it is a legitimate task for philosophy of science to develop a theory of heuristics or problem solving. But this position was the minority view during much of 20 th -century philosophy of science. Philosophers of discovery were thus compelled to demonstrate that scientific discovery was in fact a legitimate part of philosophy of science. Philosophical reflections about the nature of scientific discovery had to be bolstered by meta-philosophical arguments about the nature and scope of philosophy of science.

Today, however, there is wide agreement that philosophy and empirical research are not mutually exclusive. Not only do empirical studies of actual scientific discoveries inform philosophical thought about the structure and cognitive mechanisms of discovery, but researches in psychology, cognitive science, artificial intelligence and related fields have become an integral part of philosophical analyses of the processes and conditions of the generation of new knowledge.

Prior to the 19 th century, the term “discovery” commonly referred to the product of successful inquiry. “Discovery” was used broadly to refer to a new finding, such as a new cure, an improvement of an instrument, or a new method of measuring longitude. Several natural and experimental philosophers, notably Bacon, Descartes, and Newton, expounded accounts of scientific methods for arriving at new knowledge. These accounts were not explicitly labeled “methods of discovery ”, but the general accounts of scientific methods are nevertheless relevant for current philosophical debates about scientific discovery. They are relevant because philosophers of science have frequently presented 17 th -century theories of scientific method as a contrast class to current philosophies of discovery. The distinctive feature of the 17 th - and 18 th -century accounts of scientific method is that the methods have probative force (Nickles 1985). This means that those accounts of scientific method function as guides for acquiring new knowledge and at the same time as validations of the knowledge thus obtained (Laudan 1980; Schaffner 1993: chapter 2).

Bacon's account of his “new method” as it is presented in the Novum Organum is a prominent example. Bacon's work showed how best to arrive at knowledge about “form natures” (the most general properties of matter) via a systematic investigation of phenomenal natures. Bacon described how first to collect and organize natural phenomena and experimental facts in tables, how to evaluate these lists, and how to refine the initial results with the help of further experiments. Through these steps, the investigator would arrive at conclusions about the “form nature” that produces particular phenomenal natures. The point is that for Bacon, the procedures of constructing and evaluating tables and conducting experiments according to the Novum Organum leads to secure knowledge. The procedures thus have “probative force”.

Similarly, Newton's aim in the Philosophiae Naturalis Principia Mathematica was to present a method for the deduction of propositions from phenomena in such a way that those propositions become “more secure” than propositions that are secured by deducing testable consequences from them (Smith 2002). Newton did not assume that this procedure would lead to absolute certainty. One could only obtain moral certainty for the propositions thus secured. The point for current philosophers of science is that these approaches are generative theories of scientific method. Generative theories of scientific method assume that propositions can only be established and secured by showing that they follow from observed and experimentally produced phenomena. In contrast, non-generative theories of scientific method—such as the one proposed by Huygens—assumed that propositions must be established by comparing their consequences with observed and experimentally produced phenomena. In 20 th -century philosophy of science, this approach is often characterized as “consequentialist” (Laudan 1980; Nickles 1985).

Recent philosophers of science have used historical sketches like these to construct the prehistory of current philosophical debates about scientific discovery. The argument is that scientific discovery became a problem for philosophy of science in the 19 th century, when consequentialist theories of scientific method became more widespread. When consequentialist theories were on the rise, the two processes of conception and validation of an idea or hypothesis became distinct and the view that the merit of a new idea does not depend on the way in which it was arrived at became widely accepted.

In the course of the 19 th century, the act of having an insight—the purported “eureka moment”—was separated from processes of articulating, developing, and testing the novel insight. Philosophical discussion focused on the question of whether and to what extent rules could be devised to guide each of these processes. William Whewell's work, especially the two volumes of Philosophy of the Inductive Sciences of 1840, is an important contribution to the philosophical debates about scientific discovery precisely because he clearly separated the creative moment or “happy thought” as he called it from other elements of scientific inquiry. For Whewell, discovery comprised all three elements: the happy thought, the articulation and development of that thought, and the testing or verification of it. In most of the subsequent treatments of discovery, however, the scope of the term “discovery” is limited to either the first of these elements, the “happy thought”, or to the first two of these elements, the happy thought and its articulation. In fact, much of the controversies in the 20 th century about the possibility of a philosophy of discovery can be understood against the background of the disagreement about whether the process of discovery does or does not include the articulation and development of a novel thought.

The previous section shows that scholars like Bacon and Newton aimed to develop methodologies of scientific inquiry. They proposed “new methods” or “rules of reasoning” that guide the generation of certain propositions from observed and experimental phenomena. Whewell, by contrast, was explicitly concerned with developing a philosophy of discovery. His account was in part a description of the psychological makeup of the discoverer. For instance, he held that only geniuses could have those happy thoughts that are essential to discovery. In part, his account was an account of the methods by which happy thoughts are integrated into the system of knowledge. According to Whewell, the initial step in every discovery is what he called “some happy thought, of which we cannot trace the origin, some fortunate cast of intellect, rising above all rules. No maxims can be given which inevitably lead to discovery” (Whewell 1996 [1840]: 186). An “art of discovery” in the sense of a teachable and learnable skill does not exist according to Whewell. The happy thought builds on the known facts, but according to Whewell it is impossible to prescribe a method for having happy thoughts.

In this sense, happy thoughts are accidental. But in an important sense, scientific discoveries are not accidental. The happy thought is not a “wild guess.” Only the person whose mind is prepared to see things will actually notice them. The “previous condition of the intellect, and not the single fact, is really the main and peculiar cause of the success. The fact is merely the occasion by which the engine of discovery is brought into play sooner or later. It is, as I have elsewhere said, only the spark which discharges a gun already loaded and pointed; and there is little propriety in speaking of such an accident as the cause why the bullet hits its mark.” (Whewell 1996 [1840]: 189).

Having a happy thought is not yet a discovery, however. The second element of a scientific discovery consists in binding together—“colligating”, as Whewell called it—a set of facts by bringing them under a general conception. Not only does the colligation produce something new, but it also shows the previously known facts in a new light. More precisely, colligation works from both ends, from the facts as well as from the ideas that bind the facts together. Colligation is an extended process. It involves, on the one hand, the specification of facts through systematic observation, measurements and experiment, and on the other hand, the clarification of ideas through the exposition of the definitions and axioms that are tacitly implied in those ideas. This process is iterative. The scientists go back and forth between binding together the facts, clarifying the idea, rendering the facts more exact, and so on and so forth.

The final part of the discovery is the verification of the colligation involving the happy thought. This means, first and foremost, that the outcome of the colligation must be sufficient to explain the data at hand. Verification also involves judging the predictive power, simplicity, and “consilience” of the outcome of the colligation. “Consilience” refers to a higher range of generality (broader applicability) of the theory (the articulated and clarified happy thought) that the actual colligation produced. Whewell's account of discovery is not a deductivist system. It is essential that the outcome of the colligation be inferable from the data prior to any testing (Snyder 1997).

Whewell's theory of discovery is significant for the philosophical debate about scientific discovery because it clearly separates three elements: the non-analyzable happy thought or “eureka moment”; the process of colligation which includes the clarification and explication of facts and ideas; and the verification of the outcome of the colligation. His position that the philosophy of discovery cannot prescribe how to think happy thoughts has been a key element of 20 th -century philosophical reflection on discovery, and many philosophers have adopted the notion “happy thought” as a label for the “eureka moment” involved in discovery. Notably, however, Whewell's conception of discovery not only comprises the happy thoughts but also the processes by which the happy thoughts are to be integrated into the given system of knowledge. The procedures of articulation and test are both analyzable according to Whewell, and his conception of colligation and verification serve as guidelines for how the discoverer should proceed. A colligation, if properly done, has as such justificatory force. Similarly, the process of verification is an integral part of discovery and it too has justificatory force. Whewell's conception of verification thus comprises elements of generative and consequential methods of inquiry. To verify a hypothesis, the investigator needs to show that it accounts for the known facts, that it foretells new, previously unobserved phenomena, and that it can explain and predict phenomena which are explained and predicted by a hypothesis that was obtained through an independent happy thought-cum-colligation (Ducasse 1951).

Whewell's conceptualization of scientific discovery offers a useful framework for mapping the philosophical debates about discovery and for identifying major issues of concern in recent philosophical debates. First and foremost, nearly all recent philosophers operate with a notion of discovery that is narrower than Whewell's. In the narrower conception, what Whewell called “verification” is not part of discovery proper. Secondly, until the late 20 th century, there was wide agreement that the “eureka moment,” narrowly construed, is an unanalyzable, even mysterious leap of insight. The main disagreements concerned the question of whether the process of developing a hypothesis (the “colligation” in Whewell's terms) is or is not a part of discovery proper, and if it is, whether and how this process is guided by rules. Philosophers also disagreed on the issue of whether it is a philosophical task to explicate these rules. In recent decades, philosophical attention has shifted to the “eureka moment”. Drawing on resources from cognitive science, neuroscience, computational research, and environmental and social psychology, they have “demystified” the cognitive processes involved in the generation of new ideas.

In the early 20 th century, the view that discovery is or at least crucially involves a non-analyzable creative act of a gifted genius was widespread but not unanimously accepted. Alternative conceptions of discovery emphasize that discovery is an extended process, i.e., that the discovery process includes the reasoning processes through which a new insight is articulated and further developed. Moreover, it was assumed that there is a systematic, formal aspect to that reasoning. While the reasoning involved does not proceed according to the principles of demonstrative logic, it is systematic enough to deserve the label “logical”. Proponents of this view argued that traditional (here: Aristotelian) logic is an inadequate model of scientific discovery because it misrepresents the process of knowledge generation as grossly as the notion of “happy thought”. In this approach, the term “logic” is used in the broad sense. It is the task of the logic of discovery to draw out and give a schematic representation of the reasoning strategies that were applied in episodes of successful scientific inquiry. Early 20 th -century logics of discovery can best be described as theories of the mental operations involved in knowledge generation. Among these mental operations are classification, determination of what is relevant to an inquiry, and the conditions of communication of meaning. It is argued that these features of scientific discovery are either not or insufficiently represented by traditional logic (Schiller 1917: 236–7).

Philosophers advocating this approach agree that the logic of discovery should be characterized as a set of heuristic principles rather than as a process of applying inductive or deductive logic to a set of propositions. These heuristic principles are not understood to show the path to secure knowledge. Heuristic principles are suggestive rather than demonstrative (Carmichael 1922, 1930). One recurrent feature in these accounts of the reasoning strategies leading to new ideas is analogical reasoning (Schiller 1917; Benjamin 1934). In the 20 th century, it is widely acknowledged that analogical reasoning is a productive form of reasoning that cannot be reduced to inductive or deductive reasoning. However, these approaches to the logic of discovery remained scattered and tentative at that time, and attempts to develop more systematically the heuristics guiding discovery processes were eclipsed by the advance of the distinction between contexts of discovery and justification.

The distinction between “context of discovery” and “context of justification” dominated and shaped the discussions about discovery in 20 th -century philosophy of science. The context distinction marks the distinction between the generation of a new idea or hypothesis and the defense (test, verification) of it. As the previous sections have shown, the distinction among different features of scientific inquiry has a longer history, but in philosophy of science it became potent in the first half of the 20 th century. In the course of the ensuing discussions about scientific discovery, the distinction between the different features of scientific inquiry turned into a powerful demarcation criterion. The boundary between context of discovery (the de facto thinking processes) and context of justification (the de jure defense of the correctness of these thoughts) was now understood to determine the scope of philosophy of science. The underlying assumption is that philosophy of science is a normative endeavor. Advocates of the context distinction argue that the generation of a new idea is an intuitive, irrational process; it cannot be subject to normative analysis. Therefore, the study of scientists' actual thinking can only be the subject of psychology, sociology, and other empirical sciences. Philosophy of science, by contrast, is exclusively concerned with the context of justification.

The terms “context of discovery” and “context of justification” are often associated with Hans Reichenbach's work. Reichenbach's original conception of the context distinction is quite complex, however (Howard 2006; Richardson 2006). It does not map easily on to the disciplinary distinction mentioned above, because for Reichenbach, philosophy of science proper is partly descriptive. Reichenbach maintains that philosophy of science includes a description of knowledge as it really is. Descriptive philosophy of science reconstructs scientists' thinking processes in such a way that logical analysis can be performed on them, and it thus prepares the ground for the evaluation of these thoughts (Reichenbach 1938: § 1). Discovery, by contrast, is the object of empirical—psychological, sociological—study. According to Reichenbach, the empirical study of discoveries shows that processes of discovery often correspond to the principle of induction, but this is simply a psychological fact (Reichenbach 1938: 403).

While the terms “context of discovery” and “context of justification” are widely used, there has been ample discussion about how the distinction should be drawn and what their philosophical significance is (c.f. Kordig 1978; Gutting 1980; Zahar 1983; Leplin 1987; Hoyningen-Huene 1987; Weber 2005: chapter 3; Schickore and Steinle 2006). Most commonly, the distinction is interpreted as a distinction between the process of conceiving a theory and the validation of that theory, that is, the determination of the theory's epistemic support. This version of the distinction is not necessarily interpreted as a temporal distinction. In other words, it is not usually assumed that a theory is first fully developed and then validated. Rather, conception and validation are two different epistemic approaches to theory: the endeavor to articulate, flesh out, and develop its potential and the endeavor to assess its epistemic worth. Within the framework of the context distinction, there are two main ways of conceptualizing the process of conceiving a theory. The first option is to characterize the generation of new knowledge as an irrational act, a mysterious creative intuition, a “eureka moment”. The second option is to conceptualize the generation of new knowledge as an extended process that includes a creative act as well as some process of articulating and developing the creative idea.

Both of these accounts of knowledge generation served as starting points for arguments against the possibility of a philosophy of discovery. In line with the first option, philosophers have argued that neither is it possible to prescribe a logical method that produces new ideas nor is it possible to reconstruct logically the process of discovery. Only the process of testing is amenable to logical investigation. This objection to philosophies of discovery has been called the “discovery machine objection” (Curd 1980: 207). It is usually associates with Karl Popper's Logic of Scientific Discovery .

The initial state, the act of conceiving or inventing a theory, seems to me neither to call for logical analysis not to be susceptible of it. The question how it happens that a new idea occurs to a man—whether it is a musical theme, a dramatic conflict, or a scientific theory—may be of great interest to empirical psychology; but it is irrelevant to the logical analysis of scientific knowledge. This latter is concerned not with questions of fact (Kant's quid facti ?) , but only with questions of justification or validity (Kant's quid juris ?) . Its questions are of the following kind. Can a statement be justified? And if so, how? Is it testable? Is it logically dependent on certain other statements? Or does it perhaps contradict them? […]Accordingly I shall distinguish sharply between the process of conceiving a new idea, and the methods and results of examining it logically. As to the task of the logic of knowledge—in contradistinction to the psychology of knowledge—I shall proceed on the assumption that it consists solely in investigating the methods employed in those systematic tests to which every new idea must be subjected if it is to be seriously entertained. (Popper 2002 [1934/1959]: 7-8)

With respect to the second way of conceptualizing knowledge generation, many philosophers argue in a similar fashion that because the process of discovery involves an irrational, intuitive process, which cannot be examined logically, a logic of discovery cannot be construed. Other philosophers turn against the philosophy of discovery even though they explicitly acknowledge that discovery is an extended, reasoned process. They present a meta-philosophical objection argument, arguing that a theory of articulating and developing ideas is not a philosophical but a psychological theory.

The impact of the context distinction on studies of scientific discovery and on philosophy of science more generally can hardly be overestimated. The view that the process of discovery (however construed) is outside the scope of philosophy of science proper was widely shared amongst philosophers of science for most of the 20 th century and is still held by many. The last section shows that there were a few attempts to develop logics of discovery in the 1920s and 1930s. But for several decades, the context distinction dictated what philosophy of science should be about and how it should proceed. The dominant view was that theories of mental operations or heuristics had no place in philosophy of science. Therefore, discovery was not a legitimate topic for philosophy of science. The wide notion of discovery is mostly deployed in sociological accounts of scientific practice. In this perspective, “discovery” is understood as a retrospective label, which is attributed as a sign of accomplishment to some scientific endeavors. Sociological theories acknowledge that discovery is a collective achievement and the outcome of a process of negotiation through which “discovery stories” are constructed and certain knowledge claims are granted discovery status (Brannigan 1981; Schaffer 1986, 1994). Until the last third of the 20 th century, there were few attempts to challenge the disciplinary distinction tied to the context distinction. Only in the 1970s did the interest in philosophical approaches to discovery begin to increase. But the context distinction remained a challenge for philosophies of discovery.

There are three main lines of response to the disciplinary distinction tied to the context distinction. Each of these lines of response opens up a philosophical perspective on discovery. Each proceeds on the assumption that philosophy of science may legitimately include some form of analysis of actual reasoning patterns as well as information from empirical sciences such as cognitive science, psychology, and sociology. All of these responses reject the idea that discovery is nothing but a mystical event. Discovery is conceived as an analyzable reasoning process, not just as a creative leap by which novel ideas spring into being fully formed. All of these responses agree that the procedures and methods for arriving at new hypotheses and ideas are no guarantee that the hypothesis or idea that is thus formed is necessarily the best or the correct one. Nonetheless, it is the task of philosophy of science to provide rules for making this process better. All of these responses can be described as theories of problem solving, whose ultimate goal is to make the generation of new ideas and theories more efficient.

But the different approaches to scientific discovery employ different terminologies. In particular, the term “logic” of discovery is sometimes used in a narrow sense and sometimes broadly understood. In the narrow sense, “logic” of discovery is understood to refer to a set of formal, generally applicable rules by which novel ideas can be mechanically derived from existing data. In the broad sense, “logic” of discovery refers to the schematic representation of reasoning procedures. “Logical” is just another term for “rational”. Moreover, while each of these responses combines philosophical analyses of scientific discovery with empirical research on actual human cognition, different sets of resources are mobilized, ranging from AI research and cognitive science to historical studies of problem-solving procedures. Also, the responses parse the process of scientific inquiry differently. Often, scientific inquiry is regarded as having two aspects, viz. generation and validation of new ideas. At times, however, scientific inquiry is regarded as having three aspects, namely generation, pursuit or articulation, and validation of knowledge. In the latter framework, the label “discovery” is sometimes used to refer just to generation and sometimes to refer to both generation and pursuit.

The first response to the challenge of the context distinction draws on a broad understanding of the term “logic” to argue that a logic of scientific discovery can be developed ( section 6 ). The second response, drawing on a narrow understanding of the term “logic”, is to concede that there is no logic of discovery, i.e., no algorithm for generating new knowledge. Philosophers who take this approach argue that the process of discovery follows an identifiable, analyzable pattern ( section 7 ). Others argue that discovery is governed by a methodology . The methodology of discovery is a legitimate topic for philosophical analysis ( section 8 ). All of these responses assume that there is more to discovery than a “eureka moment.” Discovery comprises processes of articulating and developing the creative thought. These are the processes that can be examined with the tools of philosophical analysis. The third response to the challenge of the context distinction also assumes that discovery is or at least involves a creative act. But in contrast to the first two responses, it is concerned with the creative act itself. Philosophers who take this approach argue that scientific creativity is amenable to philosophical analysis ( section 9 ).

6. Logics of discovery after the context distinction

The first response to the challenge of the context distinction is to argue that discovery is a topic for philosophy of science because it is a logical process after all. Advocates of this approach to the logic of discovery usually accept the overall distinction between the two processes of conceiving and testing a hypothesis. They also agree that it is impossible to put together a manual that provides a formal, mechanical procedure through which innovative concepts or hypotheses can be derived: There is no discovery machine. But they reject the view that the process of conceiving a theory is a creative act, a mysterious guess, a hunch, a more or less instantaneous and random process. Instead, they insist that both conceiving and testing hypotheses are processes of reasoning and systematic inference, that both of these processes can be represented schematically, and that it is possible to distinguish better and worse paths to new knowledge.

This line of argument has much in common with the logics of discovery described in section 4 above but it is now explicitly pitched against the disciplinary distinction tied to the context distinction. There are two main ways of developing this argument. The first is to conceive of discovery in terms of abductive reasoning ( section 6.1 ). The second is to conceive of discovery in terms of problem-solving algorithms, whereby heuristic rules aid the processing of available data and enhance the success in finding solutions to problems ( section 6.2 ). Both lines of argument rely on a broad conception of logic, whereby the “logic” of discovery amounts to a schematic account of the reasoning processes involved in knowledge generation.

One argument, elaborated prominently by Norwood R. Hanson, is that the act of discovery—here, the act of suggesting a new hypothesis—follows a distinctive logical pattern, which is different from both inductive logic and the logic of hypothetico-deductive reasoning. The special logic of discovery is the logic of abductive or “retroductive” inferences (Hanson 1958). The argument that it is through an act of abductive inferences that plausible, promising scientific hypotheses are devised goes back to C.S. Peirce. This version of the logic of discovery characterizes reasoning processes that take place before a new hypothesis is ultimately justified. The abductive mode of reasoning that leads to plausible hypotheses is conceptualized as an inference beginning with data or, more specifically, with surprising or anomalous phenomena.

In this view, discovery is primarily a process of explaining anomalies or surprising, astonishing phenomena. The scientists' reasoning proceeds abductively from an anomaly to an explanatory hypothesis in light of which the phenomena would no longer be surprising or anomalous. The outcome of this reasoning process is not one single specific hypothesis but the delineation of a type of hypotheses that is worthy of further attention (Hanson 1965: 64). According to Hanson, the abductive argument has the following schematic form (Hanson 1960: 104):

• Some surprising, astonishing phenomena p 1 , p 2 , p 3 … are encountered.
• But p 1 , p 2 , p 3 … would not be surprising were an hypothesis of H's type to obtain. They would follow as a matter of course from something like H and would be explained by it.
• Therefore there is good reason for elaborating an hypothesis of type H—for proposing it as a possible hypothesis from whose assumption p 1 , p 2 , p 3 … might be explained.

Drawing on the historical record, Hanson argues that several important discoveries were made relying on abductive reasoning, such as Kepler's discovery of the elliptic orbit of Mars (Hanson 1958). It is now widely agreed, however, that Hanson's reconstruction of the episode is not a historically adequate account of Kepler's discovery (Lugg 1985). More importantly, while there is general agreement that abductive inferences are frequent in both everyday and scientific reasoning, these inferences are no longer considered as logical inferences. Even if one accepts Hanson's schematic representation of the process of identifying plausible hypotheses, this process is a “logical” process only in the widest sense whereby the term “logical” is understood as synonymous with “rational”. Notably, some philosophers have even questioned the rationality of abductive inferences (Koehler 1991; Brem and Rips 2000).

Another argument against the above schema is that it is too permissive. There will be several hypotheses that are explanations for phenomena p 1 , p 2 , p 3 …, so the fact that a particular hypothesis explains the phenomena is not a decisive criterion for developing that hypothesis (Harman 1965; see also Blackwell 1969). Additional criteria are required to evaluate the hypothesis yielded by abductive inferences.

Finally, it is worth noting that the schema of abductive reasoning does not explain the very act of conceiving a hypothesis or hypothesis-type. The processes by which a new idea is first articulated remain unanalyzed in the above schema. The schema focuses on the reasoning processes by which an exploratory hypothesis is assessed in terms of its merits and promise (Laudan 1980; Schaffner 1993).

In more recent work on abduction and discovery, two notions of abduction are sometimes distinguished: the common notion of abduction as inference to the best explanation (selective abduction) and creative abduction (Magnani 2000, 2009). Selective abduction—the inference to the best explanation—involves selecting a hypothesis from a set of known hypotheses. Medical diagnosis exemplifies this kind of abduction. Creative abduction, by contrast, involves generating a new, plausible hypothesis. This happens, for instance, in medical research, when the notion of a new disease is articulated. However, it is still an open question whether this distinction can be drawn, or whether there is a more gradual transition from selecting an explanatory hypothesis from a familiar domain (selective abduction) to selecting a hypothesis that is slightly modified from the familiar set and to identifying a more drastically modified or altered assumption.

Another recent suggestion is to broaden Peirce's original account of abduction and to include not only verbal information but also non-verbal mental representations, such as visual, auditory, or motor representations. In Thagard's approach, representations are characterized as patterns of activity in mental populations (see also section 9.3 below). The advantage of the neural account of human reasoning is that it covers features such as the surprise that accompanies the generation of new insights or the visual and auditory representations that contribute to it. If all mental representations can be characterized as patterns of firing in neural populations, abduction can be analyzed as the combination or “convolution” (Thagard) of patterns of neural activity from disjoint or overlapping patterns of activity (Thagard 2010).

The concern with the logic of discovery has also motivated research on artificial intelligence at the intersection of philosophy of science and cognitive science. In this approach, scientific discovery is treated as a form of problem-solving activity (Simon 1973; see also Newell and Simon 1971), whereby the systematic aspects of problem solving are studied within an information-processing framework. The aim is to clarify with the help of computational tools the nature of the methods used to discover scientific hypotheses. These hypotheses are regarded as solutions to problems. Philosophers working in this tradition build computer programs employing methods of heuristic selective search (e.g., Langley et al. 1987). In computational heuristics, search programs can be described as searches for solutions in a so-called “problem space” in a certain domain. The problem space comprises all possible configurations in that domain (e.g., for chess problems, all possible arrangements of pieces on a board of chess). Each configuration is a “state” of the problem space. There are two special states, namely the goal state, i.e., the state to be reached, and the initial state, i.e., the configuration at the starting point from which the search begins. There are operators, which determine the moves that generate new states from the current state. There are path constraints, which limit the permitted moves. Problem solving is the process of searching for a solution of the problem of how to generate the goal state from an initial state. In principle, all states can be generated by applying the operators to the initial state, then to the resulting state, until the goal state is reached (Langley et al. 1987: chapter 9). A problem solution is a sequence of operations leading from the initial to the goal state.

The basic idea behind computational heuristics is that rules can be identified that serve as guidelines for finding a solution to a given problem quickly and efficiently by avoiding undesired states of the problem space. These rules are best described as rules of thumb. The aim of constructing a logic of discovery thus becomes the aim of constructing a heuristics for the efficient search for solutions to problems. The term “heuristic search” indicates that in contrast to algorithms, problem-solving procedures lead to results that are merely provisional and plausible. A solution is not guaranteed, but heuristic searches are advantageous because they are more efficient than exhaustive random trial and error searches. Insofar as it is possible to evaluate whether one set of heuristics is better—more efficacious—than another, the logic of discovery turns into a normative theory of discovery.

Arguably, because it is possible to reconstruct important scientific discovery processes with sets of computational heuristics, the scientific discovery process can be considered as a special case of the general mechanism of information processing. In this context, the term “logic” is not used in the narrow sense of a set of formal, generally applicable rules to draw inferences but again in a broad sense as a label for a set of procedural rules.

The computer programs that embody the principles of heuristic searches in scientific inquiry simulate the paths that scientists followed when they searched for new theoretical hypotheses. Computer programs such as BACON (Simon et al. 1981) and KEKADA (Kulkarni and Simon 1988) utilize sets of problem-solving heuristics to detect regularities in given data sets. The program would note, for instance, that the values of a dependent term are constant or that a set of values for a term x and a set of values for a term y are linearly related. It would thus “infer” that the dependent term always has that value or that a linear relation exists between x and y . These programs can “make discoveries” in the sense that they can simulate successful discoveries such as Kepler's third law (BACON) or the Krebs cycle (KEKADA).

AI-based theories of scientific discoveries have helped identify and clarify a number of problem-solving strategies. An example of such a strategy is heuristic means-ends analysis, which involves identifying specific differences between the present and the goal situation and searches for operators (processes that will change the situation) that are associated with the differences that were detected. Another important heuristic is to divide the problem into sub-problems and to begin solving the one with the smallest number of unknowns to be determined (Simon 1977). AI-based approaches have also highlighted the extent to which the generation of new knowledge draws on existing knowledge that constrains the development of new hypotheses.

As accounts of scientific discoveries, computational heuristics have some limitations. Most importantly, because computer programs require the data from actual experiments the simulations cover only certain aspects of scientific discoveries. They do not design new experiments, instruments, or methods. Moreover, compared to the problem spaces given in computational heuristics, the complex problem spaces for scientific problems are often ill defined, and the relevant search space and goal state must be delineated before heuristic assumptions could be formulated (Bechtel and Richardson 1993: chapter 1).

Earlier critics of AI-based theories of scientific discoveries argued that a computer cannot devise new concepts but is confined to the concepts included in the given computer language (Hempel 1985: 119–120). Subsequent work has shown that computational methods can be used to generate new results leading to refereed scientific publications in astronomy, cancer research, ecology, and other fields (Langley 2000). The most recent computational research on scientific discovery is no longer driven by philosophical interests in scientific discovery, however. Instead, the main motivation is to contribute computational tools to aid scientists in their research.

Many philosophers maintain that discovery is a legitimate topic for philosophy of science while abandoning the notion that there is a logic of discovery. One very influential approach is Thomas Kuhn's analysis of the emergence of novel facts and theories (Kuhn 1970 [1962]: chapter 6). Kuhn identifies a general pattern of discovery as part of his account of scientific change. A discovery is not a simple act, but an extended, complex process, which culminates in paradigm changes. Paradigms are the symbolic generalizations, metaphysical commitments, values, and exemplars that are shared by a community of scientists and that guide the research of that community. Paradigm-based, normal science does not aim at novelty but instead at the development, extension, and articulation of accepted paradigms. A discovery begins with an anomaly, that is, with the recognition that the expectations induced by an established paradigm are being violated. The process of discovery involves several aspects: observations of an anomalous phenomenon, attempts to conceptualize it, and changes in the paradigm so that the anomaly can be accommodated.

It is the mark of success of normal science that it does not make transformative discoveries, and yet such discoveries come about as a consequence of normal, paradigm-guided science. The more detailed and the better developed a paradigm, the more precise are its predictions. The more precisely the researchers know what to expect, the better they are able to recognize anomalous results and violations of expectations:

novelty ordinarily emerges only for the man who, knowing with precision what he should expect, is able to recognize that something has gone wrong. Anomaly appears only against the background provided by the paradigm. (Kuhn 1970 [1962]: 65)

Drawing on several historical examples, Kuhn argues that it is usually impossible to identify the very moment when something was discovered or even the individual who made the discovery. Kuhn illustrates these points with the discovery of oxygen (see Kuhn 1970 [1962]: 53–56). Oxygen had not been discovered before 1774 and had been discovered by 1777. Even before 1774, Lavoisier had noticed that something was wrong with phlogiston theory, but he was unable to move forward. Two other investigators, C. W. Scheele and Joseph Priestley, independently identified a gas obtained from heating solid substances. But Scheele's work remained unpublished until after 1777, and Priestley did not identify his substance as a new sort of gas. In 1777, Lavoisier presented the oxygen theory of combustion, which gave rise to fundamental reconceptualization of chemistry. But according to this theory as Lavoisier first presented it, oxygen was not a chemical element. It was an atomic “principle of acidity” and oxygen gas was a combination of that principle with caloric. According to Kuhn, all of these developments are part of the discovery of oxygen, but none of them can be singled out as “the” act of discovery.

In pre-paradigmatic periods or in times of paradigm crisis, theory-induced discoveries may happen. In these periods, scientists speculate and develop tentative theories, which may lead to novel expectations and experiments and observations to test whether these expectations can be confirmed. Even though no precise predictions can be made, phenomena that are thus uncovered are often not quite what had been expected. In these situations, the simultaneous exploration of the new phenomena and articulation of the tentative hypotheses together bring about discovery.

In cases like the discovery of oxygen, by contrast, which took place while a paradigm was already in place, the unexpected becomes apparent only slowly, with difficulty, and against some resistance. Only gradually do the anomalies become visible as such. It takes time for the investigators to recognize “both that something is and what it is” (Kuhn 1970 [1962]: 55). Eventually, a new paradigm becomes established and the anomalous phenomena become the expected phenomena.

Recent studies in cognitive neuroscience of brain activity during periods of conceptual change support Kuhn's view that conceptual change is hard to achieve. These studies examine the neural processes that are involved in the recognition of anomalies and compare them with the brain activity involved in the processing of information that is consistent with preferred theories. The studies suggest that the two types of data are processed differently (Dunbar et al. 2007).

8. Methodologies of discovery

Advocates of the view that there are methodologies of discovery use the term “logic” in the narrow sense of an algorithmic procedure to generate new ideas. But like the AI-based theories of scientific discovery described in section 6 , methodologies of scientific discovery interpret the concept “discovery” as a label for an extended process of generating and articulating new ideas and often describe the process in terms of problem solving. In these approaches, the distinction between the contexts of discovery and the context of justification is challenged because the methodology of discovery is understood to play a justificatory role. Advocates of a methodology of discovery usually rely on a distinction between different justification procedures, justification involved in the process of generating new knowledge and justification involved in testing it. Consequential or “strong” justifications are methods of testing. The justification involved in discovery, by contrast, is conceived as generative (as opposed to consequential) justification ( section 8.1 ) or as weak (as opposed to strong) justification ( section 8.2 ). Again, some terminological ambiguity exists because according to some philosophers, there are three contexts, not two: Only the initial conception of a new idea (the “eureka moment”) is the context of discovery proper, and between it and justification there exists a separate context of pursuit (Laudan 1980). But many advocates of methodologies of discovery regard the context of pursuit as an integral part of the process of justification. They retain the notion of two contexts and re-draw the boundaries between the contexts of discovery and justification as they were drawn in the early 20 th century.

The methodology of discovery has sometimes been characterized as a form of justification that is complementary to the methodology of testing (Nickles 1984, 1985, 1989). According to the methodology of testing, empirical support for a theory results from successfully testing the predictive consequences derived from that theory (and appropriate auxiliary assumptions). In light of this methodology, justification for a theory is “consequential justification,” the notion that a hypothesis is established if successful novel predictions are derived from the theory or claim. Generative justification complements consequential justification. Advocates of generative justification hold that there exists an important form of justification in science that involves reasoning to a claim from data or previously established results more generally.

One classic example for a generative methodology is the set of Newton's rules for the study of natural philosophy. According to these rules, general propositions are established by deducing them from the phenomena. The notion of generative justification seeks to preserve the intuition behind classic conceptions of justification by deduction. Generative justification amounts to the rational reconstruction of the discovery path in order to establish its discoverability had the researchers known what is known now, regardless of how it was first thought of (Nickles 1985, 1989). The reconstruction demonstrates in hindsight that the claim could have been discovered in this manner had the necessary information and techniques been available. In other words, generative justification—justification as “discoverability” or “potential discovery”—justifies a knowledge claim by deriving it from results that are already established. While generative justification does not retrace exactly those steps of the actual discovery path that were actually taken, it is a better representation of scientists' actual practices than consequential justification because scientists tend to construe new claims from available knowledge. Generative justification is a weaker version of the traditional ideal of justification by deduction from the phenomena. Justification by deduction from the phenomena is complete if a theory or claim is completely determined from what we already know. The demonstration of discoverability results from the successful derivation of a claim or theory from the most basic and most solidly established empirical information.

Discoverability as described in the previous paragraphs is a mode of justification. Like the testing of novel predictions derived from a hypothesis, generative justification begins when the phase of finding and articulating a hypothesis worthy of assessing is drawing to a close. Other approaches to the methodology of discovery are directly concerned with the procedures involved in devising new hypotheses. The argument in favor of this kind of methodology is that the procedures of devising new hypotheses already include elements of appraisal. These preliminary assessments have been termed “weak” evaluation procedures (Schaffner 1993). Weak evaluations are relevant during the process of devising a new hypothesis. They provide reasons for accepting a hypothesis as promising and worthy of further attention. Strong evaluations, by contrast, provide reasons for accepting a hypothesis as (approximately) true or confirmed. Both “generative” and “consequential” testing as discussed in the previous section are strong evaluation procedures. Strong evaluation procedures are rigorous and systematically organized according to the principles of hypothesis derivation or H-D testing. A methodology of preliminary appraisal, by contrast, articulates criteria for the evaluation of a hypothesis prior to rigorous derivation or testing. It aids the decision about whether to take that hypothesis seriously enough to develop it further and test it. For advocates of this version of the methodology of discovery, it is the task of philosophy of science to characterize sets of constraints and methodological rules guiding the complex process of prior-to-test evaluation of hypotheses.

In contrast to the computational approaches discussed above, strategies of preliminary appraisal are not regarded as subject-neutral but as specific to particular fields of study. Because the analysis of criteria for the appraisal of hypotheses has mostly been made with regard to the study of biological mechanism, the criteria and constraints that have been proposed are those that play a role in the discovery of biological mechanisms. Biological mechanisms are entities and activities that are organized in such a way that they produce regular changes from initial to terminal conditions (Machamer et al. 2000).

Philosophers of biology have developed a fine-grained framework to account for the generation and preliminary evaluation of these mechanisms (Darden 2002; Craver 2002; Bechtel and Richardson 1993; Craver and Darden 2013). Some philosophers have even suggested that the phase of preliminary appraisal be further divided into two phases, the phase of appraising and the phase of revising. According to Lindley Darden, the phases of generation, appraisal and revision of descriptions of mechanisms can be characterized as reasoning processes governed by reasoning strategies. Different reasoning strategies govern the different phases (Darden 1991, 2002; Craver 2002; Darden 2009). The generation of hypotheses about mechanisms, for instance, is governed by the strategy of “schema instantiation” (see Darden 2002). The discovery of the mechanism of protein synthesis involved the instantiation of an abstract schema for chemical reactions: reactant 1 + reactant 2 = product. The actual mechanism of protein synthesis was found through specification and modification of this schema.

It is important to appreciate the status of these reasoning strategies. They are not necessarily strategies that were actually used. Neither of these strategies is deemed necessary for discovery, and they are not prescriptions for biological research. Rather, these strategies are deemed sufficient for the discovery of mechanisms; they “could have been used” to arrive at the description of that mechanism (Darden 2002). The methodology of the discovery of mechanisms is an extrapolation from past episodes of research on mechanisms and the result of a synthesis of rational reconstructions of several of these historical episodes. The methodology of discovery is only weakly normative in the sense that the strategies for the discovery of mechanisms that have been identified so far may prove useful in future biological research. Moreover, the sets of reasoning strategies that have been proposed are highly specific. It is still an open question whether the analysis of strategies for the discovery of biological mechanisms can illuminate the efficiency of scientific problem solving more generally (Weber 2005: chapter 3).

9. Creativity, analogy, and mental models

The approaches to scientific discovery presented in the previous sections focus on the adoption, articulation, and preliminary evaluation of ideas or hypotheses prior to rigorous testing. They do not illuminate how a novel hypothesis or idea is first thought up. Even among philosophers of discovery, the predominant view has long been that there is an initial step of discovery that is best described as a “eureka moment”, a mysterious intuitive leap of the human mind that cannot be analyzed further.

The concept of discovery as hypothesis-formation as it is encapsulated in the traditional distinction between context of discovery and context of justification does not explicate how new ideas are formed. According to accounts of discovery informed by evolutionary biology, the generation of new ideas is akin to random, blind variations of thought processes, which have to be inspected by the critical mind and assessed as neutral, productive, or useless (Campbell 1960; see also Hull 1988). While the evolutionary approach to discovery offers a more substantial account of scientific discovery, the key processes by which random ideas are generated are still left unanalyzed.

Today, many philosophers hold the view that creativity is not mysterious and can be submitted to analysis. Margaret Boden has offered helpful analyses of the concept of creativity. According to Boden, a new development is creative if it is novel, surprising, and important. She distinguishes between psychological creativity (P-creativity) and historical creativity (H-creativity). P-creativity is a development that is new, surprising and important to the particular person who comes up with it. H-creativity, by contrast, is radically novel, surprising, and important—it is generated for the first time (Boden 2004).

The majority of recent philosophical studies of scientific discovery today focus on the act of generation of new knowledge. The distinctive feature of these studies is that they integrate approaches from cognitive science, psychology, and computational neuroscience (Thagard 2012). Recent work on creativity offers substantive analyses of the social and psychological preconditions and the cognitive mechanisms involved in generating new ideas. Some of this research aims to characterize those features that are common to all creative processes. Other research aims to identify the features that are distinctive of scientific creativity (as opposed to other forms of creativity, such as artistic creativity or creative technological invention). Studies have focused on analyses of the personality traits that are conducive to creative thinking, and the social and environmental factors that are favorable for discovery ( section 9.1 ). Two key elements of the cognitive processes involved in creative thinking are analogies ( section 9.2 ) and mental models ( section 9.3 ).

Psychological studies of creative individuals' behavioral dispositions suggest that creative scientists share certain personality traits, including confidence, openness, dominance, independence, introversion, as well as arrogance and hostility. (For overviews of recent studies on personality traits of creative scientists, see Feist 1999, 2006: chapter 5). Social situatedness has also been explored as an important resource for creativity. In this perspective, the sociocultural structures and practices in which individuals are embedded are considered crucial for the generation of creative ideas. Both approaches suggest that creative individuals usually have outsider status—they are socially deviant and diverge from the mainstream.

Outsider status is also a key feature of standpoint. According to standpoint theorists, people with standpoint are politically aware and politically engaged people outside the mainstream. Some standpoint theorists suggest exploiting this similarity for creativity research. Because people with standpoint have different experiences and access to different domains of expertise than most members of a culture, they can draw on rich conceptual resources for creative thinking. Standpoint theory may thus be an important resource for the development of social and environmental approaches to the study of creativity (Solomon 2007).

Many philosophers of science highlight the role of analogy in the development of new knowledge, whereby analogy is understood as a process of bringing ideas that are well understood in one domain to bear on a new domain (Thagard 1984; Holyoak and Thagard 1996). An important source for philosophical thought about analogy is Mary Hesse's conception of models and analogies in theory construction and development. In this approach, analogies are similarities between different domains. Hesse introduces the distinction between positive, negative, and neutral analogies (Hesse 1966: 8). If we consider the relation between gas molecules and a model for gas, namely a collection of billiard balls in random motion, we will find properties that are common to both domains (positive analogy) as well as properties that can only be ascribed to the model but not to the target domain (negative analogy). There is a positive analogy between gas molecules and a collection of billiard balls because both the balls and the molecules move randomly. There is a negative analogy between the domains because billiard balls are colored, hard, and shiny but gas molecules do not have these properties. The most interesting properties are those properties of the model about which we do not know whether they are positive or negative analogies. This set of properties is the neutral analogy. These properties are the significant properties because they might lead to new insights about the less familiar domain. From our knowledge about the familiar billiard balls, we may be able to derive new predictions about the behavior of gas molecules, which we could then test.

Hesse offers a more detailed analysis of the structure of analogical reasoning through the distinction between horizontal and vertical analogies between domains. Horizontal analogies between two domains concern the sameness or similarity between properties of both domains. If we consider sound and light waves, there are similarities between them: sound echoes, light reflects; sound is loud, light is bright, both sound and light are detectable by our senses. There are also relations among the properties within one domain, such as the causal relation between sound and the loud tone we hear and, analogously, between physical light and the bright light we see. These analogies are vertical analogies. For Hesse, vertical analogies hold the key for the construction of new theories.

Analogies play several roles in science. Not only do they contribute to discovery but they also play a role in the development and evaluation of scientific theories. Current discussions about analogy and discovery have expanded and refined Hesse's approach in various ways. Some philosophers have developed criteria for evaluating analogy arguments (Bartha 2010). Other work has identified highly significant analogies that were particularly fruitful for the advancement of science (Holyoak and Thagard 1996: 186–188; Thagard 1999: chapter 9). The majority of analysts explore the features of the cognitive mechanisms through which aspects of a familiar domain or source are applied to an unknown target domain in order to understand what is unknown. According to the influential multi-constraint theory of analogical reasoning developed by Holyoak and Thagard, the transfer processes involved in analogical reasoning (scientific and otherwise) are guided or constrained in three main ways: 1) by the direct similarity between the elements involved; 2) by the structural parallels between source and target domain; as well as 3) by the purposes of the investigators, i.e., the reasons why the analogy is considered. Discovery, the formulation of a new hypothesis, is one such purpose.

“In vivo” investigations of scientists reasoning in their laboratories have not only shown that analogical reasoning is a key component of scientific practice, but also that the distance between source and target depends on the purpose for which analogies are sought. Scientists trying to fix experimental problems draw analogies between targets and sources from highly similar domains. In contrast, scientists attempting to formulate new models or concepts draw analogies between less similar domains. Analogies between radically different domains, however, are rare (Dunbar 1997, 2001).

In current cognitive science, human cognition is often explored in terms of model-based reasoning. The starting point of this approach is the notion that much of human reasoning, including probabilistic and causal reasoning as well as problem solving takes place through mental modeling rather than through the application of logic or methodological criteria to a set of propositions (Johnson-Laird 1983; Magnani et al. 1999; Magnani and Nersessian 2002). In model-based reasoning, the mind constructs a structural representation of a real-world or imaginary situation and manipulates this structure. In this perspective, conceptual structures are viewed as models and conceptual innovation as constructing new models through various modeling operations. Analogical reasoning—analogical modeling—is regarded as one of three main forms of model-based reasoning that appear to be relevant for conceptual innovation in science. Besides analogical modeling, visual modeling and simulative modeling or thought experiments also play key roles (Nersessian 1992, 1999, 2009). These modeling practices are constructive in that they aid the development of novel mental models. The key elements of model-based reasoning are the call on knowledge of generative principles and constraints for physical models in a source domain and the use of various forms of abstraction. Conceptual innovation results from the creation of new concepts through processes that abstract and integrate source and target domains into new models (Nersessian 2009).

Some critics have argued that despite the large amount of work on the topic, the notion of mental model is not sufficiently clear. Thagard seeks to clarify the concept by characterizing mental models in terms of neural processes (Thagard 2010). In his approach, mental models are produced through complex patterns of neural firing, whereby the neurons and the interconnections between them are dynamic and changing. A pattern of firing neurons is a representation when there is a stable causal correlation between the pattern or activation and the thing that is represented. In this research, questions about the nature of model-based reasoning are transformed into questions about the brain mechanisms that produce mental representations.

The above sections show that the study of scientific discovery has become an integral part of the wider endeavor of exploring creative thinking and creativity more generally. Naturalistic philosophical approaches combine conceptual analysis of processes of knowledge generation with empirical work on creativity, drawing heavily and explicitly on current research in psychology and cognitive science, and on in vivo laboratory observations, and, most recently, on brain imaging techniques.

• Bartha, P., 2010, By Parallel Reasoning: The Construction and Evaluation of Analogical Arguments , New York: Oxford University Press.
• Bechtel, W. and R. Richardson, 1993, Discovering Complexity , Princeton: Princeton University Press.
• Benjamin, A.C., 1934, “The Mystery of Scientific Discovery ” Philosophy of Science , 1: 224–36.
• Blackwell, R.J., 1969, Discovery in the Physical Sciences , Notre Dame: University of Notre Dame Press.
• Boden, M.A., 2004, The Creative Mind: Myths and Mechanisms , London: Routledge.
• Brannigan, A., 1981, The Social Basis of Scientific Discoveries , Cambridge: Cambridge University Press.
• Brem, S. and L.J. Rips, 2000, “Explanation and Evidence in Informal Argument”, Cognitive Science , 24: 573–604.
• Campbell, D., 1960, “Blind Variation and Selective Retention in Creative Thought as in Other Knowledge Processes”, Psychological Review , 67: 380–400.
• Carmichael, R.D., 1922, “The Logic of Discovery”, The Monist , 32: 569–608.
• –––, 1930, The Logic of Discovery , Chicago: Open Court.
• Craver, C.F., 2002, “Interlevel Experiments, Multilevel Mechanisms in the Neuroscience of Memory”, Philosophy of Science Supplement , 69: 83–97.
• Craver, C.F. and L. Darden, 2013, In Search of Mechanisms: Discoveries across the Life Sciences , Chicago: University of Chicago Press.
• Curd, M., 1980, “The Logic of Discovery: An Analysis of Three Approaches”, in T. Nickles (ed.) Scientific Discovery, Logic, and Rationality , Dordrecht: Reidel. 201–19.
• Darden, L., 1991, Theory Change in Science: Strategies from Mendelian Genetics , New York: Oxford University Press.
• –––, 2002, “Strategies for Discovering Mechanisms: Schema Instantiation, Modular Subassembly, Forward/Backward Chaining”, Philosophy of Science , 69: S354-S65.
• –––, 2009, “Discovering Mechanisms in Molecular Biology. Finding and Fixing Incompleteness and Incorrectness”, in J. Meheus and T. Nickles (eds), Models of Discovery and Creativity , Dordrecht: Springer. 43–55.
• Ducasse, C.J., 1951, “Whewell's Philosophy of Scientific Discovery II”, The Philosophical Review , 60(2): 213–34.
• Dunbar, K., 1997, “How scientists think: On-line creativity and conceptual change in science”, in T.B. Ward, S.M. Smith, and J. Vaid (eds), Conceptual Structures and Processes: Emergence, Discovery , and Change , Washington: American Psychological Association Press.
• –––, 2001, “The Analogical Paradox: Why Analogy is so Easy in Naturalistic Settings Yet so Difficult in Psychological Laboratories”, in D. Gentner, K.J. Holyoak, and B.N. Kokinov (eds), The Analogical Mind: Perspectives from Cognitive Science , Cambridge: MIT Press.
• Dunbar, K, J. Fugelsang, and C Stein, 2007, “Do Naïve Theories Ever Go Away? Using Brain and Behavior to Understand Changes in Concepts”, in M. Lovett and P. Shah (eds), Thinking with Data: 33rd Carnegie Symposium on Cognition , Mahwah: Erlbaum, 193–205.
• Feist, G.J., 1999, “The Influence of Personality on Artistic and Scientific Creativity”, in R.J. Sternberg (ed.), Handbook of Creativity , New York: Cambridge University Press, 273–96.
• –––, 2006, The psychology of science and the origins of the scientific mind , New Haven: Yale University Press.
• Gutting, G., 1980, “Science as Discovery”, Revue internationale de philosophie , 131: 26–48.
• Hanson, N.R., 1958, Patterns of Discovery , Cambridge: Cambridge University Press.
• –––, 1960, “Is there a Logic of Scientific Discovery?”, Australasian Journal of Philosophy , 38: 91–106.
• –––, 1965, “Notes Toward a Logic of Discovery”, in R.J. Bernstein (ed.), Perspectives on Peirce. Critical Essays on Charles Sanders Peirce , New Haven and London: Yale University Press, 42–65.
• Harman, G.H., 1965, “The Inference to the Best Explanation”, Philosophical Review , 74.
• Hempel, C.G., 1985, “Thoughts in the Limitations of Discovery by Computer”, in K. Schaffner (ed.), Logic of Discovery and Diagnosis in Medicine , Berkeley: University of California Press, 115–22.
• Hesse, M., 1966, Models and Analogies in Science , Notre Dame: University of Notre Dame Press.
• Holyoak, K.J. and P. Thagard, 1996, Mental Leaps: Analogy in Creative Thought , Cambridge: MIT Press.
• Howard, D., 2006, “Lost Wanderers in the Forest of Knowledge: Some Thoughts on the Discovery-Justification Distinction”, in J. Schickore and F. Steinle (eds), Revisiting Discovery and Justification. Historical and Philosophical Perspectives on the Context Distinction , Dordrecht: Springer, 3–22.
• Hoyningen-Huene, P., 1987, “Context of Discovery and Context of Justification”, Studies in History and Philosophy of Science ,18: 501–15.
• Hull, D.L., 1988, Science as Practice: An Evolutionary Account of the Social and Conceptual Development of Science , Chicago: University of Chicago Press.
• Johnson-Laird, P., 1983, Mental Models , Cambridge: Cambridge University Press.
• Koehler, D.J., 1991, “Explanation, Imagination, and Confidence in Judgment”, Psychological Bulletin , 110: 499–519.
• Kordig, C., 1978, “Discovery and Justification”, Philosophy of Science , 45: 110–17.
• Kuhn, T.S., 1970 [1962], The Structure of Scientific Revolutions , 2 nd ed., Chicago: The University of Chicago Press. First edition published in 1962.
• Kulkarni, D. and H.A. Simon, 1988, “The processes of scientific discovery: The strategy of experimentation”, Cognitive Science , 12: 139–76.
• Langley, P., 2000, “The Computational Support of Scientific Discovery”, International Journal of Human-Computer Studies , 53: 393–410.
• Langley, P., H.A. Simon, G.L. Bradshaw, and J.M. Zytkow, 1987, Scientific Discovery: Computational Explorations of the Creative Processes , Cambridge, MA: MIT Press.
• Laudan, L., 1980, “Why Was the Logic of Discovery Abandoned?” in T. Nickles (ed.), Scientific Discovery Vol. I, Dordrecht: Reidel. 173–83.
• Leplin, J., 1987, “The Bearing of Discovery on Justification”, Canadian Journal of Philosophy , 17: 805–14.
• Lugg, A., 1985, “The Process of Discovery”, Philosophy of Science , 52: 207–20.
• Machamer, P., L. Darden, and C.F. Craver, 2000, “Thinking About Mechanisms”, Philosophy of Science , 67: 1–25.
• Magnani, L., 2000, Abduction, Reason, and Science: Processes of Discovery and Explanation , Dordrecht: Kluwer.
• –––, 2009, “Creative Abduction and Hypothesis Withdrawal”, in J. Meheus and T. Nickles (eds), Models of Discovery and Creativity , Dordrecht: Springer.
• Magnani, L. and N.J. Nersessian, 2002, Model-Based Reasoning: Science, Technology, and Values , Dordrecht: Kluwer.
• Magnani, L., N.J. Nersessian, and P. Thagard, 1999, Model-Based Reasoning in Scientific Discovery , Dordrecht: Kluwer.
• Nersessian, N.J., 1992, “How do scientists think? Capturing the dynamics of conceptual change in science”, in R. Giere (ed.), Cognitive Models of Science , Minneapolis: University of Minnesota Press, 3–45.
• –––, 1999, “Model-based reasoning in conceptual change”, in L. Magnani, N.J. Nersessian and P. Thagard (eds), Model-Based Reasoning in Scientific Discovery , New York: Kluwer. 5–22.
• –––, 2009, “Conceptual Change: Creativity, Cognition, and Culture ” in J. Meheus and T. Nickles (eds), Models of Discovery and Creativity , Dordrecht: Springer, 127–66.
• Newell, A. and H. A Simon, 1971, “Human Problem Solving: The State of the Theory in 1970”, American Psychologist , 26: 145–59.
• Nickles, T., 1984, “Positive Science and Discoverability”, PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association , 1984: 13–27.
• –––, 1985, “Beyond Divorce: Current Status of the Discovery Debate”, Philosophy of Science , 52: 177–206.
• –––, 1989, “Truth or Consequences? Generative versus Consequential Justification in Science”, PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association , 1988, 393–405.
• Popper, K., 2002 [1934/1959], The Logic of Scientific Discovery , London and New York: Routledge. First published in German in 1934; first English translation in 1959.
• Reichenbach, H., 1938, Experience and Prediction. An Analysis of the Foundations and the Structure of Knowledge , Chicago: The University of Chicago Press.
• Richardson, A., 2006, “Freedom in a Scientific Society: Reading the Context of Reichenbach's Contexts”, in J. Schickore and F. Steinle (eds), Revisiting Discovery and Justification. Historical and Philosophical Perspectives on the Context Distinction , Dordrecht: Springer, 41–54.
• Schaffer, S., 1986, “Scientific Discoveries and the End of Natural Philosophy”, Social Studies of Science , 16: 387–420.
• –––, 1994, “Making Up Discovery”, in M.A. Boden (ed.), Dimensions of Creativity , Cambridge: MIT Press, 13–51.
• Schaffner, K., 1993, Discovery and Explanation in Biology and Medicine , Chicago: University of Chicago Press.
• Schickore, J. and F. Steinle, 2006, Revisiting Discovery and Justification. Historical and Philosophical Perspectives on the Context Distinction , Dordrecht: Springer.
• Schiller, F.C.S., 1917, “Scientific Discovery and Logical Proof”, in C.J. Singer (ed.), Studies in the History and Method of Science Vol. 1, Oxford: Clarendon. 235–89.
• Simon, H.A., 1973, “Does Scientific Discovery Have a Logic?”, Philosophy of Science , 40: 471–80.
• –––, 1977, Models of Discovery and Other Topics in the Methods of Science , Dordrecht: Reidel.
• Simon, H.A., P.W. Langley, and G.L. Bradshaw, 1981, “Scientific Discovery as Problem Solving”, Synthese , 47: 1–28.
• Smith, G.E., 2002, “The Methodology of the Principia ”, in G.E. Smith and I.B. Cohen (eds), The Cambridge Companion to Newton , Cambridge: Cambridge University Press, 138–73.
• Snyder, L.J., 1997, “Discoverers' Induction”, Philosophy of Science , 64: 580–604.
• Solomon, M., 2007, “Standpoint and Creativity”, Hypathia : 226–37.
• Thagard, P., 1984, “Conceptual Combination and Scientific Discovery”, PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association , 1984: 3–12.
• –––, 1999, How Scientists Explain Disease , Princeton: Princeton University Press.
• –––, 2010, “How Brains Make Mental Models”, in L. Magnani, N.J. Nersessian and P. Thagard (eds), Model-Based Reasoning in Science & Technology , Berlin and Heidelberg: Springer, 447–61.
• –––, 2012, The Cognitive Science of Science , Cambridge: MIT Press.
• Weber, M., 2005, Philosophy of Experimental Biology , Cambridge: Cambridge University Press.
• Whewell, W., 1996 [1840], The Philosophy of the Inductive Sciences , Vol. II. London: Routledge/Thoemmes.
• Zahar, E., 1983, “Logic of Discovery or Psychology of Invention?”, British Journal for the Philosophy of Science , 34: 243–61.

How to cite this entry . Preview the PDF version of this entry at the Friends of the SEP Society . Look up this entry topic at the Indiana Philosophy Ontology Project (InPhO). Enhanced bibliography for this entry at PhilPapers , with links to its database.

[Please contact the author with suggestions.]

abduction | analogy and analogical reasoning | cognitive science | knowledge: analysis of | Kuhn, Thomas | models in science | molecular biology | Newton, Isaac: Philosophiae Naturalis Principia Mathematica | Popper, Karl | Whewell, William

Copyright © 2014 by Jutta Schickore < jschicko @ indiana . edu >

Support SEP

Mirror sites.

View this site from another server:

• Info about mirror sites

The Stanford Encyclopedia of Philosophy is copyright © 2016 by The Metaphysics Research Lab , Center for the Study of Language and Information (CSLI), Stanford University

Library of Congress Catalog Data: ISSN 1095-5054

Scientific Research in Education (2002)

Chapter: 3 guiding principles for scientific inquiry, 3 guiding principles for scientific inquiry.

In Chapter 2 we present evidence that scientific research in education accumulates just as it does in the physical, life, and social sciences. Consequently, we believe that such research would be worthwhile to pursue to build further knowledge about education, and about education policy and practice. Up to this point, however, we have not addressed the questions “What constitutes scientific research?” and “Is scientific research on education different from scientific research in the social, life, and physical sciences?” We do so in this chapter.

These are daunting questions that philosophers, historians, and scientists have debated for several centuries (see Newton-Smith [2000] for a current assessment). Merton (1973), for example, saw commonality among the sciences. He described science as having four aims: universalism, the quest for general laws; organization, the quest to organize and conceptualize a set of related facts or observations; skepticism, the norm of questioning and looking for counter explanations; and communalism, the quest to develop a community that shares a set of norms or principles for doing science. In contrast, some early modern philosophers (the logical positivists) attempted to achieve unity across the sciences by reducing them all to physics, a program that ran into insuperable technical difficulties (Trant, 1991).

In short, we hold that there are both commonalities and differences across the sciences. At a general level, the sciences share a great deal in common, a set of what might be called epistemological or fundamental

principles that guide the scientific enterprise. They include seeking conceptual (theoretical) understanding, posing empirically testable and refutable hypotheses, designing studies that test and can rule out competing counterhypotheses, using observational methods linked to theory that enable other scientists to verify their accuracy, and recognizing the importance of both independent replication and generalization. It is very unlikely that any one study would possess all of these qualities. Nevertheless, what unites scientific inquiry is the primacy of empirical test of conjectures and formal hypotheses using well-codified observation methods and rigorous designs, and subjecting findings to peer review. It is, in John Dewey’s expression, “competent inquiry” that produces what philosophers call “knowledge claims” that are justified or “warranted” by pertinent, empirical evidence (or in mathematics, deductive proof). Scientific reasoning takes place amid (often quantifiable) uncertainty (Schum, 1994); its assertions are subject to challenge, replication, and revision as knowledge is refined over time. The long-term goal of much of science is to produce theory that can offer a stable encapsulation of “facts” that generalizes beyond the particular. In this chapter, then, we spell out what we see as the commonalities among all scientific endeavors.

As our work began, we attempted to distinguish scientific investigations in education from those in the social, physical, and life sciences by exploring the philosophy of science and social science; the conduct of physical, life, and social science investigations; and the conduct of scientific research on education. We also asked a panel of senior government officials who fund and manage research in education and the social and behavioral sciences, and a panel of distinguished scholars from psychometrics, linguistic anthropology, labor economics and law, to distinguish principles of evidence across fields (see National Research Council, 2001d). Ultimately, we failed to convince ourselves that at a fundamental level beyond the differences in specialized techniques and objects of inquiry across the individual sciences, a meaningful distinction could be made among social, physical, and life science research and scientific research in education. At times we thought we had an example that would demonstrate the distinction, only to find our hypothesis refuted by evidence that the distinction was not real.

Thus, the committee concluded that the set of guiding principles that apply to scientific inquiry in education are the same set of principles that

can be found across the full range of scientific inquiry. Throughout this chapter we provide examples from a variety of domains—in political science, geophysics, and education—to demonstrate this shared nature. Although there is no universally accepted description of the elements of scientific inquiry, we have found it convenient to describe the scientific process in terms of six interrelated, but not necessarily ordered, 1 principles of inquiry:

Pose significant questions that can be investigated empirically.

Link research to relevant theory.

Use methods that permit direct investigation of the question.

Provide a coherent and explicit chain of reasoning.

Replicate and generalize across studies.

Disclose research to encourage professional scrutiny and critique.

We choose the phrase “guiding principles” deliberately to emphasize the vital point that they guide, but do not provide an algorithm for, scientific inquiry. Rather, the guiding principles for scientific investigations provide a framework indicating how inferences are, in general, to be supported (or refuted) by a core of interdependent processes, tools, and practices. Although any single scientific study may not fulfill all the principles—for example, an initial study in a line of inquiry will not have been replicated independently—a strong line of research is likely to do so (e.g., see Chapter 2 ).

We also view the guiding principles as constituting a code of conduct that includes notions of ethical behavior. In a sense, guiding principles operate like norms in a community, in this case a community of scientists; they are expectations for how scientific research will be conducted. Ideally, individual scientists internalize these norms, and the community monitors them. According to our analysis these principles of science are common to systematic study in such disciplines as astrophysics, political science, and economics, as well as to more applied fields such as medicine, agriculture, and education. The principles emphasize objectivity, rigorous thinking, open-mindedness, and honest and thorough reporting. Numerous scholars

have commented on the common scientific “conceptual culture” that pervades most fields (see, e.g., Ziman, 2000, p. 145; Chubin and Hackett, 1990).

These principles cut across two dimensions of the scientific enterprise: the creativity, expertise, communal values, and good judgment of the people who “do” science; and generalized guiding principles for scientific inquiry. The remainder of this chapter lays out the communal values of the scientific community and the guiding principles of the process that enable well-grounded scientific investigations to flourish.

THE SCIENTIFIC COMMUNITY

Science is a communal “form of life” (to use the expression of the philosopher Ludwig Wittgenstein [1968]), and the norms of the community take time to learn. Skilled investigators usually learn to conduct rigorous scientific investigations only after acquiring the values of the scientific community, gaining expertise in several related subfields, and mastering diverse investigative techniques through years of practice.

The culture of science fosters objectivity through enforcement of the rules of its “form of life”—such as the need for replicability, the unfettered flow of constructive critique, the desirability of blind refereeing—as well as through concerted efforts to train new scientists in certain habits of mind. By habits of mind, we mean things such as a dedication to the primacy of evidence, to minimizing and accounting for biases that might affect the research process, and to disciplined, creative, and open-minded thinking. These habits, together with the watchfulness of the community as a whole, result in a cadre of investigators who can engage differing perspectives and explanations in their work and consider alternative paradigms. Perhaps above all, the communally enforced norms ensure as much as is humanly possible that individual scientists—while not necessarily happy about being proven wrong—are willing to open their work to criticism, assessment, and potential revision.

Another crucial norm of the scientific “form of life,” which also depends for its efficacy on communal enforcement, is that scientists should be ethical and honest. This assertion may seem trite, even naïve. But scientific knowledge is constructed by the work of individuals, and like any other enterprise, if the people conducting the work are not open and candid, it

can easily falter. Sir Cyril Burt, a distinguished psychologist studying the heritability of intelligence, provides a case in point. He believed so strongly in his hypothesis that intelligence was highly heritable that he “doctored” data from twin studies to support his hypothesis (Tucker, 1994; Mackintosh, 1995); the scientific community reacted with horror when this transgression came to light. Examples of such unethical conduct in such fields as medical research are also well documented (see, e.g., Lock and Wells, 1996).

A different set of ethical issues also arises in the sciences that involve research with animals and humans. The involvement of living beings in the research process inevitably raises difficult ethical questions about a host of potential risks, ranging from confidentiality and privacy concerns to injury and death. Scientists must weigh the relative benefits of what might be learned against the potential risks to human research participants as they strive toward rigorous inquiry. (We consider this issue more fully in Chapters 4 and 6 .)

GUIDING PRINCIPLES

Throughout this report we argue that science is competent inquiry that produces warranted assertions (Dewey, 1938), and ultimately develops theory that is supported by pertinent evidence. The guiding principles that follow provide a framework for how valid inferences are supported, characterize the grounds on which scientists criticize one another’s work, and with hindsight, describe what scientists do. Science is a creative enterprise, but it is disciplined by communal norms and accepted practices for appraising conclusions and how they were reached. These principles have evolved over time from lessons learned by generations of scientists and scholars of science who have continually refined their theories and methods.

SCIENTIFIC PRINCIPLE 1 Pose Significant Questions That Can Be Investigated Empirically

This principle has two parts. The first part concerns the nature of the questions posed: science proceeds by posing significant questions about the world with potentially multiple answers that lead to hypotheses or conjectures that can be tested and refuted. The second part concerns how these questions are posed: they must be posed in such a way that it is

possible to test the adequacy of alternative answers through carefully designed and implemented observations.

Question Significance

A crucial but typically undervalued aspect of successful scientific investigation is the quality of the question posed. Moving from hunch to conceptualization and specification of a worthwhile question is essential to scientific research. Indeed, many scientists owe their renown less to their ability to solve problems than to their capacity to select insightful questions for investigation, a capacity that is both creative and disciplined:

The formulation of a problem is often more essential than its solution, which may be merely a matter of mathematical or experimental skill. To raise new questions, new possibilities, to regard old questions from a new angle, requires creative imagination and marks real advance in science (Einstein and Infeld, 1938, p. 92, quoted in Krathwohl, 1998).

Questions are posed in an effort to fill a gap in existing knowledge or to seek new knowledge, to pursue the identification of the cause or causes of some phenomena, to describe phenomena, to solve a practical problem, or to formally test a hypothesis. A good question may reframe an older problem in light of newly available tools or techniques, methodological or theoretical. For example, political scientist Robert Putnam challenged the accepted wisdom that increased modernity led to decreased civic involvement (see Box 3-1 ) and his work has been challenged in turn. A question may also be a retesting of a hypothesis under new conditions or circumstances; indeed, studies that replicate earlier work are key to robust research findings that hold across settings and objects of inquiry (see Principle 5 ). A good question can lead to a strong test of a theory, however explicit or implicit the theory may be.

The significance of a question can be established with reference to prior research and relevant theory, as well as to its relationship with important claims pertaining to policy or practice. In this way, scientific knowledge grows as new work is added to—and integrated with—the body of material that has come before it. This body of knowledge includes theo-

ries, models, research methods (e.g., designs, measurements), and research tools (e.g., microscopes, questionnaires). Indeed, science is not only an effort to produce representations (models) of real-world phenomena by going from nature to abstract signs. Embedded in their practice, scientists also engage in the development of objects (e.g., instruments or practices); thus, scientific knowledge is a by-product of both technological activities and analytical activities (Roth, 2001). A review of theories and prior research relevant to a particular question can simply establish that it has not been answered before. Once this is established, the review can help shape alternative answers, the design and execution of a study by illuminating if and how the question and related conjectures have already been examined, as well as by identifying what is known about sampling, setting, and other important context. 2

Donald Stokes’ work (Stokes, 1997) provides a useful framework for thinking about important questions that can advance scientific knowledge and method (see Figure 3-1 ). In Pasteur’s Quadrant , he provided evidence that the conception of research-based knowledge as moving in a linear progression from fundamental science to applied science does not reflect how science has historically advanced. He provided several examples demonstrating that, instead, many advancements in science occurred as a result of “use-inspired research,” which simultaneously draws on both basic and applied research. Stokes (1997, p. 63) cites Brooks (1967) on basic and applied work:

Work directed toward applied goals can be highly fundamental in character in that it has an important impact on the conceptual structure or outlook of a field. Moreover, the fact that research is of such a nature that it can be applied does not mean that it is not also basic.

FIGURE 3-1. Quadrant model of scientific research.

SOURCE: Stokes (1997, p. 73). Reprinted with permission.

Stokes’ model clearly applies to research in education, where problems of practice and policy provide a rich source for important—and often highly fundamental in character—research questions.

Empirically Based

Put simply, the term “empirical” means based on experience through the senses, which in turn is covered by the generic term observation. Since science is concerned with making sense of the world, its work is necessarily grounded in observations that can be made about it. Thus, research questions

must be posed in ways that potentially allow for empirical investigation. 3 For example, both Milankovitch and Muller could collect data on the Earth’s orbit to attempt to explain the periodicity in ice ages (see Box 3-2 ). Likewise, Putnam could collect data from natural variations in regional government to address the question of whether modernization leads to the demise of civic community ( Box 3-1 ), and the Tennessee state legislature could empirically assess whether reducing class size improves students’ achievement in early grades ( Box 3-3 ) because achievement data could be collected on students in classes of varying sizes. In contrast, questions such as: “ Should all students be required to say the pledge of allegiance?” cannot be submitted to empirical investigation and thus cannot be examined scientifically. Answers to these questions lie in realms other than science.

SCIENTIFIC PRINCIPLE 2 Link Research to Relevant Theory

Scientific theories are, in essence, conceptual models that explain some phenomenon. They are “nets cast to catch what we call ‘the world’…we endeavor to make the mesh ever finer and finer” (Popper, 1959, p. 59). Indeed, much of science is fundamentally concerned with developing and testing theories, hypotheses, models, conjectures, or conceptual frameworks that can explain aspects of the physical and social world. Examples of well-known scientific theories include evolution, quantum theory, and the theory of relativity.

In the social sciences and in education, such “grand” theories are rare. To be sure, generalized theoretical understanding is still a goal. However, some research in the social sciences seeks to achieve deep understanding of particular events or circumstances rather than theoretical understanding that will generalize across situations or events. Between these extremes lies the bulk of social science theory or models, what Merton (1973) called

mid-range theories that attempt to account for some aspect of the social world. Examples of such mid-range theories or explanatory models can be found in the physical and the social sciences.

These theories are representations or abstractions of some aspect of reality that one can only approximate by such models. Molecules, fields, or black holes are classic explanatory models in physics; the genetic code and the contractile filament model of muscle are two in biology. Similarly,

culture, socioeconomic status, and poverty are classical models in anthropology, sociology, and political science. In program evaluation, program developers have ideas about the mechanism by which program inputs affect targeted outcomes; evaluations translate and test these ideas through a “program theory” that guides the work (Weiss, 1998a).

Theory enters the research process in two important ways. First, scientific research may be guided by a conceptual framework, model, or theory

that suggests possible questions to ask or answers to the question posed. 4 The process of posing significant questions typically occurs before a study is conducted. Researchers seek to test whether a theory holds up under certain circumstances. Here the link between question and theory is straightforward. For example, Putnam based his work on a theoretical conception of institutional performance that related civic engagement and modernization.

A research question can also devolve from a practical problem (Stokes, 1997; see discussion above). In this case, addressing a complex problem like the relationship between class size and student achievement may require several theories. Different theories may give conflicting predictions about the problem’s solution, or various theories might have to be reconciled to address the problem. Indeed, the findings from the Tennessee class size reduction study (see Box 3-3 ) have led to several efforts to devise theoretical understandings of how class size reduction may lead to better student achievement. Scientists are developing models to understand differences in classroom behavior between large and small classes that may ultimately explain and predict changes in achievement (Grissmer and Flannagan, 2000).

A second more subtle way that theoretical understanding factors into the research process derives from the fact that all scientific observations are “theory laden” (Kuhn, 1962). That is, the choice of what to observe and how to observe it is driven by an organizing conception—explicit or tacit— of the problem or topic. Thus, theory drives the research question, the use of methods, and the interpretation of results.

SCIENTIFIC PRINCIPLE 3

Use methods that permit direct investigation of the question.

Research methods—the design for collecting data and the measurement and analysis of variables in the design—should be selected in light of a research question, and should address it directly. Methods linked directly to problems permit the development of a logical chain of reasoning based

on the interplay among investigative techniques, data, and hypotheses to reach justifiable conclusions. For clarity of discussion, we separate out the link between question and method (see Principle 3 ) and the rigorous reasoning from evidence to theory (see Principle 4 ). In the actual practice of research, such a separation cannot be achieved.

Debates about method—in many disciplines and fields—have raged for centuries as researchers have battled over the relative merit of the various techniques of their trade. The simple truth is that the method used to conduct scientific research must fit the question posed, and the investigator must competently implement the method. Particular methods are better suited to address some questions rather than others. The rare choice in the mid 1980s in Tennessee to conduct a randomized field trial, for example, enabled stronger inferences about the effects of class size reduction on student achievement (see Box 3-3 ) than would have been possible with other methods.

This link between question and method must be clearly explicated and justified; a researcher should indicate how a particular method will enable competent investigation of the question of interest. Moreover, a detailed description of method—measurements, data collection procedures, and data analyses—must be available to permit others to critique or replicate the study (see Principle 5 ). Finally, investigators should identify potential methodological limitations (such as insensitivity to potentially important variables, missing data, and potential researcher bias).

The choice of method is not always straightforward because, across all disciplines and fields, a wide range of legitimate methods—both quantitative and qualitative—are available to the researcher. For example when considering questions about the natural universe—from atoms to cells to black holes—profoundly different methods and approaches characterize each sub-field. While investigations in the natural sciences are often dependent on the use of highly sophisticated instrumentation (e.g., particle accelerators, gene sequencers, scanning tunneling microscopes), more rudimentary methods often enable significant scientific breakthroughs. For example, in 1995 two Danish zoologists identified an entirely new phylum of animals from a species of tiny rotifer-like creatures found living on the mouthparts of lobsters, using only a hand lens and light microscope (Wilson, 1998, p. 63).

If a research conjecture or hypothesis can withstand scrutiny by multiple methods its credibility is enhanced greatly. As Webb, Campbell, Schwartz, and Sechrest (1966, pp. 173-174) phrased it: “When a hypothesis can survive the confrontation of a series of complementary methods of testing, it contains a degree of validity unattainable by one tested within the more constricted framework of a single method.” Putnam’s study (see Box 3-1 ) provides an example in which both quantitative and qualitative methods were applied in a longitudinal design (e.g., interview, survey, statistical estimate of institutional performance, analysis of legislative docu-

ments) to generate converging evidence about the effects of modernization on civic community. New theories about the periodicity of the ice ages, similarly, were informed by multiple methods (e.g., astronomical observations of cosmic dust, measurements of oxygen isotopes). The integration and interaction of multiple disciplinary perspectives—with their varying methods—often accounts for scientific progress (Wilson, 1998); this is evident, for example, in the advances in understanding early reading skills described in Chapter 2 . This line of work features methods that range from neuroimaging to qualitative classroom observation.

We close our discussion of this principle by noting that in many sciences, measurement is a key aspect of research method. This is true for many research endeavors in the social sciences and education research, although not for all of them. If the concepts or variables are poorly specified or inadequately measured, even the best methods will not be able to support strong scientific inferences. The history of the natural sciences is one of remarkable development of concepts and variables, as well as the tools (instrumentation) to measure them. Measurement reliability and validity is particularly challenging in the social sciences and education (Messick, 1989). Sometimes theory is not strong enough to permit clear specification and justification of the concept or variable. Sometimes the tool (e.g., multiple-choice test) used to take the measurement seriously under-represents the construct (e.g., science achievement) to be measured. Sometimes the use of the measurement has an unintended social consequence (e.g., the effect of teaching to the test on the scope of the curriculum in schools).

And sometimes error is an inevitable part of the measurement process. In the physical sciences, many phenomena can be directly observed or have highly predictable properties; measurement error is often minimal. (However, see National Research Council [1991] for a discussion of when and how measurement in the physical sciences can be imprecise.) In sciences that involve the study of humans, it is essential to identify those aspects of measurement error that attenuate the estimation of the relationships of interest (e.g., Shavelson, Baxter, and Gao, 1993). By investigating those aspects of a social measurement that give rise to measurement error, the measurement process itself will often be improved. Regardless of field of study, scientific measurements should be accompanied by estimates of uncertainty whenever possible (see Principle 4 below).

SCIENTIFIC PRINCIPLE 4 Provide Coherent, Explicit Chain of Reasoning

The extent to which the inferences that are made in the course of scientific work are warranted depends on rigorous reasoning that systematically and logically links empirical observations with the underlying theory and the degree to which both the theory and the observations are linked to the question or problem that lies at the root of the investigation. There

is no recipe for determining how these ingredients should be combined; instead, what is required is the development of a logical “chain of reasoning” (Lesh, Lovitts, and Kelly, 2000) that moves from evidence to theory and back again. This chain of reasoning must be coherent, explicit (one that another researcher could replicate), and persuasive to a skeptical reader (so that, for example, counterhypotheses are addressed).

All rigorous research—quantitative and qualitative—embodies the same underlying logic of inference (King, Keohane, and Verba, 1994). This inferential reasoning is supported by clear statements about how the research conclusions were reached: What assumptions were made? How was evidence judged to be relevant? How were alternative explanations considered or discarded? How were the links between data and the conceptual or theoretical framework made?

The nature of this chain of reasoning will vary depending on the design of the study, which in turn will vary depending on the question that is being investigated. Will the research develop, extend, modify, or test a hypothesis? Does it aim to determine: What works? How does it work? Under what circumstances does it work? If the goal of the research is to test a hypothesis, stated in the form of an “if-then” rule, successful inference may depend on measuring the extent to which the rule predicts results under a variety of conditions. If the goal is to produce a description of a complex system, such as a subcellular organelle or a hierarchical social organization, successful inference may rather depend on issues of fidelity and internal consistency of the observational techniques applied to diverse components and the credibility of the evidence gathered. The research design and the inferential reasoning it enables must demonstrate a thorough understanding of the subtleties of the questions to be asked and the procedures used to answer them.

Muller (1994), for example, collected data on the inclination of the Earth’s orbit over a 100,000 year cycle, correlated it with the occurrence of ice ages, ruled out the plausibility of orbital eccentricity as a cause for the occurrence of ice ages, and inferred that the bounce in the Earth’s orbit likely caused the ice ages (see Box 3-2 ). Putnam used multiple methods to subject to rigorous testing his hypotheses about what affects the success or failure of democratic institutions as they develop in diverse social environments to rigorous testing, and found the weight of the evidence favored

the assertion that civic tradition matters more than economic affluence (see Box 3-1 ). And Baumeister, Bratslavsky, Muraven, and Tice (1998) compared three competing theories and used randomized experiments to conclude that a “psychic energy” hypothesis best explained the important psychological characteristic of “will power” (see “ Application of the Principles ”).

This principle has several features worthy of elaboration. Assumptions underlying the inferences made should be clearly stated and justified. Moreover, choice of design should both acknowledge potential biases and plan for implementation challenges.

Estimates of error must also be made. Claims to knowledge vary substantially according to the strength of the research design, theory, and control of extraneous variables and by systematically ruling out possible alternative explanations. Although scientists always reason in the presence of uncertainty, it is critical to gauge the magnitude of this uncertainty. In the physical and life sciences, quantitative estimates of the error associated with conclusions are often computed and reported. In the social sciences and education, such quantitative measures are sometimes difficult to generate; in any case, a statement about the nature and estimated magnitude of error must be made in order to signal the level of certainty with which conclusions have been drawn.

Perhaps most importantly, the reasoning about evidence should identify, consider, and incorporate, when appropriate, the alternative, competing explanations or rival “answers” to the research question. To make valid inferences, plausible counterexplanations must be dealt with in a rational, systematic, and compelling way. 5 The validity—or credibility—of a hypothesis is substantially strengthened if alternative counterhypotheses can be ruled out and the favored one thereby supported. Well-known research designs (e.g., Campbell and Stanley [1963] in educational psychology; Heckman [1979, 1980a, 1980b, 2001] and Goldberger [1972, 1983] in

economics; and Rosenbaum and Rubin [1983, 1984] in statistics) have been crafted to guard researchers against specific counterhypotheses (or “threats to validity”). One example, often called “selectivity bias,” is the counterhypothesis that differential selection (not the treatment) caused the outcome—that participants in the experimental treatment systematically differed from participants in the traditional (control) condition in ways that mattered importantly to the outcome. A cell biologist, for example, might unintentionally place (select) heart cells with a slight glimmer into an experimental group and others into a control group, thus potentially biasing the comparison between the groups of cells. The potential for a biased—or unfair—comparison arises because the shiny cells could differ systematically from the others in ways that affect what is being studied.

Selection bias is a pervasive problem in the social sciences and education research. To illustrate, in studying the effects of class-size reduction, credentialed teachers are more likely to be found in wealthy school districts that have the resources to reduce class size than in poor districts. This fact raises the possibility that higher achievement will be observed in the smaller classes due to factors other than class size (e.g.. teacher effects). Random assignment to “treatment” is the strongest known antidote to the problem of selection bias (see Chapter 5 ).

A second counterhypothesis contends that something in the research participants’ history that co-occurred with the treatment caused the outcome, not the treatment itself. For example, U.S. fourth-grade students outperformed students in others countries on the ecology subtest of the Third International Mathematics and Science Study. One (popular) explanation of this finding was that the effect was due to their schooling and the emphasis on ecology in U.S. elementary science curricula. A counter-hypothesis, one of history, posits that their high achievement was due to the prevalence of ecology in children’s television programming. A control group that has the same experiences as the experimental group except for the “treatment” under study is the best antidote for this problem.

A third prevalent class of alternative interpretations contends that an outcome was biased by the measurement used. For example, education effects are often judged by narrowly defined achievement tests that focus on factual knowledge and therefore favor direct-instruction teaching tech-

niques. Multiple achievement measures with high reliability (consistency) and validity (accuracy) help to counter potential measurement bias.

The Tennessee class-size study was designed primarily to eliminate all possible known explanations, except for reduced class size, in comparing the achievement of children in regular classrooms against achievement in reduced size classrooms. It did this. Complications remained, however. About ten percent of students moved out of their originally assigned condition (class size), weakening the design because the comparative groups did not remain intact to enable strict comparisons. However, most scholars who subsequently analyzed the data (e.g., Krueger and Whitmore, 2001), while limited by the original study design, suggested that these infidelities did not affect the main conclusions of the study that smaller class size caused slight improvements in achievement. Students in classes of 13-17 students outperformed their peers in larger classes, on average, by a small margin.

SCIENTIFIC PRINCIPLE 5 Replicate and Generalize Across Studies

Replication and generalization strengthen and clarify the limits of scientific conjectures and theories. By replication we mean, at an elementary level, that if one investigator makes a set of observations, another investigator can make a similar set of observations under the same conditions. Replication in this sense comes close to what psychometricians call reliability—consistency of measurements from one observer to another, from one task to another parallel task, from one occasion to another occasion. Estimates of these different types of reliability can vary when measuring a given construct: for example, in measuring performance of military personnel (National Research Council, 1991), multiple observers largely agreed on what they observed within tasks; however, enlistees’ performance across parallel tasks was quite inconsistent.

At a somewhat more complex level, replication means the ability to repeat an investigation in more than one setting (from one laboratory to another or from one field site to a similar field site) and reach similar conclusions. To be sure, replication in the physical sciences, especially with inanimate objects, is more easily achieved than in social science or education; put another way, the margin of error in social science replication is usually

much greater than in physical science replication. The role of contextual factors and the lack of control that characterizes work in the social realm require a more nuanced notion of replication. Nevertheless, the typically large margins of error in social science replications do not preclude their identification.

Having evidence of replication, an important goal of science is to understand the extent to which findings generalize from one object or person to another, from one setting to another, and so on. To this end, a substantial amount of statistical machinery has been built both to help ensure that what is observed in a particular study is representative of what is of larger interest (i.e., will generalize) and to provide a quantitative measure of the possible error in generalizing. Nonstatistical means of generalization (e.g., triangulation, analytic induction, comparative analysis) have also been developed and applied in genres of research, such as ethnography, to understand the extent to which findings generalize across time, space, and populations. Subsequent applications, implementations, or trials are often necessary to assure generalizability or to clarify its limits. For example, since the Tennessee experiment, additional studies of the effects of class size reduction on student learning have been launched in settings other than Tennessee to assess the extent to which the findings generalize (e.g., Hruz, 2000).

In the social sciences and education, many generalizations are limited to particular times and particular places (Cronbach, 1975). This is because the social world undergoes rapid and often significant change; social generalizations, as Cronbach put it, have a shorter “half-life” than those in the physical world. Campbell and Stanley (1963) dubbed the extent to which the treatment conditions and participant population of a study mirror the world to which generalization is desired the “external validity” of the study. Consider, again, the Tennessee class-size research; it was undertaken in a set of schools that had the desire to participate, the physical facilities to accommodate an increased number of classrooms, and adequate teaching staff. Governor Wilson of California “overgeneralized” the Tennessee study, ignoring the specific experimental conditions of will and capacity and implemented class-size reduction in more than 95 percent of grades K-3 in the state. Not surprisingly, most researchers studying California have

concluded that the Tennessee findings did not entirely generalize to a different time, place, and context (see, e.g., Stecher and Bohrnstedt, 2000). 6

SCIENTIFIC PRINCIPLE 6 Disclose Research to Encourage Professional Scrutiny and Critique

We argue in Chapter 2 that a characteristic of scientific knowledge accumulation is its contested nature. Here we suggest that science is not only characterized by professional scrutiny and criticism, but also that such criticism is essential to scientific progress. Scientific studies usually are elements of a larger corpus of work; furthermore, the scientists carrying out a particular study always are part of a larger community of scholars. Reporting and reviewing research results are essential to enable wide and meaningful peer review. Results are traditionally published in a specialty journal, in books published by academic presses, or in other peer-reviewed publications. In recent years, an electronic version may accompany or even substitute for a print publication. 7 Results may be debated at professional conferences. Regardless of the medium, the goals of research reporting are to communicate the findings from the investigation; to open the study to examination, criticism, review, and replication (see Principle 5 ) by peer investigators; and ultimately to incorporate the new knowledge into the prevailing canon of the field. 8

The goal of communicating new knowledge is self-evident: research results must be brought into the professional and public domain if they are to be understood, debated, and eventually become known to those who could fruitfully use them. The extent to which new work can be reviewed and challenged by professional peers depends critically on accurate, comprehensive, and accessible records of data, method, and inferential reasoning. This careful accounting not only makes transparent the reasoning that led to conclusions—promoting its credibility—but it also allows the community of scientists and analysts to comprehend, to replicate, and otherwise to inform theory, research, and practice in that area.

Many nonscientists who seek guidance from the research community bemoan what can easily be perceived as bickering or as an indication of “bad” science. Quite the contrary: intellectual debate at professional meetings, through research collaborations, and in other settings provide the means by which scientific knowledge is refined and accepted; scientists strive for an “open society” where criticism and unfettered debate point the way to advancement. Through scholarly critique (see, e.g., Skocpol, 1996) and debate, for example, Putnam’s work has stimulated a series of articles, commentary, and controversy in research and policy circles about the role of “social capital” in political and other social phenomena (Winter, 2000). And the Tennessee class size study has been the subject of much scholarly debate, leading to a number of follow-on analyses and launching new work that attempts to understand the process by which classroom behavior may shift in small classes to facilitate learning. However, as Lagemann (2000) has observed, for many reasons the education research community has not been nearly as critical of itself as is the case in other fields of scientific study.

APPLICATION OF THE PRINCIPLES

The committee considered a wide range of literature and scholarship to test its ideas about the guiding principles. We realized, for example, that empiricism, while a hallmark of science, does not uniquely define it. A poet can write from first-hand experience of the world, and in this sense is an empiricist. And making observations of the world, and reasoning about their experience, helps both literary critics and historians create the

interpretive frameworks that they bring to bear in their scholarship. But empirical method in scientific inquiry has different features, like codified procedures for making observations and recognizing sources of bias associated with particular methods, 9 and the data derived from these observations are used specifically as tools to support or refute knowledge claims. Finally, empiricism in science involves collective judgments based on logic, experience, and consensus.

Another hallmark of science is replication and generalization. Humanists do not seek replication, although they often attempt to create work that generalizes (say) to the “human condition.” However, they have no formal logic of generalization, unlike scientists working in some domains (e.g., statistical sampling theory). In sum, it is clear that there is no bright line that distinguishes science from nonscience or high-quality science from low-quality science. Rather, our principles can be used as general guidelines for understanding what can be considered scientific and what can be considered high-quality science (see, however, Chapters 4 and 5 for an elaboration).

To show how our principles help differentiate science from other forms of scholarship, we briefly consider two genres of education inquiry published in refereed journals and books. We do not make a judgment about the worth of either form of inquiry; although we believe strongly in the merits of scientific inquiry in education research and more generally, that “science” does not mean “good.” Rather, we use them as examples to illustrate the distinguishing character of our principles of science. The first— connoisseurship —grew out of the arts and humanities (e.g., Eisner, 1991) and does not claim to be scientific. The second— portraiture —claims to straddle the fence between humanistic and scientific inquiry (e.g., Lawrence-Lightfoot and Davis, 1997).

Eisner (1991, p. 7) built a method for education inquiry firmly rooted in the arts and humanities, arguing that “there are multiple ways in which the world can be known: Artists, writers, and dancers, as well as scientists, have important things to tell about the world.” His method of inquiry combines connoisseurship (the art of appreciation), which “aims to

appreciate the qualities . . . that constitute an act, work, or object and, typically . . . to relate these to the contextual and antecedent conditions” (p. 85) with educational criticism (the art of disclosure), which provides “connoisseurship with a public face” (p. 85). The goal of this genre of research is to enable readers to enter an event and to participate in it. To this end, the educational critic—through educational connoisseurship— must capture the key qualities of the material, situation, and experience and express them in text (“criticism”) to make what the critic sees clear to others. “To know what schools are like, their strengths and their weaknesses, we need to be able to see what occurs in them, and we need to be able to tell others what we have seen in ways that are vivid and insightful” (Eisner, 1991, p. 23, italics in original).

The grounds for his knowledge claims are not those in our guiding principles. Rather, credibility is established by: (1) structural corroboration—“multiple types of data are related to each other” (p. 110) and “ disconfirming evidence and contradictory interpretations ” (p. 111; italics in original) are considered; (2) consensual validation—“agreement among competent others that the description, interpretation, evaluation, and thematics of an educational situation are right” (p. 112); and (3) referential adequacy— “the extent to which a reader is able to locate in its subject matter the qualities the critic addresses and the meanings he or she ascribes to these” (p. 114). While sharing some features of our guiding principles (e.g., ruling out counterinterpretations to the favored interpretation), this humanistic approach to knowledge claims builds on a very different epistemology; the key scientific concepts of reliability, replication, and generalization, for example, are quite different. We agree with Eisner that such approaches fall outside the purview of science and conclude that our guiding principles readily distinguish them.

Portraiture (Lawrence-Lightfoot, 1994; Lawrence-Lightfoot and Davis, 1997) is a qualitative research method that aims to “record and interpret the perspectives and experience of the people they [the researchers] are studying, documenting their [the research participants’] voices and their visions—their authority, knowledge, and wisdom” (Lawrence-Lightfoot and Davis, 1997, p. xv). In contrast to connoisseurship’s humanist orientation, portraiture “seeks to join science and art” (Lawrence-Lightfoot and Davis, 1997, p. xv) by “embracing the intersection of aesthetics and empiricism” (p. 6). The standard for judging the quality of portraiture is authenticity,

“. . . capturing the essence and resonance of the actors’ experience and perspective through the details of action and thought revealed in context” (p. 12). When empirical and literary themes come together (called “resonance”) for the researcher, the actors, and the audience, “we speak of the portrait as achieving authenticity” (p. 260).

In I’ve Known Rivers , Lawrence-Lightfoot (1994) explored the life stories of six men and women:

. . . using the intensive, probing method of ‘human archeology’—a name I [Lawrence-Lightfoot] coined for this genre of portraiture as a way of trying to convey the depth and penetration of the inquiry, the richness of the layers of human experience, the search for ancestral and generational artifacts, and the painstaking, careful labor that the metaphorical dig requires. As I listen to the life stories of these individuals and participate in the ‘co-construction’ of narrative, I employ the themes, goals, and techniques of portraiture. It is an eclectic, interdisciplinary approach, shaped by the lenses of history, anthropology, psychology and sociology. I blend the curiosity and detective work of a biographer, the literary aesthetic of a novelist, and the systematic scrutiny of a researcher (p. 15).

Some scholars, then, deem portraiture as “scientific” because it relies on the use of social science theory and a form of empiricism (e.g., interview). While both empiricism and theory are important elements of our guiding principles, as we discuss above, they are not, in themselves, defining. The devil is in the details. For example, independent replication is an important principle in our framework but is absent in portraiture in which researcher and subject jointly construct a narrative. Moreover, even when our principles are manifest, the specific form and mode of application can make a big difference. For example, generalization in our principles is different from generalization in portraiture. As Lawrence-Lightfoot and Davis (1997) point out, generalization as used in the social sciences does not fit portraiture. Generalization in portraiture “. . . is not the classical conception . . . where the investigator uses codified methods for generalizing from specific findings to a universe, and where there is little interest in findings that reflect only the characteristics of the sample. . . .” By contrast, the portraitist seeks to “document and illuminate the complexity

and detail of a unique experience or place, hoping the audience will see itself reflected in it, trusting that the readers will feel identified. The portraitist is very interested in the single case because she believes that embedded in it the reader will discover resonant universal themes” (p. 15). We conclude that our guiding principles would distinguish portraiture from what we mean by scientific inquiry, although it, like connoisseurship, has some traits in common.

To this point, we have shown how our principles help to distinguish science and nonscience. A large amount of education research attempts to base knowledge claims on science; clearly, however, there is great variation with respect to scientific rigor and competence. Here we use two studies to illustrate how our principles demonstrate this gradation in scientific quality.

The first study (Carr, Levin, McConnachie, Carlson, Kemp, Smith, and McLaughlin, 1999) reported on an educational intervention carried out on three nonrandomly selected individuals who were suffering severe behavioral disorders and who were residing in group-home settings. Since earlier work had established remedial procedures involving “simulations and analogs of the natural environment” (p. 6), the focus of the study was on the generalizability (or external validity) to the “real world” of the intervention (places, caregivers).

Over a two to three week period, “baseline” frequencies of their problem behaviors were established, these behaviors were remeasured after an intervention lasting for some years was carried out. The researchers took a third measurement during the maintenance phase of the study. While care was taken in describing behavioral observations, variable construction and reliability, the paper reporting on the study did not provide clear, detailed depictions of the interventions or who carried them out (research staff or staff of the group homes). Furthermore, no details were given of the changes in staffing or in the regimens of the residential settings—changes that were inevitable over a period of many years (the timeline itself was not clearly described). Finally, in the course of daily life over a number of years, many things would have happened to each of the subjects, some of which might be expected to be of significance to the study, but none of them were documented. Over the years, too, one might expect some developmental changes to occur in the aggressive behavior displayed by the research subjects, especially in the two teenagers. In short, the study focused on

generalizability at too great an expense relative to internal validity. In the end, there were many threats to internal validity in this study, and so it is impossible to conclude (as the authors did) from the published report that the “treatment” had actually caused the improvement in behavior that was noted.

Turning to a line of work that we regard as scientifically more successful, in a series of four randomized experiments, Baumeister, Bratslavsky, Muraven, and Tice (1998) tested three competing theories of “will power” (or, more technically, “self-regulation”)—the psychological characteristic that is posited to be related to persistence with difficult tasks such as studying or working on homework assignments. One hypothesis was that will power is a developed skill that would remain roughly constant across repeated trials. The second theory posited a self-control schema “that makes use of information about how to alter one’s own response” (p. 1254) so that once activated on one trial, it would be expected to increase will power on a second trial. The third theory, anticipated by Freud’s notion of the ego exerting energy to control the id and superego, posits that will power is a depletable resource—it requires the use of “psychic energy” so that performance from trial 1 to trial 2 would decrease if a great deal of will power was called for on trial 1. In one experiment, 67 introductory psychology students were randomly assigned to a condition in which either no food was present or both radishes and freshly baked chocolate chip cookies were present, and the participants were instructed either to eat two or three radishes (resisting the cookies) or two or three cookies (resisting the radishes). Immediately following this situation, all participants were asked to work on two puzzles that unbeknownst to them, were unsolvable, and their persistence (time) in working on the puzzles was measured. The experimental manipulation was checked for every individual participating by researchers observing their behavior through a one-way window. The researchers found that puzzle persistence was the same in the control and cookie conditions and about 2.5 times as long, on average, as in the radish condition, lending support to the psychic energy theory—arguably, resisting the temptation to eat the cookies evidently had depleted the reserve of self-control, leading to poor performance on the second task. Later experiments extended the findings supporting the energy theory to situations involving choice, maladaptive performance, and decision making.

However, as we have said, no single study or series of studies satisfy all of our guiding principles, and these will power experiments are no exception. They all employed small samples of participants, all drawn from a college population. The experiments were contrived—the conditions of the study would be unlikely outside a psychology laboratory. And the question of whether these findings would generalize to more realistic (e.g., school) settings was not addressed.

Nevertheless, the contrast in quality between the two studies, when observed through the lens of our guiding principles, is stark. Unlike the first study, the second study was grounded in theory and identified three competing answers to the question of self-regulation, each leading to a different empirically refutable claim. In doing so, the chain of reasoning was made transparent. The second study, unlike the first, used randomized experiments to address counterclaims to the inference of psychic energy, such as selectivity bias or different history during experimental sessions. Finally, in the second study, the series of experiments replicated and extended the effects hypothesized by the energy theory.

CONCLUDING COMMENT

Nearly a century ago, John Dewey (1916) captured the essence of the account of science we have developed in this chapter and expressed a hopefulness for the promise of science we similarly embrace:

Our predilection for premature acceptance and assertion, our aversion to suspended judgment, are signs that we tend naturally to cut short the process of testing. We are satisfied with superficial and immediate short-visioned applications. If these work out with moderate satisfactoriness, we are content to suppose that our assumptions have been confirmed. Even in the case of failure, we are inclined to put the blame not on the inadequacy and incorrectness of our data and thoughts, but upon our hard luck and the hostility of circumstances. . . . Science represents the safeguard of the [human] race against these natural propensities and the evils which flow from them. It consists of the special appliances and methods... slowly worked out in order to conduct reflection under conditions whereby its procedures and results are tested.

Researchers, historians, and philosophers of science have debated the nature of scientific research in education for more than 100 years. Recent enthusiasm for "evidence-based" policy and practice in education—now codified in the federal law that authorizes the bulk of elementary and secondary education programs—have brought a new sense of urgency to understanding the ways in which the basic tenets of science manifest in the study of teaching, learning, and schooling.

Scientific Research in Education describes the similarities and differences between scientific inquiry in education and scientific inquiry in other fields and disciplines and provides a number of examples to illustrate these ideas. Its main argument is that all scientific endeavors share a common set of principles, and that each field—including education research—develops a specialization that accounts for the particulars of what is being studied. The book also provides suggestions for how the federal government can best support high-quality scientific research in education.

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

Switch between the Original Pages , where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.