Limitations in an experiment are the factors, conditions, and design choices that constrain how confidently you can interpret the results and how broadly you can apply them. Every experiment has them, from a high school chemistry project to a billion-dollar clinical trial. They include things like small sample sizes, uncontrolled variables, biased participant pools, measurement tools that only approximate what you really want to measure, and ethical rules that prevent you from running the ideal version of the study. Understanding what these limitations are, and why researchers go out of their way to acknowledge them, is key to reading any scientific finding with clear eyes.
Limitations Versus Delimitations
One common source of confusion is the difference between a limitation and a delimitation. Delimitations are the boundaries a researcher chooses to set: deciding to study only adults over 50, focusing on one geographic region, or excluding people taking a certain medication. Those are deliberate scope decisions. Limitations, by contrast, are the things outside the researcher’s control: a tight budget, unexpected events, time constraints, a lack of prior research on the topic, or unavailable technology.1AJE. Scope and Delimitations in Research Both shape what the experiment can and cannot tell you, but the distinction matters because delimitations are acknowledged up front as part of the design, while limitations are weaknesses the researcher discovers or must live with.
Small Sample Sizes and Statistical Power
One of the most frequently cited limitations in any study is a small sample size. When you study only a handful of people or run only a few trials, random variation can easily swamp the real effect you are looking for. A study might conclude “we found no difference between the two groups” not because there truly is no difference, but because the study was too small to detect one. This is sometimes called a Type II error, and it is one of the most common ways that real effects slip through the cracks of scientific research.2PubMed Central. Sample size, power and effect size revisited: simplified and practical approaches in pre-clinical, clinical and laboratory studies
Small samples also make results less stable. If you flip a coin ten times, getting seven heads would not be shocking. Flip it ten thousand times, and getting 70% heads would be almost impossible unless the coin is rigged. The same logic applies to experiments: the fewer data points you have, the more your results bounce around from run to run. This is why replication, repeating the experiment independently, matters so much. A single small study that finds a dramatic effect could easily be a statistical fluke.
Who Was Actually Studied
Even a large sample can be a problem if it does not represent the population you want to draw conclusions about. This is where sampling bias enters the picture. The most famous example in the social sciences is the overreliance on so-called WEIRD participants: people from Western, educated, industrialized, rich, and democratic societies. Research published in leading psychology journals overwhelmingly draws from these populations, and the findings are then presented as if they apply to humans in general.3PubMed Central. Toward a psychology of Homo sapiens: Making psychological science more representative of the human population Psychological data are dominated by samples drawn from Western nations, and the United States in particular.4PubMed Central. Beyond Western, Educated, Industrial, Rich, and Democratic (WEIRD) Psychology: Measuring and Mapping Scales of Cultural and Psychological Distance
This is not just a psychology problem. Medical trials have historically underrepresented women, older adults, and ethnic minorities. A drug tested primarily on middle-aged white men may work differently in other groups due to differences in metabolism, body composition, or genetics. When you read a study’s results, one of the first things worth checking is who the participants were. If the sample does not look like you or the group you care about, the findings may not fully transfer.
Confounding Variables
A confounding variable is something that influences both the factor you are studying and the outcome you are measuring, creating a false impression that one causes the other. The classic example: people who carry lighters are more likely to develop lung cancer, but lighters do not cause cancer. Smoking is the confounder, because it is associated with both carrying a lighter and getting cancer. To count as a true confounder, a variable has to meet three criteria: it must be a risk factor for the outcome, it must be unevenly distributed between the groups being compared, and it must not be a consequence of the exposure itself.5PubMed. Confounding: what it is and how to deal with it
Researchers try to control for confounders using techniques like randomization, matching participants across groups, or adjusting for known confounders in their statistical analysis. But here is the uncomfortable part: even when all known confounders are accounted for, residual confounding can still distort the results. One study demonstrated that after controlling for every identified confounding factor using standard statistical methods, the observed link between an exposure and an outcome could still be driven largely by leftover confounding effects.6PubMed Central. An Investigation of the Significance of Residual Confounding Effect Newer methods, such as using “negative control” exposures as a kind of diagnostic check, can partially reduce this problem, but they do not eliminate it.7American Journal of Epidemiology. A New Method for Partial Correction of Residual Confounding in Time-Series and Other Observational Studies This is one reason why observational studies, no matter how large, are treated with more caution than randomized controlled trials.
When Participants Know They Are Being Watched
Human behavior changes under observation, and this is a significant limitation in any experiment involving people. Several well-documented psychological processes can bias participant responses. Demand characteristics lead participants to guess what the experiment is “supposed” to find and then, consciously or not, behave in line with that guess. The Hawthorne effect describes the tendency for people to alter their behavior simply because they know they are part of a study. And response shift can invalidate before-and-after comparisons if the experience of being treated changes how a person evaluates their own symptoms.8Academic Press. Psychological Processes that can Bias Responses to Placebo Treatment for Pain
These effects are not minor technicalities. In pain research, for example, the mere belief that you are receiving a treatment can produce measurable physiological changes. This is one reason placebo-controlled trials exist in the first place: you need a comparison group that goes through the same motions without receiving the active ingredient, so you can separate the real drug effect from the psychological response. But even placebos introduce their own complications, as the next section discusses.
Ethical Boundaries on Study Design
Sometimes the best possible experiment is one you cannot ethically run. If you wanted to know whether a particular chemical causes cancer in humans, you would ideally expose one group and compare them against a control. That experiment will never happen, for obvious reasons. Researchers must rely instead on observational data, animal models, or natural experiments where exposure happened for reasons outside the study.
Even in less extreme cases, ethics shape study design in ways that introduce limitations. In chronic pain research, for instance, using a placebo control means that one group of patients has their pain left untreated for the duration of the trial. Researchers must weigh the importance of rigorous methodology against the harm of withholding treatment from people who are suffering.9PubMed Central. Ethical considerations in the design, execution, and analysis of clinical trials of chronic pain treatments – Section: Placebo controls and strategies to manage the placebo response This is why many trials use an “active comparator” design, testing a new treatment against the current best option rather than against nothing, but that trade-off means you learn less about the absolute effect of the new treatment.
Researcher Bias and the Value of Blinding
Bias does not only come from participants. Researchers themselves can unconsciously influence outcomes, especially when they know which group a participant belongs to. A systematic review of randomized clinical trials that included both blinded and non-blinded outcome assessors found that non-blinded assessments exaggerated treatment effects by an average of about 36% compared to blinded assessments.10PubMed. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors That is a substantial distortion, and it does not require any intentional dishonesty. A researcher who believes a treatment works may unconsciously interpret ambiguous symptoms more favorably in the treatment group, or may record borderline outcomes differently.
Double-blinding, where neither the participant nor the researcher knows who received the active treatment, is the gold standard for minimizing this problem. But blinding is not always possible. In surgical trials, the surgeon obviously knows whether they performed the real procedure. In behavioral interventions like therapy or exercise programs, both parties know what is happening. These are real structural limitations that no amount of statistical adjustment can fully overcome.
Measuring the Wrong Thing
Even when you can run a clean, well-controlled experiment, there is the question of whether you are actually measuring what you think you are measuring. In many areas of science, researchers rely on proxy measurements because the outcome they truly care about is too slow, too expensive, or too invasive to measure directly. In medicine, for example, clinical trials frequently use biomarkers as stand-ins for actual health outcomes. A cholesterol-lowering drug might be evaluated based on its ability to reduce LDL levels rather than its ability to prevent heart attacks, because waiting for heart attacks to occur takes years and requires much larger studies.11PubMed. Biomarkers as Surrogate Endpoints: Ongoing Opportunities for Validation
The problem is that a proxy does not always track perfectly with the outcome it is supposed to represent. The reliability and interpretability of a trial built around a surrogate endpoint can be compromised when researchers do not fully understand how large or how lasting an effect on the biomarker needs to be in order to translate into a real clinical benefit.12PubMed Central. Biomarkers and Surrogate Endpoints In Clinical Trials – Section: A Correlate does not a Surrogate Make A treatment can look great on the surrogate and still fail to help patients, or even harm them. This has happened more than once in the history of medicine and is a limitation that applies broadly whenever an experiment measures something adjacent to the thing it really cares about.
The Lab Versus the Real World
There is a fundamental tension in experimental science between controlling conditions and reflecting real life. A highly controlled laboratory experiment can isolate a single variable and test it cleanly, which gives the study strong internal validity: you can be confident that A caused B within the walls of the experiment. But that same level of control may strip away the messy, interacting factors that exist in the real world, weakening external validity: you are less sure that A would cause B out in real life.13American Journal of Agricultural Economics. Internal and External Validity in Economics Research: Tradeoffs between Experiments, Field Experiments, Natural Experiments, and Field Data
In psychology, this is sometimes called the “real-world or the lab” dilemma. Critics have questioned whether what happens in a tightly controlled psychology lab generalizes to how people actually behave in daily life.14PubMed Central. The ‘Real-World Approach’ and Its Problems: A Critique of the Term Ecological Validity A memory test administered in a quiet room with no distractions tells you something about memory, but it may not predict how well someone remembers information in a noisy office while juggling three tasks. Similarly, an economic experiment where college students play a simulated market game may not reflect how experienced traders behave with real money on the line. Neither approach is wrong, but each has limitations that the other partially addresses, which is why the strongest conclusions come from converging evidence across different types of studies.
Time Constraints and Short-Term Studies
Many experiments run for weeks or months when the effects they are trying to study play out over years or decades. This creates a genuine limitation, especially in medical research. A review of studies on inhaled corticosteroids found that short-term tests had limited value in predicting long-term adverse effects, even when the short-term results were highly statistically significant.15PubMed. Limitations of short-term studies in predicting long-term adverse effects of inhaled corticosteroids The body’s physiological systems have normal fluctuations that can swamp real signals in a brief study window, and some harms accumulate so gradually that they simply cannot appear in a short trial.
This limitation is not unique to pharmacology. In environmental science, a one-year pollution study cannot capture the effects of a decade of exposure. In psychology, a two-week intervention may show impressive short-term improvements that fade entirely within six months. Whenever a study’s time frame is much shorter than the natural timeline of the process being studied, any positive or negative result should be interpreted cautiously.
Post-Hoc Analysis and Data Selection Traps
After an experiment is complete, researchers face a temptation to slice the data in ways that were not part of the original plan. Post-hoc analysis, analyzing subgroups or outcomes that were not pre-specified, can generate interesting hypotheses, but it can also produce misleading results. One well-documented issue is that when researchers select data after the fact based on a particular criterion, regression to the mean can lead to erroneous conclusions. For instance, in studies of unconscious mental processing, researchers sometimes focus only on the trials where participants showed no conscious awareness of a stimulus. Because awareness fluctuates randomly, selecting only the trials where it was absent biases the remaining data in predictable ways.16PubMed Central. Regressive research: The pitfalls of post hoc data selection in the study of unconscious mental processes
A related problem involves how researchers handle confounding variables. When checking whether treatment and control groups are balanced on background characteristics, researchers sometimes manipulate which participants or data points are included until the groups look statistically balanced, a practice that has been called “reverse P-hacking.” Unlike traditional P-hacking, where the goal is to make results look significant, reverse P-hacking aims to make group differences in confounders look non-significant so that reviewers will not question the study’s validity.17PLOS Biology. Evidence that nonsignificant results are sometimes preferred: Reverse P-hacking or selective reporting? Both practices distort what the data actually show and represent a serious limitation in how some research is conducted and reported.
Why Reporting Limitations Matters
Given that every experiment has limitations, openly acknowledging them is one of the most important things a researcher can do. A paper published in Science of the Total Environment argued that a dedicated “Limitations” section should be mandatory in all scientific papers, on the grounds that it would increase honesty, openness, and transparency for both the scientific community and the public.18PubMed. A ‘Limitations’ section should be mandatory in all scientific papers In practice, the peer-review process already pushes authors in this direction: a study of randomized trial reports found that manuscripts acknowledged more of their own limitations after going through editorial handling and peer review than they did in the initial submission.19PubMed Central. Impact of peer review on discussion of study limitations and strength of claims in randomized trial reports: a before and after study
Interestingly, that same study found that while peer review led to more self-acknowledgment of limitations, it did not change the linguistic nuance of the authors’ conclusions. In other words, researchers added limitations sections but did not necessarily soften their claims. This is worth keeping in mind as a reader: the presence of a limitations section does not automatically mean the authors have fully adjusted the weight of their conclusions to match those limitations.
How Communicating Uncertainty Affects Trust
You might wonder whether being upfront about limitations and uncertainty makes the public trust science less. The evidence suggests it barely moves the needle. A series of experiments involving nearly 5,800 participants, including a field experiment on the BBC News website, found that while people do perceive greater uncertainty when it is communicated to them, the decrease in trust was small, and it was mainly associated with verbal expressions of uncertainty rather than numerical ranges.20PubMed Central. The effects of communicating uncertainty on public trust in facts and numbers Saying “we’re not sure” appears to be less damaging to public trust than many scientists fear, which undercuts one of the common justifications for downplaying limitations in public-facing communication.
Animal Models and the Translation Gap
In biomedical research, many experiments begin with animal models long before they involve humans. Mice, rats, fruit flies, and zebrafish serve as stand-ins because their biology overlaps with ours in important ways. But the overlap is imperfect. Biological pathways that function one way in a mouse may work differently in a human, and treatments that look promising in animal studies frequently fail when tested in people.21Nucleic Acids Research. Human pathways in animal models: possibilities and limitations This translation gap is one of the most expensive limitations in science: billions of dollars are spent developing drugs that pass animal testing only to wash out in human trials.
The limitation is not that animal models are useless. They are essential for understanding basic biology and for ruling out treatments that are clearly toxic. The limitation is that success in an animal model provides weaker evidence than many people assume. A headline announcing “Scientists cure cancer in mice” is describing a finding that has cleared an early and relatively low bar, not one that is ready for human application. Responsible reporting of animal studies would routinely note this limitation, but it is often glossed over in media coverage.