Scientific claims in biology and health range from rock-solid to dangerously misleading, and telling them apart requires knowing what kind of evidence sits behind each one. A claim like “this drug lowers blood pressure” rests on a very different foundation than “people who eat more blueberries live longer” or “gut bacteria cause depression,” even though all three sound equally confident when they appear in a headline. The type of study, the way risk is reported, and the distance between what was actually measured and what is claimed all shape how seriously you should take any given finding.
Association Claims Are Not Causation Claims
The single most common source of confusion in health science is the gap between correlation and causation. When a study reports that people who drink coffee live longer, it has found an association. That is not the same as showing that coffee extends life. The coffee drinkers might share other habits, like exercising more or having better access to healthcare, that actually explain the survival difference. Researchers have formal tools for thinking through whether an association is likely causal, the most influential being the framework published by Austin Bradford Hill in 1965. His nine “viewpoints,” including the strength of the association, whether a plausible biological mechanism exists, and whether the effect follows a dose-response pattern, remain the most frequently cited framework for causal inference in epidemiology decades later.1PubMed Central. Applying the Bradford Hill criteria in the 21st century: how data integration has changed causal inference in molecular epidemiology
These criteria are useful, but they are not a checklist with a passing score. A modern reassessment found that some of Hill’s viewpoints hold up better than others. Strength of association, temporality (the cause must precede the effect), biological plausibility, and experimental evidence remain central. But dose-response, once considered strong evidence for causation, can arise from confounding alone. And “specificity,” the idea that one cause should lead to one effect, rarely holds in real biology.2PubMed Central. Assessing causality in epidemiology: revisiting Bradford Hill to incorporate developments in causal thinking The practical takeaway is that any single study claiming “X causes Y” based on observational data alone deserves skepticism, no matter how large the study is or how impressive the statistical analysis looks.
When Observational Studies and Randomized Trials Disagree
The gold standard for testing a causal claim in medicine is the randomized controlled trial: you assign people randomly to receive a treatment or a placebo, so the groups are comparable in every way except the treatment itself. Observational studies, where researchers simply watch what people do and track outcomes, sit lower in the evidence hierarchy because hidden differences between groups can distort the results.3PubMed Central. Understanding the Levels of Evidence in Medical Research
How often do the two types of study actually disagree? Less often than you might expect. Two landmark analyses found that for most clinical questions, the average results from observational studies were remarkably similar to those of randomized trials.4PubMed Central. Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs5PubMed. A comparison of observational studies and randomized, controlled trials But when they do disagree, the consequences can be enormous.
The most famous example is hormone replacement therapy. By the late 1990s, a meta-analysis of 25 observational studies suggested that HRT reduced the risk of coronary heart disease by roughly 30%. Then, in 2002, the Women’s Health Initiative, a large randomized trial of about 16,000 postmenopausal women, was stopped early because HRT was actually increasing the risk of breast cancer and heart disease. The likely explanation is selection bias: women who chose to take HRT in the observational studies tended to come from higher socioeconomic groups with better lifelong access to preventive care. Their better health outcomes had little to do with the hormones themselves.6JACC: Cardiovascular Interventions. Randomized Clinical Trials and Observational Studies: Guidelines for Assessing Respective Strengths and Limitations This case reshaped how doctors and researchers think about the limits of observational evidence, especially for preventive therapies where the people who choose to use them are probably different from the people who don’t in ways that are hard to measure.
Relative Risk Versus Absolute Risk
Even when a finding is solid, the way it gets communicated can make it sound far more dramatic than it is. A drug that “cuts your risk in half” sounds transformative. But if the risk drops from 2% to 1%, the absolute change is just one percentage point. Both descriptions are technically accurate, yet they produce very different reactions. Research on how people process risk information confirms that when health effects are reported only in relative terms, people consistently overestimate the benefit.7PubMed Central. Understanding and Communicating Risk: Assessing Both Relative and Absolute Risk Is Absolutely Necessary
This is not just an academic concern. Drug advertisements, news stories, and even some clinical guidelines lean on relative risk reductions because they are more impressive. If you see a claim that a supplement reduces your risk of some disease by 40%, ask two questions: what was the baseline risk, and what is the absolute difference? A 40% relative reduction from a 50% baseline risk is genuinely life-changing. A 40% relative reduction from a 0.5% baseline risk means one fewer person in 400 is affected. Both deserve attention, but they demand very different levels of urgency.
Why Animal Studies Often Don’t Translate to Humans
A large number of health claims originate in animal studies or laboratory experiments on cells in a dish. These are valuable for understanding biological mechanisms, but the gap between what happens in a mouse and what happens in a person is routinely underestimated. Translational failure, where a promising preclinical finding collapses in human trials, remains one of the biggest barriers in medical research.8PubMed Central. Can patient-derived in vitro models improve clinical translation in critical care research when used before animal studies?
The reasons are straightforward. Animal models cannot fully mirror the complexity and variability of human disease, and they are poor predictors of whether a therapy will actually work in people.9PubMed Central. Why animal model studies are lost in translation A mouse with an artificially induced tumor is not the same as a person who developed cancer over decades amid a unique combination of genetic, environmental, and lifestyle factors. Systematic assessments of how well animal results translate to humans remain limited, but the need for them is widely recognized.10PubMed Central. Understanding the limits of animal models as predictors of human biology: lessons learned from the sbv IMPROVER Species Translation Challenge
When you encounter a headline claiming a substance “kills cancer cells,” that study was almost certainly done in a petri dish or in mice. It may well be an interesting lead for future research, but it is a long way from a treatment that works in people. The same applies to claims about compounds that “reverse aging” or “repair brain damage” based on rodent models. Worth following, but not worth changing your behavior over.
The Problem with Surrogate Endpoints
Many clinical trials do not directly measure the outcomes patients actually care about, like living longer or feeling better. Instead, they measure surrogate endpoints: lab values or biomarkers that are believed to predict those outcomes. Blood pressure is a well-validated surrogate for heart attacks and strokes. But not all surrogates are equally trustworthy. A biomarker that is strongly correlated with a disease outcome in observational data, but is not actually on the causal pathway of the disease, can give misleading information about whether a treatment truly works.11PubMed Central. Biomarkers and Surrogate Endpoints In Clinical Trials
The validity of a surrogate endpoint requires careful demonstration that improving the surrogate reliably improves the clinical outcome. When that validation is incomplete, surprises follow.12Journal of Clinical Epidemiology. Surrogate endpoints: a key concept in clinical epidemiology A clear example comes from cancer drugs granted accelerated approval by the FDA. Among cancer drugs approved on an accelerated basis between 2013 and 2017, roughly 41% failed to show an improvement in overall survival or quality of life in follow-up trials, even after more than five years. About 60% of the drugs that were eventually converted to regular approval had their conversions based on surrogate measures rather than direct evidence of patients living longer or better.13PubMed Central. Clinical Benefit and Regulatory Outcomes of Cancer Drugs Receiving Accelerated Approval That does not mean surrogate endpoints are useless. They speed up drug development enormously for serious diseases. But they introduce a real risk that a treatment looks effective on paper while delivering little to no benefit where it counts.
The Reproducibility Problem
One of the more sobering discoveries in recent years is how often published scientific findings, particularly in the life sciences, fail to hold up when another team tries to replicate them. A major replication project in preclinical cancer biology found that only about 46% of the effects it attempted to replicate were successfully reproduced.14PubMed Central. Investigating the replicability of preclinical cancer biology An independent economic analysis estimated that more than half of all preclinical research in the United States is irreproducible, translating to roughly $28 billion per year spent on findings that don’t hold up.15PubMed Central. The Economics of Reproducibility in Preclinical Research
Several factors contribute. One is p-hacking, where researchers collect or select data or tweak their statistical analyses until a nonsignificant result crosses the threshold into significance.16PubMed Central. The extent and consequences of p-hacking in science Another is publication bias: journals are far more willing to publish studies that found a positive result than studies that found nothing. In one remarkable case, a research group in oxytocin studies openly acknowledged that their published portfolio had become less and less representative of their actual results, because studies with null findings stayed unpublished in their file drawers.17PubMed. Is there a Publication Bias in Behavioural Intranasal Oxytocin Research on Humans? Opening the File Drawer of One Laboratory If the studies showing no effect never see the light of day, the published literature makes a treatment look more effective or a phenomenon look more reliable than it actually is.
For you as a reader, the implication is that a single study, even one published in a reputable journal, is best treated as a data point rather than a settled finding. When multiple independent groups reproduce the same result, confidence grows. When a finding has only been reported once, it remains provisional regardless of how exciting it sounds.
How Headlines Exaggerate Science
Scientific findings are distorted long before they reach you. University press releases are a major amplification point. A study of health-related press releases from major research universities found that about 40% contained exaggerated advice, a third inflated correlations into causal claims, and more than a third extrapolated animal results to humans without justification.18PubMed Central. The association between exaggeration in health related science news and academic press releases: retrospective observational study When the press release was exaggerated, the news coverage that followed was dramatically more likely to be exaggerated as well. For causal claims in particular, news stories were about 20 times more likely to overstate the findings when the press release had already done so.
A follow-up study examined whether these patterns held across a broader sample and found that the link between press release exaggeration and news exaggeration replicated for causal claims and for extrapolation from animal studies to humans, though it did not replicate for advice exaggeration.19PubMed Central. The association between exaggeration in health-related science news and academic press releases: a replication study A separate analysis confirmed the strength of the pattern: when press releases exaggerated the causal language in correlation-based studies, news stories were nearly 11 times more likely to contain the same exaggeration.20PubMed Central. Exaggerations and Caveats in Press Releases and Health-Related Science News
This means the information pipeline from lab to headline has systematic leaks at every stage. The original paper might carefully note that results were correlational and observed only in mice. The university press office might omit those qualifications to make the story more newsworthy. The journalist, working from the press release under a tight deadline, might not read the original paper at all. By the time you see a headline that says “eating chocolate prevents heart disease,” the actual finding was probably something like “a statistical association between self-reported chocolate intake and a heart disease biomarker in an observational cohort.” Those are very different claims.
Nutritional Epidemiology and Its Measurement Challenges
Nutrition research faces a unique credibility problem. Almost all large-scale studies of what people eat rely on self-reported instruments like food frequency questionnaires, where you try to recall how often you ate certain foods over the past year. The evidence shows consistent, systematic underreporting of calorie intake across both adults and children, and the degree of underreporting gets worse as body mass increases.21PubMed Central. Traditional Self-Reported Dietary Instruments Are Prone to Inaccuracies and New Approaches Are Needed Not all foods are underreported equally, either, though exactly which ones are systematically underreported remains unclear.
These measurement errors flow directly into the claims that emerge from nutritional studies. If people who are overweight systematically underreport certain foods, any association found between those foods and health outcomes is contaminated by measurement error.22PubMed Central. Microbiome-based correction for random errors in nutrient profiles derived from self-reported dietary assessments This is one reason why nutrition claims seem to reverse every few years. Eggs are bad, then fine, then bad again. The underlying data was never precise enough to produce stable conclusions in the first place. When you see a nutrition study claiming that eating a specific food raises or lowers your risk of a disease by some modest percentage, the noise baked into the data collection methods often rivals the size of the claimed effect.
Microbiome Research and Overstated Causality
The human gut microbiome has become one of the most hyped areas in biology. Research linking altered gut bacteria to conditions ranging from obesity and depression to autoimmune disease generates constant headlines. But for most of these links, the fundamental question of whether the altered microbiome causes the disease, results from it, or is merely a bystander remains unresolved.23Cell. Establishing or Exaggerating Causality for the Gut Microbiome: Lessons from Human Microbiota-Associated Rodents
The standard method for testing causation in microbiome research involves transplanting human gut bacteria into germ-free mice and seeing if the mice develop the same disease. A systematic review of these experiments found that 95% of published studies reported a successful transfer of disease-related traits to the mice. That number is suspiciously high and almost certainly overstates how often the microbiome is truly driving human disease. The researchers argue that the experimental setup itself biases the outcome and that the causal role of gut bacteria in human health has been exaggerated.23Cell. Establishing or Exaggerating Causality for the Gut Microbiome: Lessons from Human Microbiota-Associated Rodents Philosophical scrutiny of the causal claims in microbiome science echoes this concern, arguing that attributing a causal role to entire microbial communities rather than specific mechanisms needs far more rigorous justification than the field has provided so far.24BioEssays. Have causal claims about the gut microbiome been over-hyped?
Genetic Risk Scores and the Limits of Prediction
Another category of claim that often sounds more definitive than it is involves genetic risk. Polygenic risk scores aggregate the effects of hundreds or thousands of small genetic variations to estimate someone’s lifetime probability of developing a disease. These scores are a genuine scientific advance, but their current ability to discriminate between people who will and will not develop a disease is low in the general population.25PubMed Central. Polygenic risk scores: from research tools to clinical instruments Two major caveats apply. First, most scores were developed using data predominantly from people of European ancestry, so their accuracy drops for other populations. Second, translating a statistical percentile into a concrete lifetime risk for an individual involves assumptions that are difficult to verify.
When a consumer genetic testing company tells you that you have a “3x higher risk” of some condition, the claim might be technically accurate at a population level while being practically meaningless for your individual situation. A threefold increase over a very small baseline risk is still a small risk. And the prediction does not account for the lifestyle, environmental, and medical factors that actually determine most health outcomes. Genetic determinism, the idea that your DNA seals your fate, is one of the most persistent misunderstandings in health communication.
Brain Imaging Claims and Overinterpretation
Neuroscience claims carry a special kind of persuasive power because brain scans look objective and scientific in a way that questionnaire-based research does not. But the most common type of brain imaging study, resting-state functional MRI, measures blood flow as an indirect proxy for neural activity and relies on correlations between brain regions. These correlation-based analyses cannot yield causal conclusions on their own, yet they are frequently used to support causal narratives, especially in clinical contexts.26bioRxiv. Resting-State fMRI and the Risk of Overinterpretation: Noise, Mechanisms, and a Missing Rosetta Stone Saying “this brain region lights up when people feel empathy” is not the same as saying the region causes empathy, and the distance between those two statements is routinely collapsed in popular science coverage.
When Scientific Consensus Changes
It is worth remembering that established scientific claims do get overturned, and that process is a feature of science, not a bug. For decades, the medical establishment believed stomach ulcers were caused by stress and spicy food. When Barry Marshall and Robin Warren proposed in the 1980s that the bacterium Helicobacter pylori was responsible, they were widely dismissed. It took years of accumulating evidence, including Marshall famously drinking a broth of the bacteria and developing gastritis, before the medical community accepted the bacterial cause. The subsequent recognition that H. pylori was responsible not only for most ulcers but also for changes in patterns of gastric disease reshaped an entire field.27PubMed Central. History of Helicobacter pylori, duodenal ulcer, gastric ulcer and gastric cancer
That story is sometimes used to argue that maverick scientists should always be given the benefit of the doubt. The actual lesson is more nuanced. Marshall and Warren succeeded because they had a testable mechanism, they gathered experimental evidence, and the claim eventually met the standard criteria for causal inference. The system was slow to accept their claim, but it did accept it once the evidence was strong enough. The process worked; it just was not fast. Most contrarian claims in health science, unlike H. pylori, do not survive scrutiny. The occasional dramatic reversal does not mean every iconoclastic idea deserves equal credibility. It means that the quality of evidence behind a claim matters more than its novelty or its conformity to the current consensus.
Evaluating Claims You Encounter in the Wild
Given everything above, a few practical habits can help you evaluate the biological and health claims you encounter daily. When a news story describes a breakthrough, look for what kind of study it was. Cells in a dish and mouse models are early-stage science. Observational studies in humans are a step closer but cannot prove causation. Randomized trials in humans are the strongest test for whether a treatment works, and systematic reviews that pool many such trials carry the most weight. Ask whether the study measured what patients actually care about, or a surrogate. Ask how risk is being reported. And ask whether the finding has been replicated.
The noise in the system is real. Press releases exaggerate. Journals favor positive results. Measurement tools in fields like nutrition are blunt. Animal findings frequently fail to hold up in people. None of this means science is broken. It means that individual scientific claims exist on a spectrum of reliability, and the confident-sounding headline rarely conveys where on that spectrum the claim actually sits. Your best protection is not skepticism about science itself, but literacy about what different types of evidence can and cannot tell you.