Umbrellaology is a fictional discipline invented to show just how slippery the line between science and non-science really is. The thought experiment, published in the journal Philosophy of Science, describes a researcher who has spent eighteen years meticulously collecting data about umbrellas and asks, point-blank, whether that counts as science.1Philosophy of Science. Umbrellaology, or, Methodology in Social Science The question sounds absurd, but it is a sharper version of a puzzle philosophers have wrestled with for centuries, known as the demarcation problem: what, exactly, separates genuine science from non-science or pseudoscience? The answer turns out to depend heavily on which philosopher you ask, what era you live in, and sometimes which courtroom you are standing in.
What Umbrellaology Actually Is
The concept comes from a short, deliberately provocative essay. Its author describes founding a new field devoted entirely to the umbrella. He and his “faithful disciples” have amassed enormous collections of data: measurements, classifications, historical records, geographic distributions of umbrella use. The data is genuine. The methods of collection are careful. There are journals, conferences, and an academic community. Everything looks like science from the outside. And yet something feels off. The essay forces readers to articulate why a rigorous, data-heavy, institutionally supported field might still fail to qualify as science.1Philosophy of Science. Umbrellaology, or, Methodology in Social Science
The trick is that umbrellaology has no theories. It collects facts but does not try to explain anything. There are no hypotheses about why people carry umbrellas, no models predicting umbrella adoption in different climates, no proposed mechanisms connecting umbrella design to social behavior. The discipline is pure description. That quality, the absence of explanatory ambition, is what makes it such a useful test case for demarcation criteria. Every major philosopher of science in the twentieth century proposed a different reason why umbrellaology should or should not count, and none of them fully agreed.
Popper and the Falsifiability Test
The most famous attempt to draw a line between science and non-science came from Karl Popper, who argued that what makes a theory scientific is not that it can be proven true, but that it can, in principle, be proven false. A claim like “all swans are white” is scientific because finding one black swan would refute it. A claim like “invisible forces guide your destiny” is not, because no observation could ever count against it.
By Popper’s standard, umbrellaology fails almost immediately. A field that only collects and catalogs data, without generating testable predictions about the world, never puts itself at risk of being wrong. There is nothing to falsify. You can count every umbrella on Earth and still not have produced a single scientific claim, because you have not said anything that the next observation could contradict.
This seems decisive, but Popper’s criterion has well-known problems. Whole areas of inquiry that most people consider obviously scientific, like taxonomy in biology, much of geology, and large parts of observational astronomy, are primarily descriptive. They catalog and classify rather than propose bold, falsifiable theories. The criticism that natural history is “mere stamp collecting” has a long, fraught history in the sciences, and it says more about rivalries between scientific communities than about the actual value of descriptive work.2Archives of Natural History. Natural history as stamp collecting: a brief history If falsifiability is the only line, a startling amount of what happens in university science departments gets pushed to the wrong side of it.
Kuhn and the Paradigm Approach
Thomas Kuhn offered a fundamentally different way of thinking about the problem. Rather than looking for a logical criterion that separates science from non-science, Kuhn argued that science is defined by a social and intellectual structure: the paradigm. A scientific community works within a shared framework of assumptions, methods, and exemplary problem-solutions. What Kuhn called “normal science” is essentially puzzle-solving, working out the details and implications of whatever paradigm is currently dominant.3Semina Scientiarum. Is normal science good science?
Under this view, the question about umbrellaology shifts. It is no longer “does this field make falsifiable predictions?” but rather “does this field have a paradigm?” Does the umbrella-studies community share a theoretical framework that tells them which problems matter, which methods are appropriate, and what counts as a solution? If the answer is no, if umbrella scholars just collect data without any shared explanatory goals, then by Kuhn’s lights the field is “pre-paradigmatic,” a stage that many future sciences pass through before they mature.
The implication is surprisingly generous to umbrellaology. Kuhn’s framework suggests that data collection is not inherently unscientific. It is what happens before a science crystallizes. If someone eventually proposed a compelling theory of umbrella use, one that organized the existing data, generated new research questions, and attracted a community of problem-solvers, umbrellaology could graduate into genuine science. The data itself was never the problem. The absence of a shared theoretical structure was.
Lakatos and the Progressive Research Programme
Imre Lakatos tried to thread the needle between Popper’s strict logical standard and Kuhn’s more sociological approach. He proposed that science should be evaluated not at the level of individual theories, but at the level of research programmes, ongoing sequences of theories that share a common core of assumptions. The key distinction for Lakatos was between “progressive” research programmes, which keep making novel predictions that turn out to be confirmed, and “degenerative” ones, which only add ad hoc patches to explain away anomalies without predicting anything new.4PubMed Central. Progressive and degenerative journals: on the growth and appraisal of knowledge in scholarly publishing
Applied to umbrellaology, Lakatos’s framework asks a dynamic question rather than a static one. It is not “is this science right now?” but “is this field going somewhere?” A discipline that keeps generating new, confirmed predictions about the world is progressive and counts as good science. A discipline that merely accumulates data, or that keeps revising its claims to accommodate whatever turns up next without ever getting ahead of the evidence, is degenerative. Umbrellaology, with its pure-collection ethos and zero predictive ambition, would land firmly in degenerative territory. But the framework leaves the door open: if the umbrella scholars started making and confirming predictions, their status could change.
Feyerabend’s Radical Skepticism About the Question Itself
Paul Feyerabend took the most extreme position in the debate. His view, sometimes called epistemological anarchism, holds that there are no universal methodological rules that define science. Every proposed criterion, falsifiability, paradigm structure, progressive problem-shifts, has historical counterexamples where great science violated it. Galileo used rhetoric and propaganda alongside evidence. Early atomic theory was unfalsifiable for decades. Feyerabend concluded that the only methodological principle that does not inhibit scientific progress is “anything goes.”5Problems of Modern Education (Problemy Sovremennogo Obrazovaniya). Rejection of the Method: Paul Feyerabend and Pluralism in Science
From Feyerabend’s perspective, asking whether umbrellaology is a science is asking the wrong question entirely. The demarcation problem is itself a mistake, an attempt to impose artificial boundaries on human inquiry that only serve to protect the prestige of established fields. If umbrella scholarship produces useful knowledge, interesting patterns, or new ways of seeing the world, then policing whether it deserves the label “science” is a power game, not a philosophical insight.
This position is genuinely held by some philosophers and historians of science, not just as a provocation but as a serious critique of how the word “science” functions in culture. The label carries enormous institutional and financial weight. Calling something science grants it funding, credibility, and authority. Calling it non-science denies all three. Feyerabend’s point is that these material consequences may be driving the philosophical arguments more than the other way around.
When Courts Have to Draw the Line
However frustrating the demarcation problem is for philosophers, it cannot stay abstract forever. Courts regularly have to decide what counts as reliable scientific testimony and what is junk. In the United States, the Daubert standard, established by a landmark Supreme Court decision, gives judges the role of “gatekeepers” who evaluate whether expert testimony meets basic scientific criteria before it reaches a jury.6PubMed Central. Black Robes and White Coats: Daubert Standard and Medical and Legal Considerations for Medical Expert Witnesses This standard replaced the older Frye rule, which simply asked whether a scientific method was “generally accepted” within its field.
The Daubert framework is interesting because it essentially codifies a rough-and-ready version of several philosophical criteria at once. Judges are asked to consider whether a theory or technique can be tested, whether it has been subjected to peer review, whether it has a known error rate, and whether it is generally accepted. These criteria pull from Popper (testability), from the broader scientific community standard (peer review and acceptance), and from practical reliability (error rates).7International Journal For Multidisciplinary Research. The Daubert Standard and Admissibility of Psychological Testimony: Scrutinizing Science in the Courtroom
Umbrellaology would almost certainly fail a Daubert challenge. It has no testable theories, no peer-reviewed predictions, and no error rates because it makes no claims that could be in error. But the legal standard also reveals something important about how demarcation works in practice. Courts do not need a philosophically perfect criterion. They need a workable one. The Daubert framework is a pragmatic compromise that filters out the worst pseudoscience without pretending to solve a problem that philosophers have debated for a century.
Description, Prediction, and the Question of Value
One of the deepest tensions in the demarcation debate is between description and explanation. Umbrellaology is purely descriptive. But some of the most respected branches of science have been too. Charles Darwin spent years meticulously cataloging barnacles. The Linnaean system of biological classification is fundamentally a scheme for organizing observations. Astronomy was primarily a record-keeping enterprise for millennia before Newton came along with a theory of gravity.
The dismissal of descriptive science as “stamp collecting” reflects a hierarchy within the sciences that favors theory and prediction over observation and classification. That hierarchy has real consequences. Fields perceived as merely descriptive tend to receive less funding and lower prestige, regardless of how carefully their data is collected or how useful it turns out to be.2Archives of Natural History. Natural history as stamp collecting: a brief history The competition is not just about intellectual merit; it is about resources and institutional survival.
At the same time, there is a real insight behind the concern. A field that collects data without ever attempting to explain patterns or make predictions is, in a meaningful sense, incomplete. Psychology, for example, has faced sustained criticism for building intricate causal theories of behavior while having surprisingly little ability to predict what people will actually do.8PubMed Central. Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning The mirror image of umbrellaology’s problem, too much explanation with too little prediction, turns out to be just as troubling. Good science seems to need both.
Mertonian Norms and the Social Side of Science
Robert Merton took a sociological approach to the demarcation problem. Rather than asking what logical structure makes something scientific, he asked what social norms characterize scientific communities. He identified a set of ideals, including universalism (claims are evaluated on their merit, not who made them), communism (knowledge is shared openly), disinterestedness (scientists are motivated by truth, not personal gain), and organized skepticism (claims are subjected to critical scrutiny). These are sometimes called the Mertonian norms, and researchers have studied whether and how much practicing scientists actually subscribe to them.9PubMed Central. Extending the Mertonian Norms: Scientists’ Subscription to Norms of Research
This approach is interesting when applied to umbrellaology. The fictional discipline could, in theory, meet every Mertonian norm. Its practitioners could share data openly, evaluate each other’s collections impartially, welcome criticism, and be motivated purely by curiosity about umbrellas. On sociological grounds, it could look like a model scientific community. And yet it would still feel like it is missing something, which tells you that social norms alone do not solve the demarcation problem. You can have all the right institutional furniture, journals, peer review, conferences, open data, and still not be doing science if you are not generating explanatory or predictive knowledge.
Big Data and the Modern Return of the Problem
The demarcation problem has found renewed urgency in the era of big data and machine learning. Modern technology allows researchers to collect and analyze enormous datasets, often finding patterns and correlations without any underlying theory about why those patterns exist. In some ways, this is umbrellaology with better tools. The data is vast, the patterns are real, but the explanatory framework is thin or absent.
The risks of this approach are concrete. As one analysis in a Royal Society journal noted, the fact that a large share of published research findings turn out to be false suggests that even peer-reviewed conclusions cannot be trusted without evidence of sound experimental design and careful statistical analysis. Purely data-driven approaches also face the problem of spurious correlations, patterns that look meaningful but reflect no actual causal relationship, and the problem of extrapolation, where past success in describing trends is no guarantee of future accuracy.10The Royal Society. Big data need big theory too
This is exactly the weakness that the umbrellaology thought experiment was designed to highlight, decades before anyone had heard of big data. Collecting enormous amounts of information, no matter how precisely, does not by itself constitute science. Data needs theory. Patterns need explanations. Correlations need mechanisms. Without those, you can build impressive databases, even make useful short-term predictions, but you have not produced scientific understanding in any deep sense. The umbrella collectors of the thought experiment just happened to be doing it with filing cabinets instead of server farms.
Why No Criterion Has Won
After more than a century of debate, no single demarcation criterion has gained universal acceptance among philosophers. Falsifiability is too strict, excluding legitimate descriptive science and early-stage inquiry. Paradigm structure is too sociological, letting well-organized communities of nonsense count as science. Progressive research programmes require historical hindsight, making them useless for real-time evaluation. Social norms can be met by non-scientific communities. Legal standards are pragmatic but philosophically shallow. And Feyerabend’s rejection of the whole enterprise, while intellectually honest, leaves us with no tools at all for telling science from pseudoscience.
Some philosophers have responded by arguing that demarcation is not a single-criterion problem. Instead of looking for one defining feature of science, they suggest that science is characterized by a cluster of features, testability, explanatory power, predictive success, internal consistency, peer scrutiny, openness to revision, and connection to empirical observation. No single feature is strictly necessary, and no single feature is sufficient. A discipline that has most of these features is more scientific; one that has few is less so. Umbrellaology has some (careful observation, community structure, openness to new data) but lacks others (testable theories, predictive power, explanatory ambition).
This cluster approach trades the satisfying crispness of a bright line for something messier but arguably more honest. In practice, most working scientists and most ordinary people already think about science this way, even if they have never read a word of philosophy. When someone asks whether astrology or homeopathy is a science, the answer is not usually “it fails Popper’s criterion.” The answer is more like “it doesn’t make testable predictions, it doesn’t revise its claims in light of evidence, its practitioners don’t subject their ideas to real scrutiny, and its track record of accuracy is terrible.” That is a cluster judgment, not a single-criterion test.
What Umbrellaology Gets Right
The thought experiment endures because it is more subtle than it first appears. It does not just ask “is data collection science?” It forces you to notice how many of the trappings of science, institutional structures, peer-reviewed journals, careful methodology, professional communities, can exist in the total absence of scientific understanding. This is not just a hypothetical concern. Entire subfields within otherwise respectable disciplines have been criticized for producing sophisticated methodology in the service of trivial or untestable questions. When a field has all the machinery of science but none of the intellectual engine, the umbrellaology analogy applies.
The thought experiment also highlights how much the word “science” does in everyday discourse. Calling something scientific is not just a neutral description; it is a claim to authority. It means: trust this, fund this, teach this in schools, take it seriously. The demarcation problem matters not because philosophers need a tidy definition, but because real decisions, about funding, education, policy, and courtroom testimony, depend on whether we can tell science from its imitators. Umbrellaology, with its absurd subject matter and impeccable methodology, is designed to make you uncomfortable precisely because it shows how hard that judgment can be.