A chi-square test result comes down to two core numbers: the chi-square statistic and a p-value. The statistic measures how far your observed data drifts from what you would expect if nothing interesting were going on, and the p-value translates that drift into a probability. A small p-value (typically below 0.05) suggests the pattern in your data is unlikely to be a fluke, while a large one means you cannot rule out chance. But reading those two numbers correctly requires understanding what sits behind them, and a surprising number of pitfalls can make a chi-square result misleading even when the math is done perfectly.
What the Chi-Square Statistic and P-Value Tell You
The chi-square statistic is a single number that summarizes the total discrepancy between what you observed in your data and what you would expect under the assumption that there is no relationship (or no difference from a theoretical distribution). A chi-square value of zero would mean your observed counts matched the expected counts exactly. The farther the statistic climbs above zero, the larger the gap between reality and the “nothing is happening” scenario.
The p-value converts that gap into a probability statement. Specifically, it answers this question: if the null hypothesis were true and there really were no effect or no association, how likely would you be to see a chi-square statistic at least this large? A p-value of 0.03, for instance, means there is roughly a 3 percent chance of getting a discrepancy this big from chance alone. Most fields treat p-values below 0.05 as the cutoff for calling a result “statistically significant,” though that threshold is a convention, not a law of nature.
An important subtlety: the p-value does not tell you the probability that your hypothesis is true. It tells you the probability of the data given the null hypothesis, which is a different thing. Mixing those up is one of the most common interpretation errors across all statistical tests, not just the chi-square.
Degrees of Freedom Set the Baseline
You cannot interpret a chi-square statistic without knowing its degrees of freedom. Two tests can produce the same chi-square value but lead to opposite conclusions if their degrees of freedom differ, because the shape of the chi-square distribution changes depending on how many categories are involved.
For a test of independence (the most common chi-square scenario, where you are checking whether two categorical variables are related), degrees of freedom equal the number of rows minus one, multiplied by the number of columns minus one. A simple two-by-two table has just one degree of freedom. A table with three row categories and four column categories has six. For a goodness-of-fit test, where you are checking whether a single variable’s distribution matches some expected pattern, degrees of freedom equal the number of categories minus one.
The practical consequence is straightforward: the more categories you have, the higher a chi-square value needs to be before it is considered significant. A chi-square of 6.0 with one degree of freedom has a p-value well below 0.05. The same chi-square of 6.0 with five degrees of freedom does not even come close. Software handles this automatically, but if you are reading someone else’s results, always check the degrees of freedom before judging the statistic.
Expected Frequencies and How They Work
The chi-square calculation depends on comparing observed frequencies to expected frequencies in every cell of your table. Expected frequencies represent what the data would look like if the null hypothesis were true. In a goodness-of-fit test, you supply the expected pattern yourself. A shop owner who believes customers arrive equally across five weekdays, for example, would set each day’s expected frequency to 20 percent of the weekly total.1Statology. How to Calculate Expected Frequency In a test of independence, the expected frequencies are calculated from the row and column totals of the observed data.
Each cell contributes to the overall chi-square statistic based on how much its observed count deviates from the expected count, scaled by the expected count. Cells where the observed number is much higher or lower than expected contribute the most. This matters for interpretation because a significant overall result does not tell you which specific cells are driving the discrepancy. If you have a large table, the significance could come from one dramatically off cell while the rest are close to expected, or it could come from many cells each being slightly off. Identifying which cells matter requires a separate step.
What Significance Actually Means Here
When a chi-square result crosses the significance threshold, the conclusion is that the observed pattern is unlikely to have arisen by chance. In a goodness-of-fit test, that means the distribution of your variable differs from the theoretical distribution you proposed. In a study of contact lens usage patterns, for instance, a significant chi-square indicated that the observed frequencies of different usage types deviated from what was expected, leading to rejection of the null hypothesis that usage was evenly distributed.2Medical hypothesis, discovery & innovation in optometry. Chi-square test applications In a test of independence, a significant result means the two variables appear to be associated rather than independent.
A non-significant result does not prove there is no association. It means the data you collected did not provide strong enough evidence to rule out chance. With a small sample, a real but modest association could easily produce a non-significant chi-square. This is why failing to reject the null hypothesis is not the same as confirming it.
Equally important, a significant chi-square tells you nothing about the strength or practical importance of the association. A huge sample can make a trivially small difference reach statistical significance. This is where effect size measures come in, and they deserve their own attention.
Effect Size Tells You How Strong the Association Is
The chi-square statistic grows with sample size, so a statistically significant result from a large dataset could reflect a relationship so weak it has no practical meaning. Effect size measures strip out the influence of sample size and give you a standardized sense of how strong the association is.
For two-by-two tables, the phi coefficient is the standard measure. For larger tables, Cramér’s V is the go-to. Both range from 0 (no association) to 1 (perfect association). Rough benchmarks treat values around 0.1 as small, 0.3 as medium, and 0.5 or above as large, though what counts as “meaningful” depends entirely on the field and context.
One quirk of Cramér’s V is that it tends to produce relatively low values even when the chi-square test is highly significant, particularly in tables with many categories.3PubMed Central. The chi-square test of independence This can be disorienting if you expect a highly significant p-value to come with a large effect size. The two are measuring different things. Always report both so readers can judge for themselves whether the association matters in practice.
Assumptions That Can Undermine Your Results
The chi-square test is built on several assumptions, and violating them can make your p-value unreliable. The most critical ones are:
- Independent observations: Each data point must come from a separate, unrelated unit. If the same person appears twice in your table, or if observations are clustered (students within classrooms, patients within hospitals), the standard chi-square is not appropriate.
- Adequate expected frequencies: The classic rule of thumb is that all expected cell frequencies should be at least five. When expected counts drop below that, the chi-square distribution no longer approximates the true sampling distribution well, and p-values become unreliable.
- Categorical data: The variables must be genuinely categorical, not continuous variables that have been artificially binned. Binning continuous data into categories for a chi-square test throws away information and can change your conclusions depending on where you place the bin boundaries.
Tables with a large number of categories create their own interpretation challenges. With 20 or more categories in either variable, the test becomes difficult to interpret meaningfully because the overall chi-square can be significant while the actual pattern is scattered across dozens of cells with no clear story.3PubMed Central. The chi-square test of independence
Small Samples and the Fisher’s Exact Alternative
When expected cell frequencies are small, the chi-square approximation starts to break down. The standard advice for decades has been to switch to Fisher’s exact test in those situations, and many software packages do this automatically for two-by-two tables when any expected count falls below five.4PubMed. The Choice Between Pearson’s χ(2) Test and Fisher’s Exact Test for 2 × 2 Tables
Fisher’s exact test does not rely on the chi-square distribution at all. Instead, it calculates the exact probability of observing your data (and all more extreme arrangements) under the null hypothesis. This makes it valid regardless of sample size. The trade-off is that it can be computationally intensive for larger tables, though modern software handles most practical cases easily.
The choice between Pearson’s chi-square and Fisher’s exact test is less clear-cut than textbooks sometimes suggest. The Pearson chi-square performs reasonably well even with some expected counts below five, as long as the sample is not tiny. Fisher’s test, meanwhile, can be conservative in some settings, meaning it may fail to detect real associations. For a standard two-by-two table with a moderate sample, both methods usually give similar answers.
The Yates’s Correction Debate
You may encounter a “Yates-corrected” chi-square, especially in older studies or when software applies it by default to two-by-two tables. Yates’s continuity correction was introduced to adjust for the fact that the chi-square distribution is continuous while the data is discrete. The correction shrinks the chi-square statistic slightly, making it harder to reach significance.
The correction has been controversial for decades. Research has shown that Yates-corrected statistics tend to be overly conservative, meaning they reject the null hypothesis less often than they should. The uncorrected Pearson chi-square generally provides adequate control over false positive rates without the correction.5PubMed. Yates’s correction for continuity and the analysis of 2 x 2 contingency tables Despite this, the correction remains a default in some software and is still routinely applied in medical research.
If you are interpreting results from a study that used Yates’s correction, keep in mind that the p-value is likely slightly higher (more conservative) than what an uncorrected test would produce. If the corrected result is significant, the uncorrected one certainly would be too. If the corrected result is borderline non-significant, it might be worth checking whether the uncorrected version crosses the threshold, though the broader lesson is that findings hanging on such fine distinctions are not particularly robust regardless.
What to Do After a Significant Result
A significant chi-square on a table larger than two-by-two is like an alarm going off somewhere in a building. You know something is happening, but you do not know which room. Post-hoc analysis is the process of figuring out which specific cells are driving the overall significance.
The most common approach is to examine standardized residuals for each cell. A standardized residual compares the observed count to the expected count in a way that accounts for the cell’s size. Cells with residuals above roughly 2 or below roughly negative 2 are usually flagged as contributing meaningfully to the significant result. Adjusted residuals go a step further by accounting for the row and column totals.
The challenge is that these residual-based approaches can disagree with each other. Different types of residuals (standardized versus adjusted) can lead to different conclusions about which cells are significant, and their p-values may be over- or under-estimated because the underlying approximations do not always control false positives well.6PubMed Central. Fisher’s exact approach for post hoc analysis of a chi-squared test A more rigorous alternative uses Fisher’s exact approach for post-hoc cell comparisons, which avoids these approximation issues. If you are working with a large table and the specific cell-level conclusions matter for your decision, it is worth using the exact method rather than relying on residuals alone.
You also need to consider the multiple comparisons problem. When you test many cells individually, the chance of at least one false positive rises. Adjustments like the Bonferroni correction, which divides your significance threshold by the number of comparisons, help control this. Without some correction, you risk chasing cells that look interesting purely by chance.
Large Samples Can Create Their Own Problems
The chi-square test is sensitive to sample size in both directions. Small samples produce unreliable p-values because the chi-square approximation breaks down. But large samples create a different problem: they make it easy to reach statistical significance for associations that are negligibly small in practical terms.7PubMed. Chi-Squared Test of Fit and Sample Size-A Comparison between a Random Sample Approach and a Chi-Square Value Adjustment Method
With thousands of observations, even a tiny departure from the expected pattern will produce a chi-square statistic large enough to be significant. A survey of 50,000 people might find a statistically significant relationship between hair color and coffee preference with a Cramér’s V of 0.02, meaning the association is real in a narrow statistical sense but explains almost none of the variation in the data. This is why interpreting chi-square results without an effect size measure is genuinely risky in large-sample studies. The p-value alone can be actively misleading.
Several approaches have been developed to handle the sample-size sensitivity of chi-square fit tests, including adjusting the chi-square value itself or using resampling methods.7PubMed. Chi-Squared Test of Fit and Sample Size-A Comparison between a Random Sample Approach and a Chi-Square Value Adjustment Method In practice, the simplest safeguard is to always pair your chi-square result with an effect size and to treat a significant p-value as necessary but not sufficient evidence of a meaningful association.
Confounding Variables and Simpson’s Paradox
A chi-square test examines the relationship between two categorical variables. It does not and cannot account for other variables that might be influencing both. This means a significant chi-square can reflect a genuine association, but it can also reflect a confounded one, where some third variable is creating the appearance of a relationship that does not actually exist (or hiding one that does).8PubMed Central. Misleading Epidemiological and Statistical Evidence in the Presence of Simpson’s Paradox: An Illustrative Study Using Simulated Scenarios of Observational Study Designs
The most dramatic version of this is Simpson’s paradox, where an association that holds in an overall table reverses direction when you split the data by a third variable. A treatment might look worse overall but better within every subgroup, simply because of how patients were distributed across groups. The chi-square test on the combined table would give you the wrong answer, and you would have no way of knowing from the test output alone.
If confounding is plausible in your data, you have several options. Stratified analysis, where you run separate chi-square tests within levels of the confounding variable, is the simplest. The Cochran-Mantel-Haenszel test extends this by providing a single summary statistic that controls for the stratifying variable. For more complex situations with multiple potential confounders, logistic regression is usually the better tool, since it can handle several confounders simultaneously while still working with categorical outcomes.
When Your Data Is Paired
The standard chi-square test assumes that observations are independent. When you have paired or matched data, like before-and-after measurements on the same individuals or matched case-control pairs, you need McNemar’s test instead. McNemar’s test is designed for two-by-two tables where the rows and columns represent the same binary variable measured under two conditions.9PubMed Central. Biostatistics Series Module 4: Comparing Groups – Categorical Variables
The key difference is what McNemar’s test focuses on. In a before-and-after design where people can be “yes” or “no” at each time point, the test ignores the people who stayed the same and focuses entirely on the “discordant” pairs, people who switched. If roughly equal numbers switched in each direction, the test is non-significant. If the switching is lopsided, it is significant. Running a standard chi-square on paired data ignores the pairing structure and can give misleading results, usually by underestimating the evidence for a difference.
Ordinal Data Deserves Special Treatment
Standard chi-square tests treat all categories as unordered. If your categories have a natural ranking (like pain rated as none, mild, moderate, or severe), the regular chi-square test ignores that ordering completely. It would treat the difference between “none” and “mild” the same as the difference between “none” and “severe.”
Tests designed for ordinal data, like the Cochran-Armitage trend test or the linear-by-linear association test, exploit the ordering to look specifically for a trend across categories. These tests are more powerful than the standard chi-square when a trend exists because they focus the analysis on the directional pattern rather than scattering it across all possible departures from independence. If you have ordinal variables and your research question is about whether there is a trend or dose-response pattern, an ordinal-specific test is the better choice. The standard chi-square will still work, but it throws away information, which means you need a larger sample to detect the same effect.
How to Report Chi-Square Results
If you need to write up chi-square results for an academic paper or report, the standard format includes the chi-square value, degrees of freedom, sample size, and p-value. For a test of independence, the reporting structure looks something like: “There was a significant relationship between treatment group and recovery status, χ²(1, N = 240) = 5.38, p = .021.” For a goodness-of-fit test, it follows a similar pattern but states what distribution was being tested.
A few formatting conventions worth knowing: the chi-square value is typically rounded to two decimal places, the p-value to three. Leading zeros are dropped from both in some style guides. Always include the degrees of freedom and sample size, since the chi-square value is uninterpretable without them. If you computed an effect size (and you should), report it alongside the chi-square. A Cramér’s V or phi coefficient gives readers the information they need to judge whether the statistically significant result is actually worth paying attention to in the real world.
When reporting non-significant results, state clearly that the test did not find a significant association, and still report the exact chi-square value and p-value rather than just saying “not significant.” Readers and reviewers need the actual numbers to assess how close or far the result was from the threshold, and to include your study in any future meta-analyses that combine data across multiple investigations.