What Does It Mean When Error Bars Overlap?

Overlapping error bars do not necessarily mean two values are statistically the same, and non-overlapping error bars do not always mean they are different. This is one of the most persistent misunderstandings in all of data visualization, and it trips up not just casual readers but working scientists and journal reviewers. The confusion runs deep because the answer depends entirely on what kind of error bars you are looking at, how much they overlap, and how the data were collected.

The Overlap Fallacy

When you see a bar chart or a line graph with little whiskers extending from each data point, the tempting move is to eyeball whether those whiskers touch or cross. If they overlap, you might assume the two groups are not meaningfully different. If they don’t overlap, you might assume they are. Both conclusions can be wrong. A comprehensive guide to statistical misinterpretations lays this out explicitly: two 95% confidence intervals can overlap substantially, yet a direct statistical test can still return a significant difference. The authors give a concrete example where two intervals overlap and the test of no difference still yields a p-value of 0.03.¹

The reverse also holds. If one group’s 95% confidence interval contains the other group’s point estimate (the central value), the test for a difference between the groups will typically return p greater than 0.05. So the degree of overlap matters, not just its presence or absence. Treating error bar overlap as a binary signal for statistical significance is a shortcut that regularly leads to wrong conclusions.²

Why the Shortcut Fails

The reason the overlap-equals-no-difference rule breaks down is mathematical but fairly intuitive once you see it. Each error bar reflects uncertainty about its own group’s average. When you compare two groups, the relevant question is not “how uncertain am I about each group separately?” but “how uncertain am I about the difference between them?” Those are two different questions, and they have different answers.

A study in the Journal of Insect Science worked out exactly how badly the shortcut performs. If you use standard 95% confidence intervals and treat overlap as your test for significance at the 0.05 level, you are actually running a far more conservative test than you think. The 95% intervals will overlap more than 99% of the time even when the null hypothesis is true, meaning you would almost never declare a significant difference using the overlap method. You would be missing real differences constantly.³

Put differently, the 95% confidence intervals are built to capture each group’s own mean, not to compare two means to each other. Using them for comparison is like using a ruler designed for one job to measure something completely different. It will sometimes give you the right answer by accident, but you cannot rely on it.

Not All Error Bars Are Created Equal

Part of the confusion comes from the fact that the little whiskers on a graph could represent at least three different things, and many graphs do not clearly state which one they are showing. A paper focused on error bars in biomedical research explains the key distinction between two of the most common types: the standard deviation describes the spread of the underlying data, while the standard error of the mean describes how precisely the study estimated the average.⁴

Here is why the distinction matters for overlap:

• Standard deviation (SD): These bars tell you about the variability in the data itself. A wide SD bar means individual measurements are scattered far from the average. Two groups can have wildly overlapping SD bars and still have means that are clearly different, because SD bars are not designed to tell you about the precision of the group mean.
• Standard error of the mean (SEM): These bars are narrower than SD bars and shrink as sample size grows. They reflect how confident you are in the group average. Overlap of SEM bars is somewhat more informative for comparing groups, but still not a reliable substitute for a proper test.
• Confidence intervals (CI): Usually shown at the 95% level, these are the bars where people most often attempt the overlap test. As already noted, that test is overly conservative and will miss many real differences.

A set of widely cited rules for error bars in experimental biology makes the point that these different bar types give “quite different information” and that figure legends must make clear which type is shown.⁵ If a graph does not label its error bars, you essentially cannot interpret them at all. This is more common than it should be, and it is a red flag for any graph you encounter in the wild.

A Rough Rule That Actually Works

If you must make a quick judgment from a figure, there is a rule of thumb that performs far better than the crude overlap test. A paper in Statistics in Medicine found that when two independent confidence intervals overlap by about half the length of one arm, the difference between the two groups is approximately significant at p = 0.05. If the intervals just barely touch end to end without overlapping, p is roughly 0.01, meaning the difference is quite strong.⁶

This half-arm rule gives you a much better intuition than the binary overlap question. Instead of asking “do they touch?” you ask “how much do they overlap relative to their size?” A tiny sliver of overlap with long bars means something very different from intervals that overlap almost entirely. The half-arm benchmark is approximate, and it assumes the intervals are independent and constructed with the same assumptions, but it beats the naive approach by a wide margin.

There is also a more precise trick used in ecology and some other fields. Researchers have shown that if you construct 84% confidence intervals instead of 95% intervals, their overlap or non-overlap mimics a 0.05 significance test surprisingly well, for both symmetric and asymmetric intervals.⁷ You will rarely see 84% CIs in practice because the convention is so firmly locked into the 95% level, but if you are generating your own plots and want overlap to be visually informative, this is a legitimate approach.

Within-Subjects Designs Make Everything Harder

Everything discussed so far assumes you are comparing two separate, independent groups. But many experiments use a within-subjects design, where the same people or samples are measured under multiple conditions. Think of a study that measures your reaction time before and after drinking coffee. Because the same individuals appear in both conditions, their measurements are correlated, and standard error bars built from each condition separately do not capture that correlation.

This creates a particularly nasty problem. The error bars on the graph might look wide and overlapping, suggesting no difference, when in fact the within-person changes are highly consistent and the statistical test finds a clear effect. A paper in Advances in Methods and Practices in Psychological Science highlights that standard confidence intervals do not allow valid visual comparison between repeated measures, between groups, or in several other common designs including clustered sampling.⁸ The mismatch between what the plot shows and what the statistical test concludes can be dramatic.

Researchers have developed adjusted error bars for within-subjects designs that account for the correlation between conditions. The general approach involves removing between-subject variability first and then computing the error bars from the remaining within-subject variation, sometimes with an additional correction factor.⁹ When these adjusted bars are used, non-overlapping intervals for individual means do correspond to a confidence interval for their difference that excludes zero.¹⁰ But standard software does not always produce these adjusted bars by default, and many published figures simply use the uncorrected version without warning you. If a graph shows error bars from a repeated-measures study and does not specifically say the bars have been adjusted for the within-subjects design, the overlap is essentially uninterpretable.

How Error Bars Trick Your Brain

Beyond the mathematical issues, there is a perceptual problem. The way error bars are drawn on a chart genuinely changes how people reason about the data, even when they know the rules. A series of crowd-sourced experiments found that the visual encoding of uncertainty significantly affects the decisions people make. Participants looking at standard bar charts with error bars made judgments about group differences that were less aligned with what the statistics actually showed, compared to when the same data were presented in other formats.¹¹

This is not just a problem for untrained readers. Scientists with statistical training also show biased reasoning when looking at error bars, particularly when the bars are symmetric and attached to bar charts. The clean, crisp boundary of an error bar invites a binary interpretation: inside versus outside, overlapping versus not. But the actual uncertainty is graded. The edge of the bar is not a cliff; it is the point where the probability density happens to hit a particular threshold.

The researchers who ran these experiments suggested that gradient plots, which use transparency to encode uncertainty, and violin plots, which use width to show the data’s distribution, led to more accurate reasoning about group differences.¹¹ In these formats, you can see that uncertainty fades out gradually rather than stopping at a hard line, which better matches the underlying reality.

Better Ways to Show Uncertainty

The problems with error bars have pushed many fields toward richer visualizations. A popular alternative is the raincloud plot, which combines a dot plot of the raw data, a smoothed density curve showing the distribution shape, and a boxplot or confidence interval for summary statistics. A paper introducing these plots argued that traditional bar charts with error bars distort effect sizes, hide patterns in the raw data, and obscure the assumptions behind common statistical tests.¹²

Showing the raw data points alongside the summary statistics gives the reader something error bars alone cannot: a sense of whether the data are normally distributed, whether there are outliers pulling the mean around, and whether the groups actually overlap in their individual values. Two groups can have non-overlapping error bars for their means while the individual data points blend together almost completely. Seeing that would change your interpretation, but a standard bar chart hides it.

If you are making your own graphs, the simplest improvement over plain error bars is to plot the individual data points behind them. Even this small change makes the graph far more informative and harder to misread. If your software supports it, showing the full distribution with a violin or density plot is even better.

What to Do When You Are Reading a Graph

When you encounter error bars in a journal article, a news graphic, or a presentation, there are a few practical questions worth running through before you draw any conclusions:

• What type of bar? Check the figure legend. If it says SD, you are looking at data spread, not precision of the mean. If it says SEM, the bars will be narrower but still not ideal for comparing groups. If it says 95% CI, the half-arm rule gives you a rough guide, but remember that overlapping CIs can still accompany a significant difference.
• Independent or repeated? If the same participants were measured in both conditions, standard error bars are not valid for visual comparison unless the figure explicitly states they have been adjusted for the within-subjects design.
• Is there a statistical test? If the authors report the result of a direct comparison, that test supersedes anything you can glean from the error bars. The bars are supplementary, not definitive.
• How many groups? When a graph compares three or more groups, the overlap problem multiplies. Two specific groups might differ significantly even if their error bars overlap, and the risk of misreading the figure goes up with every additional bar.

If the legend does not specify what the error bars represent, be skeptical. A review of published figures found that this omission is disturbingly common, and it makes the graph essentially decorative rather than informative.⁵

Frequentist Confidence Intervals Versus Bayesian Credible Intervals

Most error bars you will encounter are frequentist confidence intervals, which have a specific and somewhat counterintuitive interpretation. A 95% confidence interval does not mean there is a 95% probability that the true value falls within the bar. What it means is that if you repeated the experiment many times and constructed a new interval each time, about 95% of those intervals would contain the true value.¹³

Bayesian credible intervals, by contrast, do allow the more intuitive statement: given the observed data, there is a 95% probability the true value lies within this range.¹³ This distinction matters for overlap interpretation because it changes what the bars actually claim about the world. When you look at two overlapping frequentist CIs and think “there’s probably no difference,” you are applying something closer to a Bayesian interpretation to a frequentist object, which is one reason intuitions about overlap lead people astray.

In practice, for reasonably large samples and common problems, the two interval types often end up in similar places. But the conceptual gap matters when you are reasoning about what overlap means. With credible intervals, a statement like “these intervals overlap, so the values are probably not that different” is at least closer to a valid inference, even though it is still not a formal test. With frequentist confidence intervals, that reasoning rests on a misunderstanding of what the interval represents.

When Non-Overlap Is Informative

There is one scenario where the overlap rule actually works well, just in one direction. If two 95% confidence intervals fail to overlap entirely, you can conclude that the difference between the groups is statistically significant at the 0.05 level, assuming the intervals were constructed under the same standard assumptions.¹ This is a conservative but valid inference. The catch is that it only works in one direction: non-overlap implies significance, but overlap does not imply non-significance. It is an asymmetric rule, and remembering which direction it works in is half the battle.

For SEM bars, the same asymmetry applies but with a different threshold. Because SEM bars are narrower than CI bars, non-overlap of SEM bars implies a bigger gap between the means relative to their uncertainty. Some textbooks state that non-overlapping SEM bars correspond roughly to significance, but the exact relationship depends on whether the sample sizes are equal and the variances are similar. The confidence interval version of the rule is more robust.

The practical takeaway is simple but goes against instinct: you can use non-overlap to confirm a difference, but you cannot use overlap to rule one out. In a world where most people look at error bars specifically to decide whether two things are different, this means the bars fail you precisely in the situation where you most want them to work.