Range, IQR, and Absolute Deviations

Before reaching variance, it is worth exploring measures that are often simpler and sometimes more appropriate.

---

1. Range

The range is

[
R=x_{\max}-x_{\min}.
]

For

[
2,;3,;4,;5,;6
]

the range is

[
6-2=4.
]

Its greatest advantage is interpretability.

Its greatest weakness is equally obvious.

Only two observations determine it.

A dataset containing one million observations has its range determined entirely by its minimum and maximum.

---

2. Interquartile range

The interquartile range is

[
IQR=Q_3-Q_1.
]

It describes the width occupied by the middle half of the observations.

Because observations below (Q_1) and above (Q_3) do not directly affect the endpoints, it is much less sensitive to extremes than the range or standard deviation.

That makes the IQR especially useful for skewed distributions.

It is also the machinery behind the familiar boxplot.

---

3. Mean absolute deviation

Instead of squaring deviations, why not simply take their absolute values?

Around the mean:

\frac1n
\sum_{i=1}^{n}|x_i-\bar x|.
]

Or around the median:

\frac1n
\sum_{i=1}^{n}|x_i-\tilde x|.
]

These should not be confused with the median absolute deviation, which we will encounter later.

Absolute deviations have a useful property: extreme observations grow linearly rather than quadratically in influence.

Consider deviations of 2 and 20.

Under absolute loss:

[
2 \rightarrow 2,\qquad 20\rightarrow20.
]

Under squared loss:

[
2\rightarrow4,\qquad20\rightarrow400.
]

Squaring turns the second observation into a statistical megaphone.

---

4. Python comparison

import numpy as np
import pandas as pd

x = np.array([10, 11, 12, 13, 14, 15, 50])

mean = np.mean(x)
median = np.median(x)

results = {
    "Range": np.ptp(x),
    "IQR": np.percentile(x, 75)
           - np.percentile(x, 25),
    "Mean abs dev about mean":
        np.mean(np.abs(x - mean)),
    "Mean abs dev about median":
        np.mean(np.abs(x - median)),
    "SD": np.std(x, ddof=1)
}

print(pd.Series(results))

Now progressively increase the outlier.

outliers = np.arange(15, 101, 5)

rows = []

for o in outliers:
    z = np.array([10, 11, 12, 13, 14, 15, o])

    rows.append({
        "outlier": o,
        "range": np.ptp(z),
        "IQR": np.percentile(z, 75)
               - np.percentile(z, 25),
        "SD": np.std(z, ddof=1),
        "AAD": np.mean(
            np.abs(z - np.mean(z))
        )
    })

df = pd.DataFrame(rows)
print(df)

Plot the trajectories:

import matplotlib.pyplot as plt

plt.plot(df["outlier"], df["range"],
         label="Range")
plt.plot(df["outlier"], df["SD"],
         label="SD")
plt.plot(df["outlier"], df["IQR"],
         label="IQR")
plt.plot(df["outlier"], df["AAD"],
         label="Absolute deviation")

plt.xlabel("Extreme observation")
plt.ylabel("Dispersion")
plt.legend()
plt.show()

This plot is wonderfully revealing.

Range grows directly with the extreme observation.

SD grows rapidly.

Absolute deviation responds more gently.

IQR barely notices.

---

5. R version

x <- c(10, 11, 12, 13, 14, 15, 50)

aad_mean <- mean(abs(x - mean(x)))
aad_median <- mean(abs(x - median(x)))

c(
  Range = diff(range(x)),
  IQR = IQR(x),
  AAD_mean = aad_mean,
  AAD_median = aad_median,
  SD = sd(x)
)

Simulation:

outliers <- seq(15, 100, by = 5)

result <- data.frame(
  outlier = outliers,
  range = NA,
  IQR = NA,
  SD = NA,
  AAD = NA
)

for (i in seq_along(outliers)) {
  z <- c(10, 11, 12, 13, 14, 15,
         outliers[i])

  result$range[i] <- diff(range(z))
  result$IQR[i] <- IQR(z)
  result$SD[i] <- sd(z)
  result$AAD[i] <- mean(abs(z - mean(z)))
}

matplot(
  result$outlier,
  result[, c("range", "IQR", "SD", "AAD")],
  type = "l",
  lty = 1,
  xlab = "Extreme observation",
  ylab = "Dispersion"
)

legend(
  "topleft",
  legend = c("Range", "IQR", "SD", "AAD"),
  lty = 1,
  col = 1:4
)

---

6. Advantages and limitations

Measure	Strength	Main limitation
Range	Extremely intuitive	Determined by two observations
IQR	Robust and easy to interpret	Ignores much tail information
Mean absolute deviation	Same units and moderate tail sensitivity	Less algebraically convenient
SD	Rich mathematical theory	Highly sensitive to tails

A useful habit is to report more than one.

For skewed or contamination-prone data, reporting

[
\text{median + IQR}
]

may be far more informative than

[
\text{mean + SD}.
]

What Does "Spread" Actually Mean?

 

1. Location is only half the story

Suppose two laboratories measure the same quantity.

Laboratory A reports:

[
10,;10,;10,;10,;10
]

Laboratory B reports:

[
2,;6,;10,;14,;18
]

Both have mean 10.

Yet describing them as statistically equivalent would clearly be absurd.

A location statistic tells us where the distribution sits.

A dispersion statistic tells us how broadly the observations occupy the space around that location.

This distinction appears everywhere:

• biology: variability in gene expression,
• ecology: variability in species abundance,
• manufacturing: process consistency,
• finance: volatility,
• medicine: heterogeneity of patient responses,
• genomics: read depth variability,
• machine learning: variability of representations or prediction uncertainty.

Sometimes variability is noise.

Sometimes variability is the biological phenomenon.

That distinction matters enormously.

---

2. Four different ideas of dispersion

Consider:

[
x=(1,2,3,4,20).
]

There are several reasonable ways to describe its spread.

Extreme separation

[
\text{Range}=\max(x)-\min(x)
]

This asks:

How far apart are the most extreme observations?

Central spread

[
IQR=Q_{0.75}-Q_{0.25}
]

This asks:

How wide is the middle 50%?

Typical deviation from a center

For example,

[
\frac{1}{n}\sum |x_i-\bar{x}|
]

or

[
\operatorname{median}|x_i-\operatorname{median}(x)|.
]

These ask:

How far does a typical observation lie from some center?

Pairwise spread

We can instead ask how far apart observations are from one another:

[
\frac{1}{\binom n2}
\sum_{i<j}|x_i-x_j|.
]

This idea leads to Gini's mean difference.

None of these questions is inherently more correct than the others.

They simply measure different geometries of variability.

---

3. A short historical detour

The modern vocabulary emerged gradually.

Karl Pearson introduced the term standard deviation in lectures in 1893 and used it in print in 1894, replacing older terminology such as "mean error" and "error of mean square."

R. A. Fisher introduced the statistical term variance in his famous 1918 work on the resemblance between relatives, where variation could be decomposed into meaningful components.

Corrado Gini introduced his mean-difference approach to variability in 1912, providing an alternative family of ideas based on absolute pairwise differences rather than squared deviations.

So even historically, variance was never the only road through the forest.

---

4. A first experiment in Python

import numpy as np

x = np.array([1, 2, 3, 4, 20])

mean = np.mean(x)
median = np.median(x)
data_range = np.ptp(x)
variance = np.var(x, ddof=1)
sd = np.std(x, ddof=1)
iqr = np.percentile(x, 75) - np.percentile(x, 25)
mad_raw = np.median(np.abs(x - median))

print("Mean:", mean)
print("Median:", median)
print("Range:", data_range)
print("Variance:", variance)
print("SD:", sd)
print("IQR:", iqr)
print("MAD:", mad_raw)

Now remove the extreme observation:

y = np.array([1, 2, 3, 4])

for name, z in [("with outlier", x),
                ("without outlier", y)]:
    print("\n", name)
    print("SD =", np.std(z, ddof=1))
    print("IQR =", np.percentile(z, 75) -
                   np.percentile(z, 25))
    print("MAD =", np.median(
        np.abs(z - np.median(z))
    ))

The measures react very differently.

That reaction is not a bug. It tells us what each measure cares about.

---

5. The same experiment in R

x <- c(1, 2, 3, 4, 20)

mean(x)
median(x)
diff(range(x))
var(x)
sd(x)
IQR(x)

mad_raw <- median(abs(x - median(x)))
mad_raw

Compare with:

y <- c(1, 2, 3, 4)

metrics <- function(x) {
  c(
    SD = sd(x),
    IQR = IQR(x),
    MAD_raw = median(abs(x - median(x)))
  )
}

metrics(x)
metrics(y)

---

6. Properties we should demand from dispersion measures

A useful dispersion measure might possess several properties.

Non-negativity

[
D(X)\geq0.
]

Zero for constant data

If every observation is identical,

[
D(X)=0.
]

Translation invariance

Adding a constant should usually not change spread:

[
D(X+c)=D(X).
]

Variance, SD, IQR and MAD all satisfy this.

Scale equivariance

Multiplying the data by (a) should change a scale measure proportionally:

[
D(aX)=|a|D(X).
]

SD and MAD satisfy this.

Variance instead satisfies:

[
\operatorname{Var}(aX)=a^2\operatorname{Var}(X).
]

Robustness

How much can one pathological observation alter the answer?

This turns out to be one of the central questions in the entire series.

---

7. A crucial lesson

There is no universally best dispersion measure.

Choosing one involves deciding what kind of variation deserves influence.

Variance says:

Large deviations deserve disproportionately large influence.

MAD says:

The behavior of the majority matters more than extreme observations.

Range says:

I care specifically about the extremes.

IQR says:

I care about the central half.

Gini mean difference says:

I care about distances among all pairs.

Those are scientific choices disguised as formulas.

And that is why dispersion deserves more thought than simply typing sd(x).

What Should Science Learn From the Career Effects of Retractions?

Retractions are necessary.

That should be the starting point.

Scientific knowledge is valuable partly because science contains mechanisms for identifying and correcting unreliable claims. A literature in which papers can never be withdrawn would not be more trustworthy. It would be less trustworthy.

But the Nature Human Behaviour study shows that retraction systems do more than modify the literature.

They affect people.

The study's findings can be summarized as a sequence.

Retraction is associated with earlier departure from scientific publishing.

The effect appears particularly concerning for researchers with less-established careers.

Greater public attention surrounding a retraction is associated with a wider attrition gap.

Among researchers who remain, collaboration networks often grow rather than shrink.

Yet those networks change in composition, with important differences in collaborator seniority, productivity and impact.

What should institutions do with this information?

First, distinguish correction from culpability.

A retraction tells us something went wrong with a publication. It does not necessarily tell us that every author committed misconduct.

Second, make retraction notices more informative.

Readers should be able to distinguish honest error, plagiarism, fabrication, methodological failure and author-initiated correction whenever the evidence allows such distinctions.

Third, pay particular attention to junior researchers.

Because early-career scientists possess less accumulated reputational capital, institutions and mentors may need procedures ensuring that involvement in a retracted paper is evaluated according to actual contribution and responsibility.

Fourth, rethink how self-correction is rewarded.

If scientists believe voluntarily retracting erroneous work will permanently damage their careers, the system creates incentives to defend questionable results rather than correct them.

The authors themselves identify self-retraction, scientific-community support and changes in collaboration strategies as important mechanisms that future studies should examine.

Fifth, study the role of publicity.

A correction that receives almost no public attention and one that becomes an international scandal may have radically different career consequences. Future work needs to distinguish attention from condemnation and scientific discussion from personal exposure.

Finally, research integrity should be evaluated as a system rather than as a collection of individual retraction events.

The ideal system has to accomplish two things simultaneously:

correct science aggressively and assign responsibility accurately.

Those goals are not in conflict.

Indeed, both are necessary for a culture in which researchers are willing to acknowledge mistakes while deliberate misconduct remains consequential.

The deeper message of this study is therefore not that retractions are too harsh or too lenient.

It is that retraction is a much more powerful institutional intervention than simply placing a warning label on a PDF.

A retraction changes the scientific record.

It changes how researchers see one another.

It can reshape collaboration networks.

And, for some scientists, it can mark the point at which a publishing career ends.

Understanding those consequences is essential if science wants its mechanisms of self-correction to be both rigorous and fair.

Beyond the Standard Deviation

 

A Practical Series on Measuring Variability, Spread, and Statistical Dispersion

Most introductory statistics courses teach a familiar sequence:

mean → variance → standard deviation.

That sequence is useful, but it can accidentally suggest that once we know the standard deviation, the problem of measuring variability has been solved.

It has not.

There are many legitimate meanings of "spread":

• How far apart are the extremes?
• How wide is the central half of the data?
• How far is a typical observation from the center?
• How different are two randomly selected observations?
• How variable is the quantity relative to its magnitude?
• How much of the spread is caused by rare observations?
• How dispersed is a multidimensional cloud?
• What does dispersion even mean for angles, compositions, probability distributions, networks, images, or embeddings?

Different measures answer different questions.

This series explores those questions mathematically, historically, computationally, and practically.

The posts are:

1. What Does "Spread" Actually Mean?
2. Range, IQR, and Absolute Deviations
3. Variance and Standard Deviation: Why Squaring Won
4. Relative Dispersion: CV, Fano Factor, and Scale-Free Measures
5. Robust Dispersion: MAD, Qn, Sn, and Gini Mean Difference
6. Comparing Dispersion Between Groups
7. Multivariate and High-Dimensional Dispersion
8. When Ordinary Variance Stops Making Sense
9. Where Dispersion Research Could Go Next

Are Retraction Systems Fair to Co-Authors?

A paper may have one author.

It may also have fifty.

Yet when that paper is retracted, every author becomes permanently associated with the word “retracted.”

This creates a difficult fairness problem.

Authorship is collective.

Responsibility is often not.

Consider a hypothetical paper containing fabricated microscopy images.

The researcher who created those images may be directly responsible.

Another researcher may have performed an unrelated computational analysis.

A junior student may have contributed samples.

A senior principal investigator may have supervised the project.

All appear on the same paper.

Should they experience the same reputational consequences?

The study cannot determine the precise culpability of every author, and the authors explicitly acknowledge this limitation. Researchers associated with retracted papers differ in their awareness of the problems that ultimately produced the retraction, and mentors or colleagues may respond differently depending on those circumstances. The paper calls for future research capable of distinguishing authors according to their involvement in the reasons behind retraction.

This is more than a methodological problem.

It is an institutional-design problem.

Retraction notices often function as both corrections to the scientific literature and reputational documents.

Those two roles should perhaps be separated more clearly.

A good correction notice could explain:

what part of the paper is unreliable;

why it is unreliable;

whether misconduct was established;

whether the retraction was initiated by authors or the journal;

which contributions were implicated;

and, where investigations have established it, which individuals bear responsibility.

There are obvious legal and procedural challenges. Journals cannot simply accuse individual authors without adequate evidence.

But ambiguity also has consequences.

When notices provide too little information, readers may infer collective culpability.

This is particularly concerning given the paper's finding that less-established researchers appear more vulnerable to career exit after retraction.

Research-integrity systems therefore face two simultaneous obligations.

First, protect the scientific record.

Second, avoid converting the correction of a publication into indiscriminate punishment of everyone associated with it.

Those goals are compatible.

In fact, greater specificity in retraction notices could strengthen both.

Clearer explanations would help readers understand why a result should no longer be trusted while allowing the scientific community to distinguish error, negligence and deliberate misconduct.

A mature research-integrity system should be capable of saying:

“This paper is unreliable.”

without automatically implying:

“Every scientist whose name appears on it is unreliable.”

That distinction may be one of the most important lessons to emerge from this research.

Can We Really Say Retractions Cause Scientists to Leave? A Look at the Methods

Studies of academic careers face a difficult causal problem.

Suppose researchers with retracted papers leave science earlier than other researchers.

Did the retraction cause their departure?

Not necessarily.

Perhaps researchers who experience retractions differ systematically from other researchers even before the retraction occurs.

The authors therefore did more than simply compare retracted and non-retracted scientists.

They constructed matched comparison groups.

For the attrition analysis, retracted researchers were matched to non-retracted researchers using characteristics including gender, affiliation rank, discipline, publications and collaborators, while also attempting to align their career trajectories before the retracted paper appeared.

A second matching experiment examined researchers who continued publishing. Here, matching incorporated academic age, institutional ranking, discipline and pre-retraction numbers of publications, citations and collaborators. The researchers then compared collaboration outcomes over the subsequent five years.

They also used a Cox proportional hazards model for career attrition, controlling for several characteristics that can change over time. The analysis again supported an association between retraction and earlier departure from publishing.

These are substantial methodological strengths.

But they do not transform an observational study into a randomized experiment.

The researchers acknowledge several limitations.

Only 2,348 of the 14,579 authors in the filtered sample could be suitably matched for the post-retraction analysis, and this matched group was younger and generally lower-status than the full sample.

The analysis also measures average effects across highly heterogeneous cases.

A junior scientist unknowingly associated with problematic data and a senior researcher responsible for deliberate fabrication are fundamentally different cases, even if both appear in a database under “retraction.”

The Altmetric analysis adds another limitation. Attention scores measure volume of attention but provide limited information about its content or tone.

This is why careful language matters.

The study provides strong evidence that retractions are associated with changes in publishing careers, and its matching and longitudinal analyses make simplistic alternative explanations less convincing.

But we should resist translating every association into an individual causal claim.

For science-policy research, methodological humility is not a weakness.

It is precisely what allows useful findings to remain credible.

The most compelling part of this paper is therefore not simply the headline number.

It is the attempt to ask the counterfactual question:

What might have happened to a similar scientist who did not experience the retraction?

That is the right question.

Even when observational data cannot answer it perfectly.

A Retraction Is Not Just a Correction. It Is a Signal About Reputation

Why can one withdrawn paper affect an entire scientific career?

The answer may lie in how reputation works.

Science operates under substantial uncertainty.

When choosing collaborators, hiring researchers or evaluating grant applications, nobody can directly observe every relevant quality of another scientist. We cannot inspect every experiment they have conducted, independently reproduce every result or directly measure attributes such as reliability, judgement and integrity.

Instead, science relies heavily on signals.

Publications are signals.

Citations are signals.

Institutional affiliations are signals.

Collaborators are signals.

Retractions are signals too.

The authors frame their study partly through this sociological perspective. Scientific credibility accumulates throughout a career, and a retraction creates a visible signal that the quality of an author's work has been challenged.

The importance of collaboration networks becomes clearer under this interpretation.

Who chooses to work with a scientist communicates information about that scientist.

If respected researchers continue collaborating with someone after a retraction, observers may interpret those relationships as signals of continuing professional trust.

If established collaborators disappear, that sends another signal.

This makes post-retraction network rebuilding particularly interesting.

The study finds that surviving researchers often form larger collaboration networks, potentially providing a mechanism through which reputation can be reconstructed. But these networks are qualitatively altered, particularly in the seniority and productivity of collaborators retained.

Reputation is therefore not simply an individual property.

It is relational.

Part of a scientist's professional standing resides in the people willing to associate their own reputations with that scientist through collaboration.

This also helps explain why early-career researchers may be especially vulnerable.

Established researchers possess numerous independent signals of quality.

A junior researcher has fewer.

Consequently, a single negative signal can occupy a much larger share of the information available to others.

The broader lesson reaches beyond retractions.

Academic careers are often described as collections of individual achievements: papers, grants, citations and awards.

But careers also exist inside networks of trust.

Retractions reveal those networks because they create an unusually visible shock.

What happens afterward tells us something fundamental about how science functions.

Researchers do not merely produce knowledge.

They continuously evaluate whom to trust enough to produce knowledge with.