Scientific news is often written as a sequence of reversals.
Coffee prevents disease.
Coffee causes disease.
A gene increases risk.
No, it doesn't.
A diet extends life.
Another study disagrees.
To anyone following science through headlines, the whole enterprise can begin to look like a weather vane in a thunderstorm.
The National Academies report offers a better way to think.
Confidence in scientific knowledge generally comes from multiple channels of evidence, not from one paper defeating another in intellectual combat. The report argues that the aim is to understand the overall inference from a body of studies rather than obsessively asking whether one study exactly replicated another.
This is where systematic reviews, meta-analysis, converging evidence and independent methods become powerful.
A single study can be wrong.
A replication can also be wrong.
But a network of evidence gathered using different populations, methods, laboratories and assumptions can gradually constrain the plausible explanations.
The report's discussion of public communication is especially important here.
Scientific journalism tends to favour surprising single studies and apparent contradictions over slow accumulation of evidence. It also notes that uncertainty present in scientific papers frequently disappears on the journey into news coverage.
Scientists are not innocent passengers in this process.
Press releases and researchers themselves can exaggerate implications.
The report therefore recommends caution both from scientists describing their results and journalists communicating them.
For readers, there is a remarkably useful rule:
Do not rebuild your worldview because of one paper.
And do not destroy an established body of scientific knowledge because one contrary paper appears.
The report explicitly advises against serious decisions based on one promising study, while also warning that one contrary study should not be treated as refuting conclusions supported by multiple lines of evidence.
Science is strongest when evidence accumulates.
A single study is a thread.
Knowledge is the fabric.
########################################################################
Further reading: This post is based on Reproducibility and Replicability in Science, a consensus study report of the National Academies of Sciences, Engineering, and Medicine. National Academies of Sciences, Engineering, and Medicine. 2019. Reproducibility and Replicability in Science. Washington, DC: The National Academies Press. DOI: 10.17226/25303.
No comments:
Post a Comment