By CARL ZIMMER, Published: June 25, 2011
ONE of the great strengths of science is that it can
fix its own mistakes. *There are many hypotheses in
science which are wrong,” the astrophysicist Carl
Sagan once said. *That’s perfectly all right: it’s the
aperture to finding out what’s right. Science is a
self-correcting process.*
If only it were that simple. Scientists can certainly
point with pride to many self-corrections, but science
is not like an iPhone; it does not instantly
auto-correct.
As a series of controversies over the past few months
have demonstrated, science fixes its mistakes more
slowly, more fitfully and with more difficulty than
Sagan’s words would suggest.
Science runs forward better than it does backward.
Why? One simple answer is that it takes a lot of time
to look back over other scientists’ work and replicate
their experiments. Scientists are busy people,
scrambling to get grants and tenure.
As a result, papers that attract harsh criticism may
nonetheless escape the careful scrutiny required if
they are to be refuted.
In May, for instance, the journal Science published
eight critiques of a controversial paper
(http://bit.ly/kTVizb ) that it had run in December.
In the paper, a team of scientists described a species
of bacteria that seemed to defy the known rules of
biology by using arsenic instead of phosphorus to build
its DNA.
Chemists and microbiologists roundly condemned the
paper; in the eight critiques (http://bit.ly/mg55ZU ),
researchers attacked the study for using sloppy
techniques and failing to rule out more plausible
alternatives.
But none of those critics had actually tried to
replicate the initial results. That would take months of
research: getting the bacteria from the original team
of scientists, rearing them, setting up the experiment,
gathering results and interpreting them.
Many scientists are leery of spending so much time on
what they consider a foregone conclusion, and
graduate students are reluctant because they want
their first experiments to make a big splash, not
confirm what everyone already suspects.
“I’ve got my own science to do,* John Helmann, a
microbiologist at Cornell and a critic of the Science
paper, told Nature (http://bit.ly/j2IDb0 ).
The most persistent critic, Rosie Redfield, a
microbiologist at the University of British Columbia,
announced this month on her blog
(http://bit.ly/jMVieG ) that she would try to replicate
the original results - but only the most basic ones,
and only for the sake of science’s public reputation.
“Scientifically I think trying to replicate the claimed
results is a waste of time,*
she wrote in an e-mail.
For now, the original paper has not been retracted;
the results still stand.
Even when scientists rerun an experiment, and even
when they find that the original result is flawed, they
still may have trouble getting their paper published.
The reason is surprisingly mundane: journal editors
typically prefer to publish groundbreaking new
research, not dutiful replications.
In March, for instance, Daryl Bem, a psychologist at
Cornell University, shocked his colleagues by
publishing a paper in a leading scientific journal, The
Journal of Personality and Social Psychology, in which
he presented the results of experiments showing, he
claimed, that people’s minds could be influenced by
events in the future, as if they were clairvoyant.
Three teams of scientists promptly tried to replicate
his results. All three teams failed. All three teams
wrote up their results and submitted them to The
Journal of Personality and Social Psychology. And all
three teams were rejected - but not because their
results were flawed.
As the journal’s editor, Eliot Smith, explained to The
Psychologist (http://bit.ly/keLfoW ), a British
publication, the journal has a longstanding policy of
not publishing replication studies.
“This policy is not new and is not unique to this
journal,*
he said.
As a result, the original study stands.
Even when follow-up studies manage to see the light
of day, they still don’t necessarily bring matters to a
close. Sometimes the original authors will declare the
follow-up studies to be flawed and refuse to retract
their paper.
Such a standoff is now taking place over a
controversial claim that chronic fatigue syndrome is
caused by a virus.
In October 2009, the virologist Judy Mikovits and
colleagues reported in Science (http://bit.ly/iVnRai )
that people with chronic fatigue syndrome had high
levels of a virus called XMRV. They suggested that
XMRV might be the cause of the disorder.
Several other teams have since tried - and failed - to
find XMRV in people with chronic fatigue syndrome.
As they’ve published their studies over the past year,
skepticism has grown. The editors of Science asked
the authors of the XMRV study to retract their paper.
But the scientists refused; Ms. Mikovits declared that
a retraction would be “premature.*
The editors have since published an “editorial
expression of concern.* (http://bit.ly/l7iULF )
Once again, the result still stands.
But perhaps not forever. Ian Lipkin, a virologist at
Columbia University who is renowned in scientific
circles for discovering new viruses behind mysterious
outbreaks, is also known for doing what he calls
“de-discovery*: intensely scrutinizing controversial
claims about diseases.
Last September, Mr. Lipkin laid out several tips
(http://bit.ly/lPAlV6 ) for effective de-discovery in the
journal Microbiology and Molecular Biology Reviews.
He recommended engaging other scientists - including
those who published the original findings - as well as
any relevant advocacy groups (like those for people
suffering from the disease in question).
Together, everyone must agree on a rigorous series of
steps for the experiment. Each laboratory then carries
out the same test, and then all the results are
gathered together.
At the request of the National Institutes of Health,
Mr. Lipkin is running just such a project with Ms.
Mikovits and other researchers to test the link
between viruses and chronic fatigue, based on a
large-scale study of 300 subjects. He expects results
by the end of this year.
This sort of study, however, is the exception rather
than the rule. If the scientific community put more
value on replication - by setting aside time, money
and journal space - science would do a better job of
living up to Carl Sagan’s words.
Carl Zimmer writes frequently for The New York Times
about science and is the author, most recently, of “A
Planet of Viruses.*
No comments:
Post a Comment