Mon Jan 27 13:52:22 EST 2014

Crisis in Science

There are currently problems in the way science is being done and reported. Here are some references about the crisis in science that I've run across recently. The problems seem to be most prominent in the area of psychology.

  • How journals like Nature, Cell and Science are damaging science
    The incentives offered by top journals distort science, just as big bonuses distort banking.

    Just as Wall Street needs to break the hold of the bonus culture, which drives risk-taking that is rational for individuals but damaging to the financial system, so science must break the tyranny of the luxury journals. The result will be better research that better serves science and society.

  • Science has lost its way, at a big cost to humanity
    Researchers are rewarded for splashy findings, not for double-checking accuracy. So many scientists looking for cures to diseases have been building on ideas that aren't even true.

    The demand for sexy results, combined with indifferent follow-up, means that billions of dollars in worldwide resources devoted to finding and developing remedies for the diseases that afflict us all is being thrown down a rathole. NIH and the rest of the scientific community are just now waking up to the realization that science has lost its way, and it may take years to get back on the right path.

  • Weak statistical standards implicated in scientific irreproducibility
    One-quarter of studies that meet commonly used statistical cutoff may be false.

    The plague of non-reproducibility in science may be mostly due to scientists' use of weak statistical tests, as shown by an innovative method developed by statistician Valen Johnson, at Texas A&M University in College Station.

  • Psychologists strike a blow for reproducibility
    Thirty-six labs collaborate to check 13 earlier findings.

    A large international group set up to test the reliability of psychology experiments has successfully reproduced the results of 10 out of 13 past experiments. The consortium also found that two effects could not be reproduced.

  • Open Science Framework Reproducibility Project: Psychology

    Do normative scientific practices and incentive structures produce a biased body of research evidence? The Reproducibility Project is a crowdsourced empirical effort to estimate the reproducibility of a sample of studies from scientific literature. The project is a large-scale, open collaboration currently involving more than 150 scientists from around the world.

  • Science and Its Skeptics
    There is a crisis in replicability.

    Is John Horgan right in being struck by all the "breakthroughs" and "revolutions" that have failed to live up to their hype: string theory and other supposed "theories of everything," self-organized criticality and other theories of complexity, anti-angiogenesis drugs and other potential "cures" for cancer, drugs that can make depressed patients "better than well," "genes for" alcoholism, homosexuality, high IQ and schizophrenia.

Addendum 02/02/2014: Scientific Pride and Prejudice

... science might look for help to the humanities, and to literary criticism in particular.

A major root of the crisis is selective use of data. Scientists, eager to make striking new claims, focus only on evidence that supports their preconceptions. Psychologists call this "confirmation bias": We seek out information that confirms what we already believe. "We each begin probably with a little bias," as Jane Austen writes in "Persuasion," "and upon that bias build every circumstance in favor of it."


Posted by mjm | Permanent link | Comments
comments powered by Disqus