Lunch at the Labs

Problems with Science

More on a topic I've blogged about before here and before that here.

• The 7 biggest problems facing science, according to 270 scientists
  1. Academia has a huge money problem
  2. Too many studies are poorly designed
  3. Replicating results is crucial -- and rare
  4. Peer review is broken
  5. Too much science is locked behind paywalls
  6. Science is poorly communicated
  7. Life as a young academic is incredibly stressful
  Conclusion: Science is not doomed

  "I think the one thing that would have the biggest impact is removing publication bias: judging papers by the quality of questions, quality of method, and soundness of analyses, but not on the results themselves," writes Michael Inzlicht, a University of Toronto psychology and neuroscience professor.

  Some journals are already embracing this sort of research. PLOS One, for example, makes a point of accepting negative studies (in which a scientist conducts a careful experiment and finds nothing) for publication, as does the aptly named Journal of Negative Results in Biomedicine.

• A Rant on Peer Review

  George J. Borjas

  I have a few pet peeves. One of them is how "peer review" is perceived by far too many people as the gold standard certification of scientific authority. Any academic who's been through the peer review process many times (as I have) knows that the process is full of potholes and is sometimes subverted by unethical behavior on the part of editors and reviewers.
  ...
  The point is that many human emotions, including nepotism, professional jealousies, methodological disagreements, and ideological biases go into the peer review process. It would be refreshing if we interpreted the "peer-reviewed" sign of approval as the flawed signal that it is, particularly in areas where there seems to be a larger narrative that must be served. The peer-review process may well be the worst way of advancing scientific knowledge--except for all the others.

• Why Most Clinical Research Is Not Useful

  John P. A. Ioannidis

  • Blue-sky research cannot be easily judged on the basis of practical impact, but clinical research is different and should be useful. It should make a difference for health and disease outcomes or should be undertaken with that as a realistic prospect.
  • Many of the features that make clinical research useful can be identified, including those relating to problem base, context placement, information gain, pragmatism, patient centeredness, value for money, feasibility, and transparency.
  • Many studies, even in the major general medical journals, do not satisfy these features, and very few studies satisfy most or all of them. Most clinical research therefore fails to be useful not because of its findings but because of its design.
  • The forces driving the production and dissemination of nonuseful clinical research are largely identifiable and modifiable.
  • Reform is needed. Altering our approach could easily produce more clinical research that is useful, at the same or even at a massively reduced cost.

• A Crisis at the Edge of Physics

  DO physicists need empirical evidence to confirm their theories?

  A few months ago in the journal Nature, two leading researchers, George Ellis and Joseph Silk, published a controversial piece called "Scientific Method: Defend the Integrity of Physics." They criticized a newfound willingness among some scientists to explicitly set aside the need for experimental confirmation of today's most ambitious cosmic theories -- so long as those theories are "sufficiently elegant and explanatory." Despite working at the cutting edge of knowledge, such scientists are, for Professors Ellis and Silk, "breaking with centuries of philosophical tradition of defining scientific knowledge as empirical."