Bad scientific research is everywhere. There are junk studies suggesting that the same foods cause and prevent cancer, Darwinian evolution is false and, most infamously, vaccines are linked to autism. Such low-quality studies are frequently retracted by the journals that publish them, but most research found in reputable science journals still turns out to be wrong. These studies are typically evaluated by independent scientists to make sure the results are valid, a process known as peer review, so we’re prompted to ask an obvious question: how do we end up with so much questionable, exaggerated and even fraudulent data?
Well, peer review–though a necessary part of the publishing process–isn’t always the powerful junk science filter it was designed to be. As cancer epidemiologist Geoffrey Kabat pointed out in a recent story for the Genes and Science:
Epidemiologists, statisticians, and other health researchers need to publish in order to advance in their careers. But, the public and journalists – the consumers of information about health – need to be aware of something that researchers know well – there is no paper that is so dreadful that it cannot be published somewhere.
There are several important reasons “dreadful” research gets published, Kabat says. Sometimes reviewers aren’t experts on the studies they’re assigned to evaluate. In other instances, the authors of a new study get to select who reviews their paper, encouraging them to pick scientists who will green light the research for publication. But these are just two examples of the biases that can cripple the peer-review process. As the authors of an August 2019 article explained:
Bias may relate not only to author characteristics such as geography, nationality, language, specialty, gender and affiliation or prestige but also reviewer characteristics, such as preferences for type of content (e.g., by topic), type of study (e.g., bias against observational work), bias for or against interdisciplinary research, confirmation bias (i.e., tendency to endorse work in line with one’s own beliefs) and publication bias (i.e., a well-documented trend for trials with negative results to not be published and, correspondingly, for trials with positive results to be published).
This isn’t just an academic issue. Bad research can put people’s lives at risk, so addressing problems with peer review is essential. On this episode of Biotech Facts and Fallacies, Kabat joins GLP editor Cameron English to offer an inside look at peer review and answer two important questions: Can peer review be fixed, and in the meantime, how can average people learn to spot bad science when it hits the headlines?
Podcast: Play in new window | Download
Subscribe: Apple Podcasts | Android | RSS
Geoffrey Kabat is an epidemiologist, the author of over 150 peer-reviewed scientific papers, and, most recently, of the book Getting Risk Right: Understanding the Science of Elusive Health Risks. Visit his website and follow him on Twitter @GeoKabat
Cameron J. English is the GLP’s special projects editor. He is a science writer and podcast host. Follow him on Twitter @camjenglish
