Peer-review has a place

I usually find Glyn Moody‘s thoughts to be insightful, although we often disagree. His post on doing away with peer-review is definitely in the disagree category.

Anyone who reads these pages knows I am not a huge fan of the current peer-review system. IMO it is loaded in favor of incumbents, has tremendous inefficiencies and there is too much getting reviewed that doesn’t need to. However, peer review serves a purpose. As Pedro puts up in a great thread out on Friendfeed, on average peer-review adds value, even subjective opinion. Many of the commenters derive value from doing peer-review. Pedro also sees it as part of his duty as a scientist (I agree).

The problem is not with peer-review, but rather with the overall publishing ecosystem. We publish material in peer-reviewed journals that, today, can be published in a number of alternative avenues. We don’t have enough transparency and randomness in our peer review process. In the long term, to put this in terms of Glyn’s post, peer review should come in a system that deals with scarcity, as long as there is a well functioning system that deals with more abundance. To put this another way, with peer-review we should be focussing on a subset of what we publish today, while the rest goes into blogs, nature precedings, self-published articles, etc, as discussed in a previous post. The point that Glyn seems to miss is that the role of peer-review is not to just vote something up or down, but rather provide constructive, sometimes qualitative advice or suggestions, to help improve the science. I do believe that in the long run a more open peer review process might be the way to go, since we can find papers in our area of interest and provide the same suggestions via a different mechanism, but not yet. What we should rather focus on is changing the publish and reward system, which discourages scarcity, encourages “me too” and incremental publishing, and (IMO) discourages creativity.

Related articles by Zemanta

• Improving Peer Review: Let’s Provide an Ingredients List for Our Readers (scholarlykitchen.sspnet.org)
• Peer Review is No Panacea (theatlantic.com)
• Mark Midei and the failure of peer review (kevinmd.com)
• Elsevier Pilots PeerChoice in Chemical Physics Letters (prnewswire.com)