In non-mumbo-jumbo speak: Scientists who bust their asses and write a paper submit it to a journal, and that journal arranges reviewers. The paper authors don’t know who’s marking their hard work up in red ink, and the reviewers don’t know whose work they’re marking up. In theory.
It’s designed to keep everyone to focused on good science, and not gender, race, rivalry, and other forms of bias. But here’s a ticklish question: What if you deduce the identity of your reviewers?
Part of writing science news about a study is to bug the paper’s author(s). I generally trick them into thinking our chat will be about 10 minutes. But what I should really say in a blind query is this: “I don’t think I’ll need more than 10 minutes of your time. But that depends on how interesting the stuff you have to say is, and how well you say it. So, it could be more like an hour, or 2-3 minutes, if you get my gist.”
When I bug the author, I work through the bread-and-butter questions (how did you do it? what does it mean? what’s next?) and take a few interesting tangents along the way. One of the final questions I ask is: “Who would you recommend I talk to that’s familiar with this field, but wasn’t involved with your work?”
In about half of the stories on studies I’ve written, e.g. for Wired Science, the author has said something like, “Well, you should talk to X. X is very familiar with this field, and I think he/she reviewed my paper.” In every case, the author was correct. Their suggested source did review the paper.
In the other half of cases, it seems like I always find the reviewer (“Oh hello, Dave. Did you know I reviewed this paper?”) by closely looking at the references and digging up research on related subjects.
I’m certain I’m not the first to bring this issue up, and won’t be the last. But it’s on my mind, and I needed to dump it somewhere conducive to feedback.
So, do scientists knowing the identity of their reviewers, or vice-versa, matter in the scheme of things? Or is it a non-issue?
In either case, or the gray area in between, is it avoidable? Scientific fields are highly specialized, so it seems almost inevitable to me that one would review a paper and think, “Hey, this sounds like an extension of Y’s work from a year ago.”
I’d especially like to know what working scientists think, if any of you are indeed reading this.
Photo: Flickr/Sweet Pea Photography
Tags: peer review