Conference Peer Review Systems

I am becoming increasingly frustrated with the peer-review systems that are used by major conferences and journals.

Let's say that you write an article or a paper, and submit it to a technical conference or a journal. Copies of your manuscript will then go out to other researchers in the same field, who evaluate the work, decide whether or not it should be accepted, and suggest (or mandate) changes that should be made to the manuscript before it is published.

The idea is good, because only people who are doing research in a similar field of study are going to be qualified to fully understand what is contained in the manuscript and evaluate the quality of the research.

The problem is that by definition, other people in a similar field of study are your competitors. They are competing with you for funding from government agencies, publications, recognition in the field, etc.

Increasingly, what I have seen is that the comments coming back from reviewers are suggestions to cite certain papers or works. And although the identities of the reviewers aren't known, it appears that most reviewers are suggesting that authors cite works that they (the reviewers) have published in the past. Papers that are cited more often gain recognition in the field (along with their authors), so people are definitely motivated to try to get their works cited as much as possible.

It seems to me that the peer-review system is broken (or is, at least, close to being broken). Instead of receiving constructive comments, authors now seem to be receiving a lot more requirements to cite particular works. I'm not sure if I perceive the increase of this problem because I was much more naive a few years ago, or because the types of conferences and journals I have been submitting manuscripts has changed over the years.

But I'm not sure what the solution is. It is a hard sell to convince journals and conferences to police their reviewers, since 1) these organizations are already overworked as it is, and 2) reviewers are difficult to find.

Labels: