Assortative mixing for peer review

[this is an idea proposed by Adrian de Froment]

Let's say you are a great scientific reviewer. You respond in a timely fashion, provide detailed and insightful comments, and your judgments about which papers should be published tend to match the judgments of other reviewers and of journal editors.

Being a good reviewer is like wetting yourself in a dark suit - you get a warm feeling, but no one ever notices. When it comes to having your own papers reviewed, your good karma is worth nothing. You're just as likely to be assigned a slow, careless reviewer as everyone else, despite all your contributions to the community.

In evolutionary biology and network theory, the term 'assortative mixing' refers to a bias in favour of connections between network nodes with similar characteristics. In other words, good reviewers should be reviewed by other good reviewers. We should build karma directly into the system.

Note that I'm not in any way suggesting that papers written by good reviewers should be more likely to be published. But perhaps they should be more likely to be well-reviewed - that is, rapidly, fairly and carefully. Likewise, if you perennially submit your reviews late, you would be more likely to be reviewed by someone similarly tardy. What better cure for your anti-social behaviour than a dose of your own medicine?

I've used speed of response here as the dimension that determines who gets assigned whom as a reviewer, but the objective function determining what constitutes a 'good' reviewer could be anything.

If this assortative mixing of reviewers was adopted, it would reward good reviewers, punish bad reviewers, and almost certainly lead to an overall improvement in the standard of reviewing.





Belongs to these tags