What is a Peer Review?

Peer review – the assessment of works by equal persons with expertise

You want to publish a paper in a professional journal and send your draft to the publisher. You want to give a talk at a conference and submit an abstract of the talk to the organisers. You want to apply for funding for a research project and deliver the concept for it. Or you want to send a customer a patch to fix his software problem. In such situations, peer review is often used.

Peer review is an approach to quality assurance. The object of quality assurance can vary: It can be, for example

• a discussion in a scientific journal,
• a publication in book form,
• an entry in Wikipedia,
• a lecture,
• a concept,
• a draft or
• even a piece of software.

What makes the review special is the emphasis on the reviewers: peers are colleagues who work in the same field or subject area. They are on par or the same level and sometimes the same age.¹ They can be professional or academic colleagues, researchers in the same field, friends with expertise, or classmates. It is important that the peer is in a position to expertly review the work.

Peer review scenarios and types

There are two main scenarios of peer reviews:

• An internal scenario where a colleague’s work receives a review from another colleague. This happens relatively often before the work product is passed on to third parties or external parties such as customers, suppliers, partners, etc.
• An external scenario where the work is reviewed by external parties, committees, associations, customers, suppliers etc.

For the external scenario, three different types of peer reviews can be distinguished:

• In the single blind review, the producers or submitters do not know who is reviewing the work, the article, the draft.
• The double blind review is based on the single blind review, whereby the reviewers also do not know from whom the work to be assessed originates. If it is a competition, for example, it makes sense to assess only the object, e.g. the design, the model or the prototype, because in this way the focus is on the work and not the creator.
• The triple blind review goes one step further than the double blind review, because here, for example, the editors of a magazine or journal do not know who wrote the article.

The three types differ according to the degree of anonymity. A special form is open peer review, which relies on transparency instead of anonymity:

• Producer and reviewer are known, and if necessary the reviewer is also mentioned in the course of publication.
• The review report is published anonymously or with the name of the reviewer.
• If applicable, the correspondence between the parties involved in the course of the review is also disclosed.

Post-publication peer review is also mentioned in some publications. This is a variant of peer review in which a review by the community – e.g. via a comment function on websites or at panel discussions – only takes place after a publication.

Peer review procedure

Numerous recommendations can be found on the Internet on how a peer review usually proceeds. In addition to the scenarios and types mentioned, it is also important to consider the object to be reviewed in relation to the requirements defined and named in advance. It makes a big difference whether a technical article is to be reviewed in a research journal, a crime novel published in a new book, a submission to an art competition or a workshop for a software development conference.

Here you will find an exemplary process of a peer review:

• Matching of formal criteria such as terms of reference, deadlines, scope, dimensions, theme, novelty value, focus on target audience, etc.
• Review of structure and logic, of methodology and procedure, of hypotheses and evidence.
• Review of materials used, formats or referenced sources.
• Assessing content, function, reproducibility or value.
• And the decision to accept, endorse or grant, to make required improvements or to reject the work.

Peer review criticism

It is often argued that a peer review

• causes a lot of effort,
• takes a lot of time,
• real anonymity cannot be guaranteed,
• the reviewers are not neutral,
• and the reviewers’ ignorance or preferences are essential for a go-or-no-go decision.

True. Nevertheless, it is a good instrument for quality assurance. And in case of doubt, there is little to be said against the use of a second or third reviewer beyond the effort and time involved.

Impulse to discuss:

Do you perceive the label “peer reviewed” as a sign of quality in the course of a publication?

Notes (Partly in German):

[1] dict.cc: Übersetzungen für Peer

Peer reviews are considered to be the most common procedure for checking and assuring the quality of publications in scientific journals.

Peers can be found in many companies. They provide mutual support and to some extent replace mentoring or reverse mentoring concepts.

In recent years, the term peer pressure has become established as a sign of team pressure, group pressure or social pressure.

And here you will find additional information from our Smartpedia section:

What is a code review?

How does a Sprint Review work?