Open reviewing and a case study

Open reviewing

One of the criticisms of the peer review system has been a lack of transparency. Reviewers are usually anonymous - this is to allow the reviewer to feel able to give a negative review without the author knowing their name. Conversely, some journals prefer to blind the reviewer to the names of the authors - hence the review is blind to author and reviewer or double blind. However the evidence about the value of this is mixed. A nice review Open versus blind peer review: is anonymity better than transparency? concludes: 'Although double-blind peer review has advantages in the reduction of specific biases, open peer review has the advantage of transparency.' Self-awareness among reviewers of their own unconscious biases and any deficits in the methodological expertise required for a review will help improve the quality of peer review across the spectrum, enhancing the quality of published biomedical research.' Two studies have compared blinding of the names of authors with non-blinding as well as allowing reviewers to remain anonymous. The first, The effects of blinding on the quality of peer review. A randomized trial, reported: 'Blinding was successful for 73% of reviewers. Quality of reviews was higher for the blinded manuscripts (3.5 vs 3.1 on a 5-point scale). Forty-three percent of reviewers signed their reviews, and blinding did not affect the proportion who signed. There was no association between signing and quality.' A subsequent study, Effect of Blinding and Unmasking on the Quality of Peer Review: A Randomized Trial, came to a different conclusion: 'Blinding and unmasking made no editorially significant difference to review quality, reviewers' recommendations, or time taken to review. Other considerations should guide decisions as to the form of peer review adopted by a journal, and improvements in the quality of peer review should be sought via other means.' These studies were both before the advent of preprints and their open reviews, and there do not seem to have been any recent trials.

Publishing research as preprints and the process of open peer review - which we discuss later in this course, does not allow blinding of authors as the whole paper is published as open access. The many advantages of this, including transparency, speed and increased access to research knowledge outweigh any potential benefits from the blinding process.

Open reviewing has become adopted by a number of journals. F1000 research was one of the early journals to do this, and you can read about how they go about peer review here.

The paper Ten considerations for open peer review is published in F1000 Research. Experience has accumulated since that paper, and one of the key features is that the corresponding author of the paper is asked to suggest the names of potential reviewers according to pre-set criteria and checked by the editorial staff before requests are sent out. The author of this course has published in F1000 Research a few times - the latest paper required 71 names to be submitted of whom 23 were rejected by the journal - mostly due to having too few publications. There was no response from 27, and 18 unavailable. All to get 3 people who submitted a review. From this anecdote, reviewers may find it difficult to be open about their reviews! Nevertheless, the advantages of open reviewing are considerable. The author has the opportunity to respond to the comments by the reviewer and amend the submission, for the reviewer to take another look and maybe change their recommendation. All this is open and available for readers to see, including each of the various submissions over the whole review process.

Case study

This case study shows the peer review process for the paper whose citation is: Heller RF, Zurynski R, Barrett A et al. Open Online Courses in Public Health: experience from Peoples-uni [version 2; peer review: 3 approved]. F1000Research 2017, 6:170 (https://doi.org/10.12688/f1000research.10728.2). In the side panel of the paper you will see this picture:

This shows that the reviews are open - the reviewers names are published as are their reports and the authors responses. The journal requires that at least two reviews feel that the paper is acceptable for the paper to be indexed (the paper is already published in its original state in this form of open publishing, but indexation means that it has met appropriate peer review quality standards and can be found in various journal indexation systems such as PubMed). On the first occasion, only one reviewer felt that the paper had met that criterion, and two still had some reservations. When the authors responded to the reviewers' comments and amended the paper, all three were happy that it met the quality standard.

You can access all the reports and responses and different versions of the paper. The course on Peer reviewing is deigned to help build skills in peer review, and explores the peer review process demonstrated in this paper in more detail. However, everyone is able to post your opinion in the forum below.

Boosting the value of preprint repositories by posting open reviews

Reprint repositories can have an added benefit by facilitating the publication of open reviews.

PREreview (who we met previously in the Peer reviewing course for access to the resources they offer) is a platform providing 'ways for feedback to preprints to be done openly, rapidly, constructively, and by a global community of peers.'

ASAPbio (Accelerating Science and Publication in biology), another contributor the field, 'is a scientist-driven nonprofit working to drive open and innovative communication in the life sciences. We promote the productive use of preprints for research dissemination and transparent peer review and feedback on all research outputs.' Their website makes an important point to emphasise the importance of this process: 'Science only progresses as quickly and efficiently as it is shared. But even with all of the technological capabilities available today, the process of publishing scientific work is taking longer than ever.'

We will come across the BAOBAB repository for preprints and other resoruces, and PublishNow which offers both a repository and an review process,later in this course. These are part of the development of a Coalition for an open access ecosystem for the publication of public health research in Africa (COPHHA). We will be using BAOBAB for the course exercises of posting a preprint and performing an open peer review.