Quality in Peer Review: A Conversation with Lauren Reynolds, Madelyn Ray, and Kurt Fraser

Interviewed by Rosalind S.E. Carney, DPhil

Peer Review Week has returned, and this year’s theme is Quality in Peer Review. Part of eNeuro’s mission is to provide fair and transparent reviews and to maintain the high-caliber peer review that both eNeuro and JNeurosci are known for.

In 2018, SfN launched its Reviewer Mentor Program to provide an opportunity for those in the community who would like guidance on how to review a paper. The trainees are matched with experienced reviewers from our journals’ reviewer pool and editorial boards. This mentorship provides one-on-one training on how to provide constructive feedback that strengthens the science.

In honor of Peer Review Week, I asked recent graduates of the program who have since reviewed for eNeuro about their experiences in the program, their experiences as reviewers and their thoughts on whether peer review improves scientific research. Below are the responses of the questions I asked the three graduates: Lauren Reynolds, Madelyn Ray, and Kurt Fraser.

 Lauren Reynolds, PhD - Postdoctoral fellow at Institut de Biologie Paris Seine and     Neuroscience Paris Seine at Sorbonne Université
 Follow Lauren at @laurenmac_r

 Madelyn Ray- PhD Candidate at Boston College
Follow Madelyn at @maddiehray

 
 Kurt Fraser- PhD Candidate at John Hopkins University
 Follow Kurt at @Kurt_Fraser

 RC: Did you have prior reviewing experience and what was the most constructive feedback that you learned from your mentor?

Lauren: “My PI had involved me in reviewing papers before I started the program, but this was my first formal training experience. My mentor was Professor Mary Kay Lobo, and she was fantastic, she provided very clear and constructive feedback on my work. My main takeaway from the program was how to balance the tone of the review.”

Madelyn: “I had both helped my PI with reviews and had reviewed a couple of papers independently. My mentor was Professor Barry Setlow, with whom I had a great experience in the program. We spoke a lot about the importance of not only what you write, but the manner in which you write the review.”

Kurt: “I think I had reviewed four or so papers before I joined the program. Although I had assisted my advisors with reviews, I was not an independent reviewer before the program. My mentor was Professor Gavan McNally, who gave very helpful, constructive feedback. In particular, he emphasized the structure and phrasing of the review. So, rather than specific feedback about the technical comments that I had made in my review, I learned more about the style of reviewer I would like to be.”

 Did completing the program change your perception of the quality of peer review that you offer?

Lauren: “I learned that even if you have serious criticisms of a paper, they can still be presented in a constructive way. I had previously received some reviewer comments about my papers that seemed very harsh—and I took those comments very personally. Quality in peer review, to me, is about finding the positive aspects of a paper, not just any holes. I learned not to be critical in a review just for the sake of it, but that my role is to help reinforce the story that is already there. Sound scientific studies should have a home in publication, and it is a matter of finding the right place for a paper.”

Madelyn: “I am a lot less intimidated by reviewing now, and the program gave me the skills and confidence to be critical yet constructive reviewer. The program drastically changed how I think about and write the review. For example, putting my comments in the context of what would make the paper stronger, instead of only stating that I believe certain aspects of the paper need to be changed for the paper to be accepted. This early in my career, I take the reviewing process very seriously, but there is also pressure, perhaps, to find flaws in a paper to prove yourself as a reviewer. But after the program, I realized that quality in peer review is about finding the strengths in a paper. In my current reviews, I encourage authors to elaborate on what I consider to be the most interesting findings, instead of just highlighting any weaknesses.”

Kurt: “As a reviewer now, I focus more heavily on what the paper conveys in terms of novelty and the overall logic of the paper. For example, breaking down what I feel are essential points that should be addressed versus other points that might be good to fix but by no means should have any weight on whether the paper is published or not. So, as opposed to focusing on what I could perceive as insufficient in a paper, I look at it in the context of its current state to determine what could be addressed to make it a good fit for the particular journal.”

How was your experience with the eNeuro consensus review consultation process?

Lauren: “I really liked the eNeuro consultation process. It was especially valuable because it was the first paper that I reviewed independently. The consultation helped me see what the other reviewer and the editor thought. I was anxious about submitting the review because I was concerned that my comments were either too harsh or not that important for the paper. But during the process, it became clear that my concerns were shared by the other reviewer and the Reviewing Editor. We had highlighted the same strengths and weaknesses of the paper. That was very reassuring for me and reinforced that I was contributing something of value as a reviewer. Overall, the consultation was a smooth and streamlined process, which I think is very helpful for authors. I have had to respond separately to basically the same comment from more than one reviewer in the past. Reviewing for eNeuro, it felt good that as reviewers, we had reached a consensus before the authors were contacted.”

Madelyn: “I really liked it; it was nice to be able to see the other reviewer’s comments. In particular, because the other reviewer is an established scientist, and our reviews were very similar. As a graduate student, that made me feel that my comments were valuable and that I did a good job—again boosting my confidence as a reviewer. I critiqued the paper, yet I was also constructive by finding its strengths.”

Kurt: “It was my first experience with a reviewer consultation process. I thought it was fantastic and I wish it were something that every journal would do. For a paper that I reviewed for eNeuro, I had a particular point which I felt needed to be addressed for the paper to be accepted. The other reviewer had a different viewpoint on another issue that I had not considered to be as critical. During the consultation process, the other reviewer, the Reviewing Editor, and I worked together to find common ground and come up with a compromise. I felt that my point of view was completely acknowledged and realized that there was another solution for the authors to address my concern. I think the consultation process puts more weight on the Reviewing Editor to sculpt the consensus review that will be sent to the authors, but I am in favor of this process.”

Do you believe that quality in peer review improves the quality of scientific research?

Lauren: “I do think so. I believe that peer review is a really important part of scientific publishing. To an extent, it is what keeps science honest, and improves the quality of research and publications. As scientists, we have a responsibility to treat our research ethically and to disseminate the results in the best way possible. I think peer review helps us do that in a way that is controlled—it provides checks and balances. Also, I think the recent move toward more open and transparent peer review practices is especially beneficial for the research community since it demystifies the process and creates an additional level of quality control over the review process itself.” 

Madelyn: “Yes. I think peer review is an essential part of science. Peer review helps identify not only strengths and weaknesses, but also is used to evaluate methodology, study design, and statistical analyses. Overall, I think that peer review helps shape where each field of neuroscience goes. And by increasing the quality of peer review, the quality of the science will also increase. Whether you are designing, running or writing up a study, knowing that quality peer review is a part of this process, forces to you think critically and weigh up the pros and cons of the choices you make. I believe that peer review is constructive and helps improve scientific research.”

Kurt: “Yes, in the majority of cases, I believe peer review is a necessary step in scientific research. I think that journals are important, and that peer review is very important. However, peer review can be a hinderance in the cases where a paper is stuck in review for a long time. But the addition of being able to post a preprint of a paper and journals welcoming submission of papers that are on preprint servers speeds up access to the main findings. I think that peer review offers diverse perspectives from other experts about our work. Peer review represents a collective critical evaluation in which reviewers and authors work together to improve the overall points that authors intended to make with a manuscript. I know that I have had the experience in which reviewers were surprised or excited about a particular point that, as authors, we underappreciated. In the day-to-day process, I would say that peer review serves to improve the overall output, readability, and the accessibility of papers, but it is absolutely necessary to catch the rare cases of scientific misconduct.”

 Did the Reviewer Mentor Program and experience reviewing for eNeuro change how you perform your own research?

Lauren: “I definitely believe that I think more critically about my work now. Not just about the way it is designed, but also how I present my data and conclusions. Reviewing papers is valuable expertise because it makes you aware that other people may have a different perception of your paper, so clarity in the presentation is essential.”

Madelyn: “Yes, I recently was talking with my PI about designing a study. Now, I feel that I put more thought and detail in the study design because I am thinking ahead to what reviewer comments I may receive. For example, should I use a between-subjects or within-subjects design? Ultimately, as scientists, we try to make the best call, designing experiments with the least confounds. Becoming an independent reviewer brings a good type of pressure to become a better scientist and design better studies.”

Kurt: “Yes, because you become your own reviewer. I think about what comments would be made by a reviewer of my next paper. Starting to review papers independently initiates a positive feedback loop—reviewing papers enables me to improve my own scientific thinking.”