Over at Daily Nous there’s some excerpts of advice on refereeing papers. Much of the quoted advice is things not to do, and one caught my eye:
It is not your job to be copy editor…
…an apropos comment as earlier today I’d been collating my notes on a paper to turn it into a referee report, and the thought crossed my mind “refereeing would be a lot less tedious if I didn’t feel the need to NOT point out grammar and punctuation errors”.
And yet, despite the advice quoted above and my own personal inclination, I believe there are reasons why it is appropriate for referees to provide this sort of feedback (which is why I do it myself).
When I first graded undergraduate philosophy essays and marked them up for poor grammar, spelling, punctuation, etc., I often got objections along the lines of “Why are you grading my English, this isn’t an English class, you should just grade my arguments, not my language”. My response to this always was: It is not always possible to separate form from content. If your sentences are not grammatically constructed, this will often make your argument unclear, or even incomprehensible if it is not obvious how to rehabilitate the sentence into something grammatical. Misspelled words can introduce ambiguities. Misplaced commas can substantially alter the sense of a sentence. All of these play a role in the strength of the argument.
Proofreading your own writing is not only tedious, it’s actually very difficult: It is far easier to spot errors in someone else’s writing than it is to spot them in your own. I know that no matter how thorough I try to be, I will overlook duplicated words, missing words, incorrect tenses, subject-predicate match, etc., and thus I appreciate it greatly when someone else catches them for me. So, I proofread as part of my referee reports as a consequence of adopting a sort of Golden Rule of refereeing: I do unto others as I would have done unto me.
I’m in the enviable position of being a native speaker of English. It is often clear that the person whose paper I’m reading is not. I thus have a leg up on them when it comes to information crucial to producing a publishable paper. They may not have easy access to someone who has this information; it would not be kind of me to withhold it.
One of the purposes of a referee report is to provide the editor and the author(s) with answers to the questions of “Is this publishable as submitted?” and “Is this publishable after revisions?” For a paper which has grammar, spelling, and punctuation problems, the answer to the first question will be no, even if the arguments are all impeccable. If the answer to the second question is ‘yes’, it would be churlish of me not to identify to both the editor and the author the revisions I see necessary, and that includes the results of proofreading.
All of these reasons are cancellable, such that I wouldn’t go so far as to say that a referee should always copy-edit as they review. But as both an editor and a receiver of referee reports, I am always glad to see, and grateful to receive, detailed comments at the linguistic level.
Which is why as a referee, I should probably end this post and go back to the paper I was working on this morning, and finish typing up all my notes, copy-editing and otherwise!
Over at Reviewer 2 Must be Stopped, there are regularly posts saying “I’ve been waiting 6+ months for a result” which spark a slew of people saying that this wait time is inexcusable. My suspicion is that this is something which varies significantly by discipline; but I also wonder how many people who think such a wait time is inexcusable have heavy referee loads themselves that they turn around quickly. I put a poll up on the group to gather some information, but alas it doesn’t allow for anonymity, which makes me think I’ll get no responses in the “it takes me 6+ months to submit a report” option.
So I’ve created two polls which should be anonymous here, and even though this blog is aimed at people in philosophy, I still think the results should be of interest.
There’s a lengthy discussion of editorial (mal)practice over at DailyNous. In the discussion, a sub-thread developed about invited versus unsolicited papers, and whether they should be marked as such.
For example, the journal I edit, Res Philosophica, recently had a call for papers on transformative experience. We noted in the call for papers that unsolicited papers that are accepted will be published alongside invited papers (and named the authors of the invited papers). All of the papers are refereed, but obviously the invited papers are not triple anonymous. For the invited papers, I default to double anonymous, but on occasion I have used single anonymous review. (For what it’s worth, I never serve as the referee.)
I thought putting the names of the invited papers in the call for papers was transparent, but some of the discussants over at Daily Nous suggested something further: Put a note in all published papers that indicates how the paper was reviewed.
I’m initially inclined to think this is a great idea. A title note, for example, might go something like this: “This paper was invited and double anonymous reviewed by two referees.” Or “This paper was unsolicited and was triple anonymous reviewed by three referees.”
Two questions for discussion:
1) Is this a good idea? I worry that I’m missing potential downsides or pitfalls.
2) If it is a good idea, what information would be good to include? Invited vs unsolicited; triple- vs double vs single-anonymous review; number of referees; date of submission; date of acceptance? Others?
A post over at NewAPPS on the scarcity of reliable referees has sparked a comment thread containing a matter that I see come up in these context over and over again: People with solid publication records who are beyond the junior stage of their career who would be happy to referee but have never been asked to do so. What can these people do to raise their profiles, and thus the chances that they’ll be found and asked to referee? There are a few fairly easy things to do. When I am searching for referees, my selection is generally drawn from a combination of people I know personally are working in a particular area, people whose work is closely related to the submitted work on the basis of the bibliography, people I find via google, people writing on the same topic found via philpapers, and people who list that the relevant area as an area of research on academia.edu. So:
Make sure you have a webpage with an up to date CV and list of research interests, and current email. It’s amazing how many people (a) don’t have websites or (b) don’t have an easily findable current email on it.
Put your papers online, whether via academia.edu, your personal homepage, an institutional repository, etc.
Make sure your philpapers profile is up to date, again with current papers, current research interests, current email.
If you don’t have an academia.edu page, consider making one and populating it with your research interests.
Additionally, on the other side of the desk, I’ve noticed a strong correlation between submitting to a journal and being asked to referee for it within the next 6 months or so. So, another way to raise your profile as a potential referee for a journal is to submit to it!
When deciding where to submit a manuscript, authors are well-advised to read the “mission” or “focus” of the journal as well as recent articles. The Editorial Board, list of Associate Editors, and other affiliated journal staff provide another source of information for what sort of work the journal might be interested in publishing. Editorial Boards generally consist of well-established scholars who have been invited, elected, or appointed to the journal’s board by the editor or other members of the board.
Associate Editor Boards and Editorial Boards vary according to the governance structure of a journal. For instance, for some journals the Associate Editors constitute a policy-making body. They deliberate about decisions that affect the processes and future directions of the journal. For others, the Associate Editors function much like co-editors or area editors. They will receive the manuscript (from the editor, a managing editor, or an automated system) and assign appropriate referees.
The Editorial Board is often the first stop for an Editor in selecting referees for a manuscript. Editors may look to that board directly to provide the valuable service of evaluating papers. In addition, however, Editorial Board members are frequently consulted for advice for additional reviewers within a given area or field. Authors are well advised to consult that board list as a potential source of referees for their work.
In general, the Editorial Board reflects the general priorities of the journal.
I want to ask about the particular case of refereeing papers dealing with one’s own work. A friend informed me about what seems to me to be a problematic case. A junior philosopher submitted an article that was critical of the views of a dominant figure in a small sub-branch of a larger field. The paper was returned with a rejection based on a referee’s report consisting of multiple pages of detailed line-by-line criticisms, both major and minor. From the style and content, it seemed clear that the referee was the person whose position was being criticized. The paper was then rejected based on the recommendation of the referee. When the paper was submitted to a different journal, one of the referee’s reports (the one that led to it being rejected) was virtually identical to the referee’s report from the first journal. Thus it seemed that in both cases the journal editors made the person whose views were being criticized (a person who, admittedly, is one of the leading experts in this small field) the gatekeeper. This seems objectionable to me, but I don’t know how common or accepted this refereeing practice is. Is it ever acceptable for editors to ask the target of the paper to serve as a referee? Should authors always decline to referee such articles or do circumstances sometimes make this acceptable?
Should editors ask someone to referee a paper whose views are criticized in the paper? It’s not really clear to me what the norm is among journal editors, so I’d be interested to hear from the other contributors.
For my part, I’ve made it a policy not to ask the person whose views are criticized to be a primary referee. I can see why it would seem like the thing to do: Who would know the view better? But the evidence we have concerning implicit bias seem to me to suggest that it would be better to find referees without so much at stake in the debate.
Still, in cases where the two referees disagree in their assessment in a way where the person whose view is criticize could help settle the disagreement, I have invited the author as a third referee. But I’ve done that only after receiving the initial reports and trying to come to a decision after reviewing them.
What are other practices in the discipline, and what should the right practice be?
(There is a second issue worth having a discussion about—should a journal use a referee who has already referees the paper for a different journal?—but I want to focus our discussion on Caleb’s question. So I’ll leave this other issue for another post.)
It’s a routine, albeit vaguely frustrating, part of being an editor: You get a new submission, you look it over to get a sense of the content, you search for referees, you send out requests, and then wait for the declines to start trickling in. It’s worse than sending out wedding invitations.
But twice in the last two days I’ve had decline responses which have been a positive delight to receive: The referee suggested alternative people I could contact! Sometimes they’re ones I’ve already contacted, but sometimes it’s a new person I hadn’t considered.
Referees, consider doing the same, if you can. Extra bonus points if you can provide an email or webpage for the people you suggest.