Author Archives: jdjacobs

Should Invited Papers be Marked as Such?

There’s a lengthy discussion of editorial (mal)practice over at DailyNous. In the discussion, a sub-thread developed about invited versus unsolicited papers, and whether they should be marked as such.

For example, the journal I edit, Res Philosophica, recently had a call for papers on transformative experience. We noted in the call for papers that unsolicited papers that are accepted will be published alongside invited papers (and named the authors of the invited papers). All of the papers are refereed, but obviously the invited papers are not triple anonymous. For the invited papers, I default to double anonymous, but on occasion I have used single anonymous review. (For what it’s worth, I never serve as the referee.)

I thought putting the names of the invited papers in the call for papers was transparent, but some of the discussants over at Daily Nous suggested something further: Put a note in all published papers that indicates how the paper was reviewed.

I’m initially inclined to think this is a great idea. A title note, for example, might go something like this: “This paper was invited and double anonymous reviewed by two referees.” Or “This paper was unsolicited and was triple anonymous reviewed by three referees.”

Two questions for discussion:

1) Is this a good idea? I worry that I’m missing potential downsides or pitfalls.

2) If it is a good idea, what information would be good to include? Invited vs unsolicited; triple- vs double vs single-anonymous review; number of referees; date of submission; date of acceptance? Others?

Suggest a Topic: Who should Editors Invite as Referees?

Cable Cohoe writes from the Suggest a Topic page:

I want to ask about the particular case of refereeing papers dealing with one’s own work. A friend informed me about what seems to me to be a problematic case. A junior philosopher submitted an article that was critical of the views of a dominant figure in a small sub-branch of a larger field. The paper was returned with a rejection based on a referee’s report consisting of multiple pages of detailed line-by-line criticisms, both major and minor. From the style and content, it seemed clear that the referee was the person whose position was being criticized. The paper was then rejected based on the recommendation of the referee. When the paper was submitted to a different journal, one of the referee’s reports (the one that led to it being rejected) was virtually identical to the referee’s report from the first journal. Thus it seemed that in both cases the journal editors made the person whose views were being criticized (a person who, admittedly, is one of the leading experts in this small field) the gatekeeper. This seems objectionable to me, but I don’t know how common or accepted this refereeing practice is. Is it ever acceptable for editors to ask the target of the paper to serve as a referee? Should authors always decline to referee such articles or do circumstances sometimes make this acceptable?

Should editors ask someone to referee a paper whose views are criticized in the paper? It’s not really clear to me what the norm is among journal editors, so I’d be interested to hear from the other contributors.

For my part, I’ve made it a policy not to ask the person whose views are criticized to be a primary referee. I can see why it would seem like the thing to do: Who would know the view better? But the evidence we have concerning implicit bias seem to me to suggest that it would be better to find referees without so much at stake in the debate.

Still, in cases where the two referees disagree in their assessment in a way where the person whose view is criticize could help settle the disagreement, I have invited the author as a third referee. But I’ve done that only after receiving the initial reports and trying to come to a decision after reviewing them.

What are other practices in the discipline, and what should the right practice be?

(There is a second issue worth having a discussion about—should a journal use a referee who has already referees the paper for a different journal?—but I want to focus our discussion on Caleb’s question. So I’ll leave this other issue for another post.)

Preparing a Manuscript for Anonymous Review

A nuts and bolts type post about best practices:

The submission process for most journals includes the request that the author prepare their submission for anonymous review. This step ensures that referees, and editors in triple anonymous reviewed journals, can’t discover the identity of the author simply by reading the submission.

While some aspects of preparing a manuscript for blind review are universally known (e.g., delete your name from the first page), many are not. And for some aspects, there may be disagreement about best practices. So how should you prepare your manuscript, aside from the obvious step of removing your name from the paper?

Acknowledgements: Many authors include acknowledgements or thanks to those who helped them in their papers, perhaps as a first or last footnote. These should be entirely deleted, and replaced with something that indicates they were removed, like “Acknowledgments removed”. But in my view this should also include any time an individual is recognized as contributing to the paper throughout. You might want to thank a person at a particular point in the paper, rather than general acknowledgements. Those should be edited as well. Even such remarks as “Jane Smith pointed out to me in conversation . . .” should be edited, in my view, since such remarks may inadvertently cause the reader to discover your identity. (Some sub-disciplines are small, and now the referee knows that the author is not Jane Smith, for example.) Something as simple as “[Name removed] pointed out to me in conversation . . .” works well.

Document Properties: Word processing software often automatically, perhaps without even your knowledge, include your name, institution, email or other identifying marks in the meta-data of the file you submit. You can find this information in the document properties. For example, open a Microsoft Word document that you created on your computer, go to file>properties, and you may see this information. If it’s there, it’s easy to delete, and preparing your submission for anonymous review should include this step. (The way to delete these properties, and the ease with which it can be done, differs from software to software.) In spite of including specific instruction to authors to check for this information, probably half of the submissions to Res Philosophica include them. (We had to set up a process where the Editorial Manager checks for this, before sending the submission to the Editor.)

Self-citation: Often you want to cite your own prior work. Indeed, in research projects, you are often building on previous work, and so you need to cite your own work. But for anonymous review you have to anonymize those references. Standardly, this is done as I suggested above for acknowledgements. So instead of “As I argue in Jacobs 2011 . . .”, you would have “As I argue in [citation removed] . . .” Again, though, doing it this way, especially in a small sub-field, can often lead to the author being identified by the referee, since the referee knows, for example, that all the authors who are named in the submission’s bibliography are not the author of the submission. (I’ve even seen bibliographies that leave the author’s own entries in alphabetical order, but with the details deleted, so that it’s clear the author’s last name begins with, say, “M”.)

In light of this, it seems to me that there is a better way to handle self-citation: Edit the paper so as to not use first person in self-citation contexts. So instead of changing “As I argue in Jacobs 2011 . . .” to “As I argue in [citation removed] . . .”, you would change it to “As Jacobs (2011) argues . . .” This does require a bit more editing on the part of the author, but it does seem to avoid the worries that simply deleting the author’s name raises. It’s not perfect, but initially I’m inclined to think it’s the best way to handle self-citation.

What do you think? Is that the best way to handle self-citation? Are there other issues to think about when preparing a manuscript for blind review?

Inquiring with the editor

An anonymous philosopher on the job market, who has several papers under review at journals, writes (from our Suggest a Topic Page):

I am wondering about the etiquette of sending a followup email to the journal editor. Can I simply inform him/her that I am on the market and would really like to hear something on my submission soon because it might seriously influence my chances of getting a job? How long should I wait before sending such an email? 3 months? 6?

I’ll speak from my perspective and hope other editors will chime in. Res Philosophica has a What to Expect page on its website where we outline when authors should expect to hear from us. We send out a brief note after the initial desk review, aiming to do so within three weeks of the initial submission. And then we aim for an initial decision within eight weeks. So we encourage anyone who wishes to to contact the Editorial Manager if they haven’t heard from us after each of those points in the process (three weeks and eight weeks). (Because we use triple anonymous review, the author should not contact the Editor directly.) But I would also encourage authors to contact the Editorial Manager about any question they have during the process.

If the journal has not indicated by when it aims to reach an initial decision, it’s of course still acceptable to contact the Editor (or whomever is appropriate to contact). Some philosophers I know think it acceptable to contact the editor within six weeks, and others eight weeks. I don’t think I would contact earlier than six weeks, myself, absent further considerations. But given the special circumstances, I would think a brief, gentle inquiry would be completely acceptable at six weeks.

On editor’s instructions

Lewis Powell writes in from our Suggest a Topic page:

Sometimes the editors give an author fairly vague guidance on what changes to make in accord with referee reports, saying “take some account of the objections raised in the referee’s report” or the like.  How should an author interpret such advice?

I’ll certainly be interested to hear the range of responses, especially from other editors. To start us off, I’ll say just say a few things.

If there is something from the referee reports that I definitely want the author to address, I will say so in my note to the author. But there are times when either I myself wouldn’t require that an author address any one particular point by the referees (though of course the referees themselves might when making a recommendation to the editor), or the paper is not in my area and so while there may be really important points for the author to address, I’m not certain of it.

In both cases, I still want the author to take very seriously all the comments of the referees. (They are, after all, the experts on the topic of the paper from whom I’ve asked for recommendations.) Taking seriously the comments does not always require agreeing with them and making changes accordingly, however. Sometimes it involves explaining why you think the referee’s point is mistaken. But it is almost always a good idea, even for referee comments that seem to misunderstand your point, to use them to improve your paper. (At the very least you can now see how someone might misunderstand what you are trying to say, and so a brief clarification can be added to forestall such a misunderstanding.)

So I would interpret that advice from an editor as something like the following: “Please take seriously all the referees’ comments. Consider each of them fully, and then determine whether, in your considered view, making changes to your paper in light of them would make your paper better. If they would, please do. If they would not, please explain your thinking about it to me.”

On sharing referee reports: with other referees

An issue that came up in the previous thread, one which I’ve discussed several times with others, is whether the editor should share the report from referee 1 with referee 2. (There are other issues related to sharing reports with the author, but I’ll save those for another thread.)

I myself have never directly seen the report from the second referee when I referee a paper. But occasionally the author quotes from that report extensively in the reply to referees that I receive in a revise and resubmit. (Sherri Irvin reports in the previous thread that the Journal of Aesthetics and Art Criticism informs referees of the editor’s decision and includes both referee reports in that email.)

Beside simple curiosity, it seems the main benefit of sharing referee 1’s report with referee 2 is to help referee 2: it serves as a sort of calibration for referee 2 on her report. At any rate, that’s the benefit I’ve heard discussed. (There may also be the potential complication that, after reading a report from referee 2, referee 1 might change her mind, raising all sorts of issues about what an editor ought to do if that happens.)

It seems to me referees have a reasonable expectation that their reports are not being shared outside the editorial team and the author. But that’s simple to change: either inform them otherwise before they agree to referee, or ask their permission and share the report with the other referee only if permission is granted.

So what issues do we see with sharing referee reports with other referees for the same paper?

Informing Referees of Decisions

I recently had a Facebook discussion where the issue came up concerning whether editors should inform referees of the decision they made regarding a submission. Several philosophers I know said they really appreciate when they are told what an editor eventually decided, because they would like to know what is being published in their area, in general, and they might want to refer to the paper, in particular. It can also help a referee confirm their report, like jointly grading a paper. 

I can see at least two issues that complicate this from an editor’s perspective. First, a minor point: writing 200-500 notes to referees each year, especially if they are not simply form emails, is not nothing. Second: It would seem odd to me, in a case where I did not accept the recommendation of the referee, to inform her of that fact. “I’d like to let you know that I didn’t follow your advice” just doesn’t feel right. 

There are, of course, things I could say to soften the message. I could make it clear that I inform referees as a matter of policy, and that I value all recommendations, even those I do not follow in the end. And so on. But it’s not clear to me that any of that helps.

So I’d certainly be interested in hearing what you think. Should editors inform referees of their final decisions?