Author Archives: Sara L. Uckelman

Should referees proofread?

Over at Daily Nous there’s some excerpts of advice on refereeing papers. Much of the quoted advice is things not to do, and one caught my eye:

It is not your job to be copy editor…

…an apropos comment as earlier today I’d been collating my notes on a paper to turn it into a referee report, and the thought crossed my mind “refereeing would be a lot less tedious if I didn’t feel the need to NOT point out grammar and punctuation errors”.

And yet, despite the advice quoted above and my own personal inclination, I believe there are reasons why it is appropriate for referees to provide this sort of feedback (which is why I do it myself).

  • When I first graded undergraduate philosophy essays and marked them up for poor grammar, spelling, punctuation, etc., I often got objections along the lines of “Why are you grading my English, this isn’t an English class, you should just grade my arguments, not my language”. My response to this always was: It is not always possible to separate form from content. If your sentences are not grammatically constructed, this will often make your argument unclear, or even incomprehensible if it is not obvious how to rehabilitate the sentence into something grammatical. Misspelled words can introduce ambiguities. Misplaced commas can substantially alter the sense of a sentence. All of these play a role in the strength of the argument.
  • Proofreading your own writing is not only tedious, it’s actually very difficult: It is far easier to spot errors in someone else’s writing than it is to spot them in your own. I know that no matter how thorough I try to be, I will overlook duplicated words, missing words, incorrect tenses, subject-predicate match, etc., and thus I appreciate it greatly when someone else catches them for me. So, I proofread as part of my referee reports as a consequence of adopting a sort of Golden Rule of refereeing: I do unto others as I would have done unto me.
  • I’m in the enviable position of being a native speaker of English. It is often clear that the person whose paper I’m reading is not. I thus have a leg up on them when it comes to information crucial to producing a publishable paper. They may not have easy access to someone who has this information; it would not be kind of me to withhold it.
  • One of the purposes of a referee report is to provide the editor and the author(s) with answers to the questions of “Is this publishable as submitted?” and “Is this publishable after revisions?” For a paper which has grammar, spelling, and punctuation problems, the answer to the first question will be no, even if the arguments are all impeccable. If the answer to the second question is ‘yes’, it would be churlish of me not to identify to both the editor and the author the revisions I see necessary, and that includes the results of proofreading.

All of these reasons are cancellable, such that I wouldn’t go so far as to say that a referee should always copy-edit as they review. But as both an editor and a receiver of referee reports, I am always glad to see, and grateful to receive, detailed comments at the linguistic level.

Which is why as a referee, I should probably end this post and go back to the paper I was working on this morning, and finish typing up all my notes, copy-editing and otherwise!

Poll about referee practices

Over at Reviewer 2 Must be Stopped, there are regularly posts saying “I’ve been waiting 6+ months for a result” which spark a slew of people saying that this wait time is inexcusable. My suspicion is that this is something which varies significantly by discipline; but I also wonder how many people who think such a wait time is inexcusable have heavy referee loads themselves that they turn around quickly. I put a poll up on the group to gather some information, but alas it doesn’t allow for anonymity, which makes me think I’ll get no responses in the “it takes me 6+ months to submit a report” option.

So I’ve created two polls which should be anonymous here, and even though this blog is aimed at people in philosophy, I still think the results should be of interest.


Polls close in 1 week.

So you want to be a referee?

A post over at NewAPPS on the scarcity of reliable referees has sparked a comment thread containing a matter that I see come up in these context over and over again: People with solid publication records who are beyond the junior stage of their career who would be happy to referee but have never been asked to do so. What can these people do to raise their profiles, and thus the chances that they’ll be found and asked to referee? There are a few fairly easy things to do. When I am searching for referees, my selection is generally drawn from a combination of people I know personally are working in a particular area, people whose work is closely related to the submitted work on the basis of the bibliography, people I find via google, people writing on the same topic found via philpapers, and people who list that the relevant area as an area of research on academia.edu. So:

  • Make sure you have a webpage with an up to date CV and list of research interests, and current email. It’s amazing how many people (a) don’t have websites or (b) don’t have an easily findable current email on it.
  • Put your papers online, whether via academia.edu, your personal homepage, an institutional repository, etc.
  • Make sure your philpapers profile is up to date, again with current papers, current research interests, current email.
  • If you don’t have an academia.edu page, consider making one and populating it with your research interests.

Additionally, on the other side of the desk, I’ve noticed a strong correlation between submitting to a journal and being asked to referee for it within the next 6 months or so. So, another way to raise your profile as a potential referee for a journal is to submit to it!

“Reviewer Declines Assignment”…

It’s a routine, albeit vaguely frustrating, part of being an editor: You get a new submission, you look it over to get a sense of the content, you search for referees, you send out requests, and then wait for the declines to start trickling in. It’s worse than sending out wedding invitations.

But twice in the last two days I’ve had decline responses which have been a positive delight to receive: The referee suggested alternative people I could contact! Sometimes they’re ones I’ve already contacted, but sometimes it’s a new person I hadn’t considered.

Referees, consider doing the same, if you can. Extra bonus points if you can provide an email or webpage for the people you suggest.

Why Do Referee Reports Take So Long?

Every person who has submitted a paper to a journal has at some point or another asked themselves this question. With so much riding on publication, the wait for referee reports and an editor’s decision can seem interminable. Six months, nine months, a year later…”Why do referee reports take so long? Why must I sit in agony for so long?”

I’ve been there; and now, as a relatively new member of the ranks of journal editorial board members, I’m gaining experience on the other side of the equation when, as an editor, I’m awaiting referee reports from my referees with as bated breath as I do as an author. And in that capacity, I’ve started to ask this question more seriously: Why is it that referee reports take so long?

Reflecting on my own practice, the actual time spent on a referee report generally breaks down into this:

  • My first read through a paper is at a pretty superficial level, focusing on proofreading. Where are the spelling and grammar errors, where are the repetitive parts, where are the things that make me go “hunh?” and require more thought. I do this pass first because typos and grammatical errors drive me nuts, and if I don’t have them marked out, I cannot set them aside and focus on content.
  • My second read through is then focusing on the content: Are the arguments cogent, is there material that isn’t addressed that should be, do the things which made go “hunh” on the first read still do so, or do they make more sense now that I’m paying more attention to the content?
  • This second read-through often prompts a literature search where I go to either double-check my memory of something, or double-check what the authors say about a particular fact, or to compile a list of references that would be beneficial to the author to consult.
  • If the paper is technical, then it is after the second read-through that I sit down and work through any of the proofs that were not completely transparent.
  • Then, I sit down with my annotated copy of the paper and go through the laborious process of typing up my notes and turning them into a useful report (see What Makes a Good Referee Report).

Each of these stages individually takes, on average, less than an afternoon. Thus, there is in principle no reason why I shouldn’t be able to turn around referee reports in a week.

But of course, in principle, that doesn’t happen. There are often things higher in the priority queue which cause the tedious task of writing referee reports to get pushed down, so I don’t get around to printing off the paper right away, or maybe a week elapses between each of the steps. Of course, some delays are legitimate: If a referee request arrives while I’m traveling, and I don’t have access to a printer, there is an enforced delay before I can start. Or perhaps I’m on vacation, or I’m home with a sick child. But even with such delays, the only real excuse I can give for not getting the majority of referee reports turned in within a month is procrastination.

The more that I think about this, the more I think that this aspect of our publishing practices needs to change. Many editorial boards (my own included) have a policy of 60- or 90-day initial deadlines for reports, and the motivation for this policy is that it gives the referees time to do their jobs thoroughly and without being rushed, so that the end result is of higher quality. But given my own experience as a report writer, I have to question whether longer deadlines actually do result in better reports, rather than just later ones. Working under the assumption that a referee will take roughly the same amount of time to write his report whether he is given 28 days, 60 days, or 90 days, what are the potential drawbacks of having shorter referee periods? The most obvious is that more people will turn down the request, due to legitimate reasons for not having the time in the up-coming four week (say) block, due to conferences, holidays, other non-standard obligations, etc. The next worry is that the reports received will be of a lower quality, or shorter, or not as detailed. These are legitimate worries (and there may be more: Please share yours in the comments!) but I think both can be addressed.

In the first case, we have a referee who would otherwise have accepted to review a paper but declines because the referee period is too short. This can be accommodated by the introduction (or higher uptake) of another practice: Asking the referee herself to set a reasonable deadline (which will still likely be less than 60 or 90 days, even if more than 4 weeks) for the report. The few times that I’ve been approached by a journal which defaults to a 4 week, or otherwise similarly short period, and have been uncertain if I could accommodate that deadline, I’ve replied suggesting an alternative, usually only 1-2 weeks later, and these suggestions have been met with pleasure. I’ve also found, personally, that when I set my own deadline, I’m much more likely to meet it (or complete it early) than if I’m working under a deadline set by someone else — even if the latter is further in the future.

Regarding the second, I’ve already discussed above how in my own practice, having more time to write the referee report doesn’t necessarily result in me actually spending more time to write it. I expect this is similar for many others. In the case where this actually is a problem, where, e.g., two reports are received within the four week period and they are not sufficiently detailed or useful enough for the editor to make his decision, then one can simply either ask a third referee or to ask one or both of the referees who’ve already responded to expand on unclear parts in their report. Either of these options is likely to still result in a faster turn around time, from receipt of the submission to the decision by the editor.

In principle, the length of time given for referee reports seems to me to be one of philosophical culture and practice, rather than of genuine need. In other fields, short turn around time for referee reports is the norm, and the practice works well because it is a part of the academic culture. In computer science, most conferences require full papers at the time of submission, and these are rigorously refereed sometimes with extremely short turn around times (my husband has been on the programme committees of conferences where he’s given 2-3 papers to report on in a 7-10 day period). Because these papers are technical in nature, and thus take more time to do a thorough reading of the proofs and definitions, the amount of work that goes in to producing a useful report is, in many cases, significantly more than goes into refereeing a philosophy paper. Of course, one difference between this situation and that of ordinary journal submissions is that programme committee members know roughly when they will be receiving their submissions and have to write the reports on them, and so can block out the relevant time on their calendar at the time they agree to be on the committee, whereas referees for journals are generally contacted out of the blue at any given time. Nevertheless, given a deadline of 4 weeks instead of 10 days deals with much of the possibility of receiving a request when one is simply unable to deal with it in the required time, and the possibility of suggesting an alternative deadline means that such cases can be effectively and easily dealt with.

At this point, I have myself pretty well convinced that there is not much reason to have extended deadlines for referee reports (and intend to take this issue up with my editor-in-chief). I would love to hear arguments in favor of the status quo in the comments!

How Much is Too Much?

[Note: This was originally published on Monday, then accidentally deleted yesterday, and now re-published today. Apologies!]

Two anonymous readers write asking about self-plagiarism:

(#1) I’m been wondering lately about “self-plagiarism” and to what extent it is acceptable to reuse material from ones own previous papers. It seems to me that this is actually quite common in philosophy, but I wonder how to approach editors about this.

I’m thinking of submitting a paper to anthology, and (maybe) later another paper to a journal. The latter paper would reuse material from the anthology-paper, but develop it in a slightly different direction. To what extent would this be acceptable? Should one alert the journal editor about this and ask whether it is okay?

I realise that there are copyright issues here, but my question is more about the ethics of self-plagiarism. I would be interesting to hear what editors think about this.

(#2) What’s protocol for submitting two articles that have a significant amount of overlapping content (say, a page) to separate journals? The articles, as I am imagining the situation, are otherwise significantly different.

There are three sorts of contexts where the issue of self-plagiarism generally arises. The first is when you have the same material that you want to present to two, mostly disjoint, audiences. The second is when you are presenting new material which builds on previous material you’ve already published. The third is when you are presenting new material inspired by the same background setting as something you’ve previously published (even if the new paper doesn’t explicitly build on the previous publication).

Since I work in a field where I publish to two almost disjoint audiences (history of logic and mathematical logic), I face all of these situations regularly. In almost every paper I write, before I can present the new and interesting material, I have to provide some historical, expository information. (Once you’ve written more than a few papers on obligationes, it becomes very hard to present the same background material in new and exciting ways! But since my audience is rarely historians, I can’t assume any of them already know anything about the genre.) So I’ve developed a few personal guidelines:

  • There should be as little verbatim material as possible. (You cannot be sure that you won’t end up having overlapping readers — especially if the readers of the second paper are assiduous in following up references to the first — and no one likes to read exactly the same paragraph over and over.) If you can rewrite the material, even a little bit, do so.
  • Where you can get away with simply referencing the previous paper, do so. (This is easier to do when what you’re referencing is, e.g., previously proven theorems.)
  • Be explicit. If the argument has appeared in print in a different version, say so.
  • Ask yourself what you hope to gain by re-packaging the argument and presenting it to a new audience. What can the new presentation give your new audience that they wouldn’t be able to get from the previous one? What do they gain? This can be either the addition of necessary background information that was assumed in the original paper, the augmentation of the paper with new arguments, responses to criticisms raised to the previous paper, discussion of how the subject of the paper is relevant to the interests of the new audience, etc.

There is no hard-and-fast quantitative guideline that can be given as to “How much overlap is too much.” Any attempt to give such a quantitative rule (“75% is too much”; “50% is too much”; “25% is too much”; “any is too much”) could only be justified by circling back to the motivation behind both of the questions above. These questions arise from the stand-point that plagiarism is bad — a foundation that I think all agree on. In my opinion, whether the work being plagiarised belongs to the person who is plagiarising or to another does not really matter: The standards of what counts as plagiarism should be the same in both cases. (I know not everyone agrees with this, and invite dissenting arguments in the comments!) But before one can say “how much (self-plagiarism) is too much”, we need to first consider why plagiarism is frowned on, because this differs between self- and non-self-plagiarism. Plagiarising someone else’s thoughts is an attempt to appropriate for your own credit something due to someone else. Plagiarising your own thoughts is an attempt to receive credit twice for the same idea.

This gives us an important distinction for the question of “how much is too much”: Ideas. There is a qualitative difference between self-plagiarism of arguments and self-plagiarism of expository matter, with the former being significantly more problematic than the latter. In fact, for the most part, the latter is going to be almost completely unproblematic, so long as it is restricted in amount and the other guidelines I suggest above taken into account, and this is because if you’re writing more than one paper in the same subject material, there is no way to escape the need to present the same expository information in more than one paper. But in a profession like philosophy where publications matter so much, a problem arises in when there is an appearance that an author is attempting to double-dip, that is, to get two papers for the “price” of a single idea.* So what you want to do is show — both to your reader and to the editor — that you are cognizant of this, and you are trying to offer more than the same paper repackaged different.

The moral of the story: Cite your sources. Be explicit about what you are re-using. Say how the derivative piece goes beyond the original. Document, document, document, and then let the editor make the final decision as to whether there is too much overlap. Then the question becomes not one of “does this paper exhibit plagiarism” but one of “does this paper provide sufficient new and interesting content to warrant publication?”


Footnote

*. This is a norm that differs from field to field. In computer science, for example, it is completely routine for large portions of papers published in conference proceedings to be lifted with minimal change into journal articles which extend the conference results substantially. But these cases are nevertheless still explicitly marked: The journal article will include in its acknowledgements or in the introduction a statement that it is an extended version of one or more conference papers, with full citations.

The Value and Cost of Desk Rejections

Many journal policies include the option of desk-rejecting a submission: To return it to the author with a negative response without sending it to referees.

What is the value of doing this?

Clearly, there is significant value for the editor/editorial staff: A desk rejection is quick, straightforward, and takes a submission out of the queue for good. No searching for referees, no waiting for their reports, no hounding the referees when the reports are late.

For the author, there is the value of quick turnaround: Because submissions which have been desk-rejected don’t go out to referees, there is generally a much shorter time between submission and final decision. This allows the author to promptly turn around and resubmit, rather than languishing in no man’s land for months only to find that his waiting was for naught and he needs to begin the process again.

Thus, it would seem that a strong case can be made for desk-rejecting those submissions which are clearly unsuitable for publication in the particular venue.

However, I think that this conclusion might be a little too simplistic, and that is because not all desk-rejections are equal. In particular, there is a large difference between “We are not going to accept your submission because it is not suited for publication in our specific venue” and “We are not going to accept your submission because we think it is not suited for publication tout court” — but this is a difference only for the author, and not really for the editor. Thus I think the question should be not “What is the value of desk rejections?” but “What is the value of desk rejections when no reason is given?” Here, the answer is not quite so straightforward. Clearly, the reasons why it is valuable for the editor given above all still hold. But there is a sharp drop in value for the author. What good is quick turn-around time if the author ultimately haves no idea why the paper was not only not accepted, but not even sent out to referees? With absolutely no guidance, how is the author to know how, or even if, he should revise before resubmitting? A submission which has been desk-rejected because it is unsuited to a particular venue — but for which there may be a suitable venue out there — will be handled by the author in a different way from one which has been desk-rejected because it is in principle not publishable, but without knowing which is the case, or if the situation is somewhere in between, a desk-rejection has little to no value for an author.

Given the benefits to the editors of having a policy which includes the option of desk-rejecting, I think its clear that editors should retain that option: We truly don’t want to antagonize our potential referee pool by sending them papers which we already know we aren’t going to publish! However, I think that in exchange for the added value that the option gives, editors should consider paying the small cost of providing a few sentence explanation. It needn’t be detailed, but the value of providing the author guidance as to why the paper was desk-rejected outweighs, in my opinion, the cost to the editor of having to add this information to her form-letter rejection, and this cost is one that editors should be willing to pay in order to obtain the value of a quick decision and processing of an unsuitable submission.