The dos and don’ts of communicating with editors and reviewers

Some thoughts and advice from the editors at Nature Methods on communicating with us and our reviewers, particularly on matters of disagreement.

In the over nine years that we at Nature Methods have been interacting with authors and reviewers we have experienced a great variety of communication strategies. Some work well…others don’t. In our October Editorial we discuss how neglecting to word criticism productively can undermine the value of the criticism and short-circuit this critical aspect of scientific discourse.

In the three posts that follow we provide practical advice for communicating with editors and reviewers during three critical steps of the publication process. These are: the cover letter, the rebuttal letter and the appeal letter. We hope you find these guides useful and encourage readers to comment on the points made and suggest dos and don’ts of their own.

How to write a cover letter
How to write a rebuttal letter
How to write an appeal letter

Update: It has been suggested that we write a dos and don’ts for reviewers. We agree this could be just as useful for improving the peer review process, possibly more so, and hope to be able to provide this soon.

How to write a cover letter

Part one of our 3-part series on the dos and don’ts of communicating with editors and reviewers.

A good cover letter is a crucial part of the manuscript submission package to Nature Methods. It is not simply an archaic form of communication that is becoming obsolete in a digital world; rather, it should be viewed as an opportunity to convey many important pieces of information about a paper to the editors.

Manuscripts submitted to Nature Methods must first pass an editorial evaluation stage, but as professional editors, we are not experts in every scientific field that the journal covers. Providing context for the paper in a cover letter not only can help the editors reach a quicker decision but also can sometimes tip the balance in favor of sending a borderline paper out for peer review.

Here are some practical tips for potential authors.

The DOs:

  • Do give a brief, largely non-technical summary of the method. Explain how it will have an impact and why the method and its applications will be interesting to a broad biological audience. This can include more forward-looking information about potential future applications that authors may be reticent to share with reviewers or readers of their manuscript. Such a summary is especially crucial for highly technical papers, where the chance that the advance may not be fully appreciated by the editors is often higher.
  • Do put the work in context. Briefly explain the novelty and the specific advances over previous work but be realistic about what the method can and cannot achieve. Many authors are hesitant to compare their work to previous methods for fear that it will appear to reviewers that they are putting down the contributions of other researchers. But editors may not be aware of the nuances of various approaches to address a methodological problem and are more likely to reject a paper without peer review when the advance over previous work is not clear. Authors should not hesitate to discuss freely in the cover letter why they believe method is an advance (most ideally, backed up with strong performance characteristics in the manuscript!).
  • Do suggest referees. If the editors decide to send the paper for peer review, providing a list of potential referees, their email addresses, and a very short description of their expertise, can help the editor assign referees more rapidly. Of course, whether the editor decides to use any of the suggested referees is up to him or her. This is also the place to list researchers that you believe should be excluded from reviewing the paper. (Please note that the names of excluded reviewers should also be included in the relevant field of the online submission form.) The editors will honor your exclusion list as long as you don’t exclude more than five people; if you exclude everyone relevant in a scientific field such that the review process will not be productive or fair, the editor may ask you to shorten the list.
  • Do tell us about any related work from your group under consideration or in press elsewhere. Explain how it relates, and include copies of the related manuscripts with your submission.
  • Do mention any unusual circumstances. For example, known competition with another group’s paper, co-submission to Nature Methods planned with another group, or co-submission of a related results paper to another NPG journal, etc.
  • Do mention if you have previously discussed the work with an editor. As editors, we meet a lot of researchers at conferences and lab visits and many papers are pitched to us. A brief mention of when and where such a conversation occurred can help jog the memory of why we invited the authors to submit it in the first place.

The DON’Ts:

  • Don’t simply reiterate that you have submitted a paper to us and/or copy and paste the title and abstract of the paper. The cover letter should be viewed as an opportunity to present useful meta-information about the paper, and not tossed off simply as a submission requirement.
  • Don’t go on for pages about what the paper is about and summarize all of your results. The editor will always read the paper itself so long cover letters are usually redundant. A one-page cover letter in almost all cases is sufficient.
  • Don’t use highly technical jargon and acronyms. Explaining the advance in a general manner can go a long way in helping the editors reach a quicker decision; cover letters that are largely unreadable are of no help to the editors.
  • Don’t overhype or over-interpret. While a description of why the method will advance the field is definitely appreciated, obvious overstatements about the impact or reach of the work do not help and can even reflect poorly on the authors’ judgment of the needs of a field.
  • Don’t assume that going on about your scientific reputation or endorsements from others in the field will sway us. This is not pertinent to our editorial decision. Our decisions are based on whether we think the paper will be a good editorial fit for the journal, not on the laurels of the authors or because someone important in the field suggested that they submit the work to Nature Methods

And finally, a minor editorial pet peeve:

  • Don’t address your cover letter to “Dear Sir.” This is antiquated language, not to mention often incorrect, given that two-thirds of Nature Methods’ editors are women. Stick to the gender-neutral “Dear Editor” in cases where you are not addressing a specific editor.

Don’t miss parts 2 and 3 of this series of posts covering rebuttal letters and appeal letters. We encourage questions, comments and feedback below. The editors will do their best to answer any questions you have.

How to write a rebuttal letter

A well written rebuttal letter is critical in any resubmission. 

Once the initial reaction, be that joy, anger or frustration,  to receiving feedback from editors and reviewers about one’s work has subsided, it’s time for our authors to make one of two decisions:  continue to go after a Nature Method paper  or take their work to another journal.

A realistic look at how the reviewers’ requests can be met will go a long way in helping to determine whether a revision is likely successful and to avoid a futile resubmission.

If authors want to resubmit in cases where the editorial decision was negative, and referees were critical and asked for a lot of additional information, the first step, before embarking on any revision, should be an appeal (see the post on “How to write an appeal letter” for more details) and rebuttal letter to the editor to discuss whether a proposed list of additional information is likely to address the referees concerns.

Authors who receive a positive editorial decision and who are confident that they can address the reviewers’ points nevertheless have to submit a rebuttal letter with their revision.

The rebuttal letter is an author’s chance to directly reply to the reviewers, announce plans to improve the work, clear up misunderstandings or defend aspects of the work. How it is written can make a big difference in whether or not an appeal is granted and how the reviewers judge the revision.

The DOs:

  • Do acknowledge that the reviewers spent a substantial amount of time looking over the paper – rebuttal letters that thank the referees for their time and comments set a positive tone and ensure that the exchange takes place on a productive footing.
  • Do acknowledge that a misunderstanding may be due to poor presentation on your part, not lack of expertise on the reviewers’,  and phrase your reply accordingly,  taking the opportunity to clarify.
  • Do copy the full text of each reviewer’s comments in your rebuttal and reply to every concern raised by each reviewer immediately after each point in a concise manner that clearly states how you plan to address it (experimentally or editorially) or point to data that already addresses it which the reviewer appears to have missed.
  • If you cannot address a point at all, explain why not.
  • Do number the comments or at least break them into paragraphs, and use different fonts or text colors to distinguish the reviewer comments and your reply, rather than write a single reply to an entire review in summary form.
  • Do include relevant citations with full references or dois so they can be easily looked up, rather than just cite by First Author et al.
  • Do include pertinent new data as embedded figures, tables, or attachments,   indicate where in the manuscript you added the information; give page numbers, figure panels, Supplementary material etc., so editors and reviewers don’t have to go on a search for the new data. If any of this information will not be included in the revised paper explain why not.
  • Do be succinct and to the point and avoid epic discourses.  In the case where more than one referee has raised the same concern, it’s best to cite “see response to point 2 from Reviewer #1”, for example.
  • Do remember that each reviewer sees all comments and your replies so be equally respectful to all.

The DON’Ts:

  • Don’t vent or accuse the reviewers of bias or incompetence. We have read countless times that “ ref 2 is lacking expertise and completely misses the point” etc. and one wonders what the goal of such blanket statements is. They serve no productive purpose and instead potentially bias all referees, even the positive ones, against the work.
  • Don’t plead that for personal or monetary reasons critically important experiments can’t be performed. While we hear the plight of underfunded labs we don’t make exceptions for these reasons.
  • Don’t ignore specific requests by referees without comment and selectively only answer a few queries.
  • Don’t rephrase a referees’ point to give it a slightly different meaning that you can more easily address.

Don’t miss parts 1 and 3 of this series of posts covering cover letters and appeal letters. We encourage questions, comments and feedback below. The editors will do their best to answer any questions you have.

How to write an appeal letter

Although usually unsuccessful, a strong appeal letter can be an important tool for authors.

Rejection is never easy. You’ve put long weeks, months, maybe even years of work into a project that you think is perfect to publish in Nature Methods, so your feelings of disappointment, anger, frustration or self-doubt are completely understandable. Your first instinct might be to hit “reply” and send an angry email to the editor. But your best bet is to take some time to cool off, then move on and submit the paper elsewhere. If you are convinced, however, that a serious error has been made or that you can fully address the specific criticisms raised by the editors or by referees, then you may send a constructive appeal letter to the editor.

Nature Methods has different types of rejections, with or without peer review. There are outright rejections (which represent the vast majority), and then there are those rejections where the editor indicates that a manuscript could be reconsidered if the authors can address specific shortcomings. The editorial decision process is of course a subjective and imperfect one. Appeals, however, are usually unsuccessful. Those that are successful are those where the authors make a strong case for reconsideration, typically by proposing to add new data that will strengthen the application or demonstrate how the work is a strong advance over existing methods.

Be aware that appeals are necessarily given a lower priority than manuscripts still under consideration. Decisions on appeals can therefore take a considerable amount of time and the majority of appeals are turned down. Decisions are usually only reversed if the editors can be convinced that the decision to reject was a serious mistake, if the authors can add a substantial amount of data to address certain shortcomings, or if a negative referee is found to have made serious errors or show specific evidence of bias.

An appeal letter is not the same as a rebuttal letter to referees (see the related post, “How to write a rebuttal letter”). An appeal letter is only read by the editors, so sensitive information not meant to be seen by the referees can be included.

Here are some things that do and don’t work when writing an appeal letter.

The DOs:

  • Do consider whether you have a good case for appealing that is worth investing time in the process. By editorial policy, appeals must take second place to new submissions. This means that it can take as long as several weeks for the editors to discuss an appeal, possibly get input from referees, and reach a decision. Unless your case is very strong, it will save you precious time by accepting the editorial decision and submitting the manuscript elsewhere.
  • Do clearly explain the reasons why you disagree with the decision to reject. In some successful cases, authors provide new information, not apparent from the original submission, explaining how the method will have a strong impact on a broad audience. Ideally such information would have been included in the cover letter with the original submission (see the post, “How to write a cover letter”), which can help avoid the need for a lengthy appeals process for a manuscript that is otherwise a good candidate for peer review.
  • Do explain how you plan to rectify any major shortcomings pointed out by the editor or by the referees. If you are willing to add data to the paper to address the shortcomings, explain what this data is and what it shows. If you have figures or tables prepared, include them with your appeal letter. (However, don’t yet rewrite your manuscript – since most appeals are turned down, this is usually just a waste of your time.) If you have a valid reason for not including such data, explain why not. 
  • Do include a separate point-by-point rebuttal letter to referees to assist the editors in reaching a decision (see the post on “How to write a rebuttal letter”). If the editors feel that a rebuttal letter is required to help them reach a decision, and one is not included, they will request one of you.
  • Do provide evidence for any accusations of referee bias. Describe in specific terms why you believe a referee is biased or has made technical errors in their review. In our experience, it is extremely rare that ALL of the referees of a paper would be biased or misjudge its impact. Don’t try to guess who the referees were (you will most often be wrong). In cases where one set of negative referee comments is far out of line with others that are generally positive, we often will consult with the positive referees to determine whether the dissenting referees’ concerns are serious and how they should be addressed.

The DON’Ts:

  • Don’t do anything in the heat of the moment but take some time to cool down and consider whether you would be better off resubmitting elsewhere.
  • Don’t simply reaffirm the importance of the work, write “we think you are making a mistake” or urge us to send a manuscript out for peer review without providing any justification. Appeal letters lacking a good justification will not convince us to change our minds.
  • Don’t try to bribe us with promises of high citations. While of course we hope for high citations for each research paper we publish, citation potential is far from being the most important editorial consideration (and it cannot be accurately predicted, anyway). Papers must first meet our standards of methodological novelty and potential community interest and impact.
  • Don’t assume that the paper must be of interest to us because we have previously published a similar paper. Editorial standards are constantly evolving, and the methodological novelty may be compromised by our previous publication. Additionally, we strive to publish a variety of novel methods across fields, so we must consider what is currently in our pipeline.
  • Don’t bash previous work. As editors, we want to know how a new method addresses certain shortcomings or significantly expands the applicability of a previously published method, but this discussion should be fair and balanced. Don’t simply say “the previous method doesn’t work,” explain why, and ideally provide experimental evidence. Providing a detailed comparison to previous methods in your paper in the first place can help avoid the need for a lengthy appeals process for a manuscript that is otherwise a good candidate for peer review at Nature Methods.
  • Don’t expect us to be swayed by your scientific reputation. While it is informative to give some background of your expertise in a field, we make decisions based on the fit of a paper with our journal in terms of scope, novelty and potential impact, not simply because the work comes from a good lab. The fact that you have coauthored papers in high-impact journals will not lead us to reverse our decision.
  •  Don’t rely on “celebrity endorsements”. It is good to hear that a leader in the field has read and likes your paper, or that 50 people came to view your poster at a conference. But if we feel that the paper is editorially not a good fit for Nature Methods, this is unlikely to make us change our minds about rejection.
  •  Don’t insult the intelligence or competence of the editors or referees. We know that rejections are upsetting and can often seem unfair. But personal attacks and bullying could compromise your success in an otherwise promising appeal situation.
  •  Don’t appeal every decision. Remember the old saying, “you’ve got to pick your battles.”

Don’t miss parts 1 and 2 of this series of posts covering cover letters and rebuttal letters. We encourage questions, comments and feedback below. The editors will do their best to answer any questions you have.

The NIH’s BRAIN Initiative interim report – notes and thoughts

The first official report lists the scientific priorities that will be funded by the NIH as part of the BRAIN Initiative

brain_map

{credit}Margrie & Osten, Nat. Methods{/credit}

Yesterday evening we heard the first official report that delivered some details about what scientific areas the BRAIN Initiative (at least the part coordinated by the National Institutes of Health) will focus on and what its general approach to science funding will be.

Cori Bargmann and Bill Newsome (co-chairs of the NIH-appointed panel that is advising the NIH-director about the plan) spoke through a webcasted seminar to explain the conclusions that arose from the series of scientific workshops and meetings that have been taking place over the summer to discuss what could be the scientific priorities of the BRAIN Initiative. These priorities will set the ground of the research areas to be funded by the BRAIN initiative NIH funding in Fiscal Year 2014 (with a budget of US $40M).

The overall goal of the project was summarized as focusing of developing tools and resources for analyzing neuronal circuits and their function in living organisms.

The two scientists delved on a number of principles that applied to the overall initiative, such as promoting platforms for data sharing, promoting interdisciplinary research and focusing on a variety of experimental organisms and studies across temporal and spatial scales.

The specific areas that will be funded through the Initiative are summarized below:

  1. Generate a census of cell types in the nervous system. Including neurons and glia and techniques for targeting them. To be attempted in parallel in human and animal samples. Methods developed for this should apply across species.
  2. Create structural maps of the brain. This means cell to cell level connections in different animal models. This would complement the Human Connectome project (based on macroscale neuroimaging approaches).
  3. Develop new large scale methods for recording chemical and electrical activity of neurons. Scaling up of electrophysiological and imaging methods as well as completely new technologies.
  4. Develop a suite of tools for circuit manipulation and perturbation of circuit function. A push for the development of technologies like optogenetics that enable manipulating nervous activity in ways that resemble natural activity patterns.
  5. Linking brain activity to behavior. Activity monitoring at the same time that behavior is monitored. Highlighting the importance of making simultaneous measurements during long periods of time and during different types of behaviors.
  6. Integrating theory, statistics and computation with experimentation. Importance of theoretical frameworks that could explain principles of brain function.
  7. Delineate mechanisms underlying macroscale brain imaging technologies, as used in humans.
  8. Create mechanisms to enable collection of human data.
  9. Provide training so that new methods reach the community and promoting interdisciplinary research.

Although these FY2014 research priorities are presented as 9 independent entities, the goal is really to integrate these approaches as much as possible —but how exactly this integration will take place or be promoted is to be revealed by June 2014.  The goals are also highly ambitious and will require much more funding than the BRAIN Intiative’s current budget.

The speakers noted that the goals were not “to develop tools for tools sake” but tools that could have applicability. Innovative tools, thoroughly validated and applied in real nervous systems, improved through iterations and to ensure that they are disseminated efficiently to the community.

The commission also highlighted that their goal was not to deliver the solutions but the problems. Solutions to addressing these challenges are to come from ‘bottom-up’ approaches proposed by the community of scientists.

Tools for studying individual cells in the brain or the entire brain as a whole exist and continue to be very useful. But methods for understanding how connected networks of cells in the brain work and relate to behavior are still largely missing. Even maps of these connected entities remain unknown. Focusing funding on better tools to close this gap will be exciting and productive for advancing neuroscience as a whole.

The conclusions disclosed in this interim report are very much in line with what was expected of the project as announced a few months ago. They also largely agree with the main scientific goals that were deemed to be the top funding priorities for the National Science Foundation for the BRAIN Initiative (which will have US$20M to contribute, as well). Indeed, many of the topics covered in these 9 areas were things we and others discussed in editorials and commentaries related to the BRAIN Initiative in our pages and in this blog. The working group that has developed these priorities has had an inclusive, overarching frame of mind and included most of the major challenges that neuroscience currently faces, as most scientists would probably agree.

As we’ve said before, a push for technology development in neuroscience, with clear goals and challenges that these tools need to tackle, will surely be an efficient way of advancing the science of the brain.

A retraction resulting from cell line contamination

After nine years in print, Nature Methods today published its first retraction; one that could have been prevented by cell line authentication. What does this mean for journal-mandated cell line testing?

Gliomasphere image

Two-photon fluorescence image of live primary gliomasphere from retracted manuscript.

In a Nature Methods paper published in 2010, Ivan Radovanovic and colleagues described a method to isolate cancer-initiating cells in human glioma without the need for molecular markers. Based on morphology and on a green autofluorescence, the authors reported they could use FACS to sort cancer-initiating cells from gliomasphere cultures (which had been derived from primary tumors). They also detected autofluorescence in cells from fresh glioma specimens, but at a much lower level.

Cells from the autofluorescent fraction could self renew clonogenically in vitro and were tumorigenic when transplanted into mouse brains, the authors reported, and in both cases performed better than non-autofluorescent cells from the rest of the culture or tissue. The origin of this autofluorescent signal was not understood at the time. The authors speculated it may have been related to the unique metabolism of the cancer-initiating cells.

It turns out that most of the primary gliomasphere lines (7 out of 10) were contaminated with HEK cells expressing GFP, leading to retraction of the paper. Using short-tandem-repeat (STR) profiling of two of the lines the authors determined that the contamination occurred over the course of culture in the lab: samples taken from early passages match the original tissue from which the lines were derived, but later passages no longer do so.

It is hardly surprising that the first retraction in Nature Methods is due to cell line contamination, a well acknowledged problem. A 2009 Editorial in Nature pointed to the disturbing results of cell testing by repositories which indicated that 18-36% of cultures were misidentified. It called on repositories to authenticate all of their lines, and for major funders to provide testing support to grantees. At that point funders could require cell line validation for investigators to retain funding, and Nature would require that all immortalized lines used in a paper were verified before publication. Unfortunately, it is now 2013 and we are still far from this goal.

But progress is being made. Community-based efforts are alerting researchers to this problem and providing resources to help them avoid being misled by erroneous results caused by cell line contamination. A 2012 Correspondence in Nature by John R. Masters on behalf of the International Cell Line Authentication Committee (ICLAC) pointed to the following resources available to researchers:

Please go to the ICLAC website for the most recent version of each of these documents.

Meanwhile in early 2013, at the publication end of the process, the Nature journals published coordinated editorials announcing a reproducibility initiative and stating that “…authors will need to […] provide precise characterization of key reagents that may be subject to biological variability, such as cell lines and antibodies.” In practice, the Nature journals are currently requiring all authors to state whether or not testing was done but are only requiring testing in cases where it makes particular sense.

Advocates for mandatory testing have cogent arguments for a uniform mandatory testing policy. First, it would avoid sending a confusing message; second, researchers can’t be certain that cell identity or mycoplasma contamination aren’t affecting results; and finally, continued publication of inaccurate species and tissue designations of misidentified cell lines continues to propagate misinformation.

In the work described in the retracted 2010 manuscript from Radovanovic and colleagues mandatory testing would certainly have been beneficial. However, for probably the majority of work published by Nature Methods there is no question that testing would have no impact on the reported results. For example, in 2011 and 2012 we published at least 17 manuscripts reporting new fluorescence microscopy methods and using imaging data from cell lines to assess the performance of the techniques in measuring fundamental cell properties such as the appearance and width of actin or microtubule filaments, membrane vesicles or other universal cellular structures. Cell line identity and even mycoplasma contamination would not impact the efficacy or conclusions of these measurements. This same situation exists for the validation and testing of many methods in other research disciplines such as proteomics, genomics and biophysics.

Even if these labs should be doing cell validation and mycoplasma testing as a matter of course as part of proper cell culture procedure, mandating that all these studies include such testing as a requirement for publication is unjustified.

But clearly even our most recent efforts at improving compliance with good testing practice will not be sufficient to eliminate cell contamination as a problem in work published in Nature journals. A possible solution may be to require testing by default but authors would be permitted to argue why, in their case, testing is clearly unnecessary. Editors (possibly with reviewer input) would be the final arbiters and would need to ensure that although the lines must be named and sourced, no species or tissue identifiers should be included in the manuscript in the absence of proper validation.

Technology development labs or others that only use cell lines for purposes distinct from biological investigation could continue to avoid testing. But any lab that might potentially use their cell lines to obtain biological results would know that they should institute a proper testing regimen or risk their work not being publishable in a Nature journal.

At this point this is only an idea based on our experience at Nature Methods. We encourage the community to comment and let us know what they think.