Nautilus

I agree completely- many of these potentially important papers are comletely unreadable due to the expansive Supplementary Material.

I agree that the loss of the complete article is a tremendous loss to the scientific community itself. It is now tremendously difficult, if not impossible, to fully understand and critique (much less replicate!) the experiments reported in a paper using only printed materials.

While that seems fine to those at wealthy universities that can afford online access to materials and ample computers, it shuts out those at institutions all over the world who cannot afford the constant upkeep of pricey online subscriptions, internet connections, computer labs, and printers.

The digital format, while convenient, may prove to be more ephemeral than expected. Are we willing to see our scientific contributions evaporate as formats change, or are we committed to placing our work into print and depositing it around the world for generations of future scientists?

Hmm. . . /Nature/ has always been long on the exciting results and short on the boring but useful nitty-gritty. It's very difficult to follow a method from a paper published in /Nature/, and the web-based SI is a good way around this (after all, reproducibility is supposed to be the name of the scientific game).

Maybe the Editor(s) need to be a little more editorial and clamp down on the SI abuse?

I completely agree with Larry Benson.

@Kristy Lamb: My Italian university cannot sure be defined a "wealthy university" for USA/UK standards, but I can't understand how doing science is today possible at all without an Internet connection. As of today, I guess a print subscription plus archiving space could cost in the end much more to an institution than a bunch of cheap PCs, ADSL connection and cheap inkjet/laser printer. However I agree with the concern of "evaporating science history". I think the optimal solution should be that larger, national libraries could be obliged to keep print copies of whole PDF articles for archival, while smaller universities etc.can grab them from the Net or ask to the larger libraries.

@Richard Grant: I don't understand what do you mean as "clamp down the SI abuse" - that is, having shorter SI than now? The main problem with SIs is that they are usually not as readable and carefully crafted as the paper itself, but not their raw information content. I have sometimes read huge SI that were wonderfully made and useful (see for example Rothemund PW, Nature. 2006 Mar 16;440(7082):283-4.). I guess editors should stress for SIs to be as much readable and useful as the paper itself, regardless of information content. Otherwise we don't solve any problem.

Finally, somebody has told it! The supplementary information usually is more extensive than the paper. But on the other hand, how long must we describe how to perform a western-blot?

Complete agreement Larry.
More methods and data are getting marginalized to SI. The SI can be too large, is less well reviewed and formatted, and too often is a mixture of the possibly interesting with the trivial.
Perhaps a good idea originally, especially for large databases that are not readily publishable, this practice has become a tedium for the reader.

Maxine adds: thanks for your comments, Danny. Just to note that on Nature, we publish online methods as an integral part of the paper online, ie they do not appear in the print journal but they are integrated into the full-text (HTML) and PDF in the online version, and fully edited.

To Richard and Massimo: Massimo is correct to say that on the Nature journals we peer-review all material we publish as a paper, whether that information is in the main paper or published online-only as SI. The components of the SI are merged into a single online PDF for ease of download, but the text and figures are not edited first. But the process up to the point of acceptance (peer and editor review) is identical.

I agree with Larry Benson: SI were first a great way to link a paper with files that are difficult to print such as movies/sounds/large databases. These files were generally described in the main paper. Unfortunately, SI now contain vast amount of data that could either be part of the paper or not shown (at the discretion of the reviewer/editor).
Isn't it time for Nature and all the other journals to decide and define together what can be presented as supplemental and restrict the use of SI to files that cannot be printed?

There are three important problems with supplemental material that have not yet been successfully addressed by the journals. (Note: I have found these to be general problems, not problems just with Nature.)

1. Supplemental information is rarely reviewed to the same extent that the main paper is. Many journals make it difficult to review the supplemental material (forcing reviewers to go online and download each component separately, for example). Many authors seem to try to shove incomplete or likely-but-less-defensible results into the supplemental material.

2. Supplemental material is not edited and thus does not have the same reliability as the primary paper.

3. Supplementary material is not normally delivered with the primary paper in downloading formats. (Why don't journals offer downloads of one big PDF with both?)

All of these seem to me to be solvable problems if supplemental material were relabeled as an online part of the actual paper rather than "supplemental" material. Reviewers could be required to state that the supplemental material was ready for publication. Editors could edit the supplemental material. And journals could easily deliver the supplemental material with online versions of the paper.

I don't think the problem with the supplemental material is the material per se. It's the fact that it is treated as supplemental.

Dear colleagues!
Many thanks to the author and colleagues for this very useful discussion. I think that the problem is to define strictly what material can and must be posted in the Supplement section. Ideally I should like to see published in the journal all the material concerning the research without any omission. Then everybody can see that the research is robust and reproducible. For this purpose main points must be in the paper and details (derivative results, databases, and all the material with which it is convenient to work in the electronic form) can be posted in the Supplement. Thank you for attention, best wishes to all of you. Institute of Cytology, Russian Academy of Sciences, Saint-Petersburg, Russia.

I completely agree with this letter. In addition, the practice of banishing materials and methods to SI often gives authors (and referees) the false impression that this is something of secondary importance only. As a result, I have on several occasions searched for important experimental details in vain (sometimes even for something as fundamental as which sex of experimental animals had been used). In general, I am not really that much interested in results before I have seen what exactly the authors have done. In my field (behavioral neurogenetics), experimental details are absolutely necessary in order to correctly interpret data. This includes all details on how behavioral tests were conducted, full disclosure of animal breeding procedures, and clear and unequivocal information on the genetic background of experimental subjects. All too often, I find publications in journals like Nature, which hide away Materials and Methods, lacking in these respects.

Maxine adds: thank you for these comments. To clarify -- as noted in the post and comments -- Nature fully integrates materials&methods online in both PDF and HTML (full text) versions.

First, I wish to thank NPG for giving this opportunity to present readers' opinion on SI.

SI is an important one, where the authors can really present more details of experimental methods and analysis, which are essential to reproduce results of the same work and to be followed by non-specialist readers. On the other hand, the less number of pages for the main paper is optimum to present the findings of the authors, which can surely motivate the readers to go through as the paper comes in a good form. My opinion can be realized by looking at, Vol 437|1 September 2005|doi:10.1038/nature03968, pages 121-124, a 4 page letter with 38 pages of SI. But they are worth to be included with the paper as SI.

In addition to Maxine’s solution to this issue, I would like to add one more point that the SI should be welcome and should be thoroughly reviewed in the sense that whether it is essential and worth to be put together with the paper. The people from universities and institutes which do not have subscription to on-line access have to get some other ways, for which again the internet and e-mail facilities are useful.

Maxine adds: thank you for these points. Nature provides a downloaded copy of SI by mail to any reader who requests it from our editorial office, if they do not have internet access. (Reading SI does not require a subscription to the journal.)

I strongly disagree with both "Papers should not need supplementary information" and the suggestions to decrease the amount of SI.
My counterproposal: ALL the data that went into the manuscript should be included in the S.I., or links provided to appropriate publicly funded fully accessible databases, such as GenBank for DNA sequences.
l propose that the full statistical analysis be included for every p-value reported. For that matter, if the authors are performing a t-test, a graph or numerical proof that the data being tested is Guassian distributed should be done.
In biomedical research histology specimens are often examined.
Should we really trust a second year graduate student or rotating medical student to read (mouse tissue) pathology slides? Especially if they performed the experiment and know which slides are what treatment, and what result their boss expects them to see. Any human
research tissue sections should be read by a board certified pathologist, and their reports (patient identifiers anonymized) should be included (preferably in some internationally recognized database).
If the authors said they performed 3 Western blots, but show one tiny row of cropped bands in a figure in the main text why should I believe that (1) they performed the experiment in triplicate, (2) that they spent the time (and money) to perform and evaluate loading controls, (3) that they - or the machine they used - acquired and
analyzed the data correctly. When doing radioactive or
chemiluminescent Western blots, how many graduate or medical students, reviewers or Nature editors even know what optical density is or how to compute it? If you think everyone in your lab and department knows how to compute optical density on an image, take a
poll at your next lab meeting or department seminar.

I completely agree with the comments regarding the excessive use of supplementary online material in journals like Science and Nature. In fact, in my view the practice of these journals in putting the methods into online supplementary material means that these are no longer scientific journals. As such, I rarely even read them any more, because I found myself all too often sitting somewhere with the print journal in hand, but without the access to the electronic files that I needed to actually understand what the authors had done. So today (Nov. 1, 2007) I am for the first time perusing the table of contents for an issue that came out two months earlier. In my mind one cannot have a scientific discussion without a careful discussion of the methodology used to investigate the question.

I do see a role for online supplementary material. But I think this material should be confined to such things as very large tables of data, additional examples that parallel results provided in the print text (instead of using data not shown), the provision of very technical controls that only a very small number of readers might care about, and similar information.