FCW.com News – Data explosion strains peer review

The link above is a stimulating little article on FCW.com about how the US National Institute of Standards and Technology (NIST) deals with the overload of thermodynamic data reported in journals. From the FCW article:

“Recent improvements in measurement equipment mean that an already voluminous amount of thermodynamic data is doubling every 10 years.

That explosive growth is straining the traditional journal-based peer-review system and causing increasing numbers of errors to creep into the data. Companies in the chemical, pharmaceutical and energy industries depend on accurate data for their engineering applications and research projects.

As part of its responsibility for promoting U.S. competitiveness through standards and technology development, NIST worked with industry partners to create a standard data format and online system for verifying and disseminating thermodynamic data.”

The XML-based system transforms the published data into a standard format and stores it in a central database that researchers can access via the Web. When authors submit data, the system automatically checks for inconsistencies and alerts the authors to any questionable data.

south by southwest festivals conferences

At the link above is a podcast of a session from the SXSW (South by South-West) Interactive Festival in Austin, Texas, held in March. From the conference website: "New publishing technologies challenge the traditional structure of peer-reviewed scientific journals. For hundreds of years the “article” has been the primary vehicle for conveying scientific information – but semantic markup, tagging and wiki are reconstructing scientific publications into a flexible and evolving concept." The panel looked at the social and legal implications of “Web 2.0” and the “Semantic Web” as they impact science and scientific knowledge. The moderator was John Wilbanks, Executive Director of Science Commons, Creative Commons, and one of the invited speakers was Timo Hannay, Director of Web Publishing of the Nature Publishing Group. The podcast has just been uploaded to the SXSW site, and is freely available by going to the link at the top of this post.

(Cross-posted on Nautilus.)

Nature’s Journal Club

Our latest blog (link above) is the blog for the Journal Club, a weekly column published in Nature’s Research Highlights pages. Each column presents a researcher’s choice of reecnt paper, explaining the reason why he or she is enthused about it. At the Journal Club blog we invite readers to discuss the subjects raised in the columns. Please do take a look at the entries on the blog, and, as my colleague Oliver Morton puts it, “enrich their comment threads with your insight and speculation.”

(Cross-posted on Nautlius, the author blog.)

C. Cristofre Martin and Kenneth B. Storey write:

Scientists are often involved in the peer review of grant applications and/or manuscripts submitted for publication. We rely on the anonymity of the system to allow us to be completely frank and unbiased in the comments that we provide to the author(s) of the article or grant.

However, we wish to point out an alarming situation. The now heavy reliance on electronic means of communicating between reviewers and publishers/granting agencies in the form of MS Word and other electronic documents, generated PDFs, and other user-generated file formats has created in a situation where anonymity can no longer be ensured.

The reason for this breach in security is that most state-of-the-art software applications will embed information about the creator of the document with the normally invisible metadata of the file. This metadata can be viewed by means as simple as opening the file within a text editor application or by viewing the creator information for a file within the operating system such as ‘Get Info’ in the Mac world or ‘Properties’ on PCs. Typically, the source of this metadata is the user account information that is associated with the specific computer being used to generate the document.

Authors, journal editors, publishers and granting agencies need to be cautious about how ‘anonymous’ information is transmitted between the creator and the recipient. Creators of anonymous documents should check that the programs that are used to create their documents are secure and if not, adjust security settings where possible or delete creator information in the file properties before sending off their reviews. Publishers and granting agencies should also consider adjusting dissemination methods such that original reviewer-created documents are never forwarded directly to authors. The greater use of Web forms for both the input and transmission of reviews is one obvious solution.

C. Cristofre Martin, Department of Biochemistry, St. George’s University, St. George’s, Grenada , West Indies.

Kenneth B. Storey, Department of Biology, Carleton University , Ottawa , Canada.

[Note from Maxine: For information, Nature Publishing Group journals use a Web-based peer-review system to ensure anonymity, as do many, but not all, other publishers. The Nature journals also require large datasets and other supplementary information to be deposited by sumbitting authors into a public database or supplied on CD/DVD for the purposes of peer-review. We do not allow authors to post such supplementary information solely on their own or their institutional websites, partly for the reasons outlined by Drs Martin and Storey. Further details of our policies can be found at the NPG authors and reviewers’ website]

The Species 2000 & ITIS Catalogue of Life will become the comprehensive catalogue of all known species of organisms on Earth. Started in 2000, it is hoped to be complete by the year 2011. As things stand, the catalogue has just reached 1,008,965 species: probably just more than half of the world’s known species. (The final total is expected to be around 1.75 million.)

The catalogue is compiled with sectors provided by 47 taxonomic databases, many containing data and opinions from extensive networks of specialists, so that the complete work contains contributions from more than 3,000 specialists from throughout the taxonomic profession. These databases are peer-reviewed by teams from the Species 2000 and ITIS programme, who also select appropriate sectors and integrate them into a single coherent catalogue with a single hierarchical classification.

It is planned to introduce alternative taxonomic treatments and alternative classifications, but an important feature is that for those users who wish to use it, a single preferred catalogue, based on peer reviews, will continue to be provided.