Commenting on scientific articles (PLoS edition)

I’ve been taking a look at the comments left on PLoS ONE from inception until August ‘08 (data courtesy ’http://www.scienceblogs.com/clock/2008/08/postpublication_peerreview_in.php’>Bora). Last week’s crowdsourcing paid off and all of the categorization work gone done really quickly – thank you if you participated! Pedro Beltrao and Lindsay Morgan were the random reward winners and will be receiving some magnificent Nature branded marketing crapola shortly.

plos_breakdown.png

Summary

  • 18% of PLoS ONE papers have reader or author submitted comments
  • 39% if you count comments added by editors (usually reviewer’s comments)
  • Very few comments are of the ‘omg, wow’ variety (as opposed to comments on blogs – this one excepted, obviously)
  • authors are responsible for a high percentage (~ 40%) of user submitted comments
  • 17% of user submitted comments contain interpretation or journal club style precis
  • 13% of user submitted comments are direct criticism
  • 11% are direct questions or requests for clarification
  • These %s are similar to what we saw in the BMC dataset
  • The trackbacks protocol is inadequate for picking up blog chatter about papers

(more below the fold)

Read more

Crowdsourcing comment categorization, pt 2

swag1.jpg

As a reminder, if you haven’t already please do check out ploscomments.appspot.com and categorize some comments. Thanks to Grace we’ve gathered an impressive collection of swag for the one or two lucky contributors selected at random once the experiment is finished (and we remove bogus annotations).

Check it: a USB laser foutain pen from Materials, It’s in my Nature.com tshirts, Darwin anniversary Post-its and pens… all for clicking on some buttons.

http://ploscomments.appspot.com

Digital Lives, Evolution and Chilean Science in Second Life

After a long period of island redevelopment, Nature’s home in Second Life, the Elucian Islands, is suddenly busy again with three events next week!

Monday 9th February: The Nature Darwin Debate 1: Are We Still Evolving?

More details and times

Join a panel of scientists to tackle the issue of human evolution: Is natural selection shaping humans? What will humans look like in 1000 years from now? Is natural selection still shaping humans given that our survival is often more dependent on technology than genes? What are the implications for future generations from sedentary lifestyles, falling birth rates and older parents? What might our species look like 1000 years from now?

Join Sue Blackmore, Henry Gee and Andrew Pomiankowski for a live debate and discussion live in London and Second Life.

Wednesday 11th February: Digital Lives Research Conference: Personal Digital Archives for the 21st Century

More details and times

From 9-11th February, the British Library will be hosting the first Digital Lives Research Project on Personal Digital Archives in the 21st Century. Held at the British Library in London, UK, and funded by the Arts and Humanities Research Council, the conference aims to explore a wide range of aspects of digital lives and the curation of personal digital archives.

Thursday 12th February: Encuentros 2009

More details and times

The annual meeting of Chilean Scientists in Europe at the Max-Planck Institute for Experimental Medicine in Gottingen (Germany). The conference aims to encourage the integration of Chilean and European science communities to strengthen collaboration in global science research. Open to all; featuring talks by Ramon Latorre and Nobel Laureate Erwin Neher

All three are free and open – all very welcome!

Science Blogging Challenge: The Winners

As announced by Corie on Nature Network earlier today, the Science Blogging Challenge that we issued last summer has been won by Russ Altman and Shirley Wu.

Many congratulations to Russ and Shirley. Also, many thanks to my fellow judges, Peter, Cameron and Richard. But most of all, thanks to all those who entered. As Corie mentioned, when more scientists start communicating openly like this everyone’s a winner.

User generated content survey: lazyweb, please help!

faceparty.jpg
Photo by RichardAM

Couple of updates: first, thanks for all your help, things have gone exceptionally well so far with ~ 8k decisions made yesterday. Second, just so nobody is misled this is an NPG web publishing department project that uses PLoS data (with their permission). The raw results and our analysis will be made freely available here.

Got a sec? If you can read and understand a scientific abstract then we need you to help make the publishing world more science 2.0 friendly. Thirty seconds, five minutes, half an hour – whatever you can spare would be great.

Please visit ploscomments.appspot.com and categorize the comments left on papers in PLoS ONE up to Aug ’08.

PLoS ONE were one of the first journals to allow online commenting and (I think) the first to allow blog trackbacks and inline annotations. Last year PLoS’s community manager Bora kindly put together some spreadsheets to let people see the stats behind this reader generated content. Deepak Singh (quick plug: check out Deepak and Hari Jayaram’s Coast to Coast podcast if you haven’t already) and Cameron Neylon checked out the numbers.

I agree with Deepak’s assessment:

Is the commenting on PLoS One at a level that we hoped it would be? Not quite. Is it as bad as some might like to believe? Not quite.

… in the best possible way. Considering how alien the concept of commenting on a paper online is to most scientists PLoS should be pleased with their efforts.

By categorizing comments we should be able to better understand what kind of comments get left and responded to and hopefully we can get a better idea of how they should be encouraged and presented. I’ll make the results publicly available once they’ve all been processed.

Thanks in advance – and have fun!

Well, maybe not fun. It’s actually quite hard work which is why I’m hoping the blogosphere will help out. I promise that we’ll use the results for good.

Introducing Nature Network New York

Since Nature Network launched in 2007, the plan has always been to launch more new city-based ‘hubs’ to serve the many local communities of scientists active on Nature Network.

Today, we are pleased to announce the launch of the first new hub since Nature Network first went live with the Boston and London hubs – Nature Network New York .

Nature Network New York has its own blog, forum, jobs and events listings, geared towards researchers based in the greater New York area and those around the world interested in the scientific activities happening in the Big Apple. With more than a dozen top research institutes concentrated in a relatively small area, New York is a hotbed of research and Nature Network New York intends to bring that community closer together.

NN NY is run by two scientists from Columbia University, Caryn Shechtman and Barry Hudson. They will be posting regularly to the New York blog about the latest news and trends in New York city science. You can check out their latest posts here and here.

Check it out and let us know what you think. You can post to the NY forum or email us at network at nature.com.

ETech 2009

In these belt-tightening times I’m cutting back a lot on conferences. But not ETech — that’s the one event I have to attend each year if I’m to fully recharge my ideas battery. It’s also been a pleasure and privilege to once again act as a member of the program committee, helping to identify the right mix of speakers to enlighten and entertain.

To understand why you should go too, see Brady Forrest’s preview as well as the full conference schedule. A special early registration price applies until Monday (26th January), and Nascent readers can get an additional 10% discount at any time by entering the code ‘et09nat’.

See you there, I hope.

Introducing Scitable

Scitable is a new way to learn about and teach science, and also NPG’s most significant web launch for… erm… a long time. So I asked Vikram Savkar, director of Nature Education and principal force behind Scitable, to provide some info for Nascent readers. Here it is (click on the thumbnail images for full-size screenshots):

Scitable is an online learning tool for students in science. It has three main parts: content, tools, and community. Most of the content has been specially developed for this site by NPG staff editors, supported by an Editorial Board of teaching faculty, leading an author group of faculty, researchers, and science writers. Some of the content comes from various NPG journals, and another segment has been provided by Freeman and Sinauer, two excellent publishers who have partnered with us. All of the content is pitched at the university level. The most interesting tools – yet at an early stage – are intended to personalize learning paths and experiences (utilizing site content) for individual students or classrooms of students, based on their needs and preferences. The community is registered site members – for the most part they are structured within online classroom groups created by their faculty, but we intend to make it easier in time for site members to connect with each other across traditional academic boundaries for purposes of common interest, whether study or dialogue.

Right now the site is focused on genetics, but we plan to expand to other sciences over time. We’ve been in beta for a few months, and out of beta for a week. What I’ve been most excited by is seeing faculty and students in far-flung places pick the site up and begin to use it in their classrooms . . . Turkmenistan, Nigeria, Australia, Brazil, California. Even so, there’s a lot of work yet to do to build this out into its full potential, but we’ve got a great team of passionate and creative people here (near Boston) who are heads down doing just that. I expect the site to change quite a lot over the next year, even in the basics of how students and faculty experience it.

If you’ve taken a look at the site, let us know what you think . . . and if you like it, let the faculty and students near you know about it.

Nature Network bloggers among the best!

Nature Network bloggers have made a strong showing in the anthology of the best science blog posts of 2008, Open Laboratory (click here to see the 2007 edition).

A panel of science blogger-judges (headed up by our own Jenny Rohn acting as this year’s editor) pored over hundreds nominations and selected the 50 best posts, six of which were written by Nature Network bloggers! They are (drum roll, please):

“I get my kicks from thermodynamicks!”

from Reciprocal Space

by Stephen Curry

“Someone should invent a device to look at the micro world”

by Charles Darwin

“On the hardness of biology”

from The End of the Pier Show

by Henry Gee

“On the nature of networking”

from The Scientist

by Richard Grant

“Poster session paparazzi”

from Nothing’s Shocking

by Noah Gray

“In which science becomes a sport – hypothetically speaking”

from Mind the Gap

by Jennifer Rohn

You can find links to all 52 winners (50 posts, one cartoon and one poem) here.

The editors and winners are busy compiling the posts to be published later this month as a book for purchase on lulu.com.

Congratulations to everyone!

XMP Labelling for Nature

There was an interesting piece by Steve Mollman on CNN.com yesterday (Making sense of the ‘semantic Web’) which put forward this example:

"The kiosk takes advantage of the fact that MP3 files are “things” that have already been described in ways that machines can understand. That’s because they have ID3 tags, which supply information on the artist and album. An MP3 file on an iPhone is already a semantic annotated object, which means it’s easily read by a computer.“

Now if that story had instead talked about a PDF instead of an MP3, and if XMP packet was substituted for ID3 tags, then any scholarly article could lay claim to being a ”semantic annotated object … easily read by a computer".

Yesterday’s issue of Nature was the first NPG title to go live with such marked-up PDFs. The screenshots below from Acrobat (File > Properties, CMD-D / CTL-D) show what the user might see both with (bottom-left) and without (top-right) semantic markup.

pdf_props.png

Fair enough as far as that goes, but to a machine it’s a whole other game. We now have a complete bibliographic record (including DOI) embedded in the PDF using structured markup. And, moreover, we also have a solid bedrock for adding in any additional metadata should the need arise. This semantic labelling is available on all new issues of Nature and will be added to other NPG titles over the coming months.

XMP as a labelling technology could well go a long way towards addressing concerns raised by Olivia Judson in an op-ed piece earlier this week in the New York Times: Defeating Bedlam. The author decries that “downloading papers from journal Web sites” means that “access to information is easier and faster than ever before … but there’s been no obvious way to manage it once you’ve got it.” Those days may soon be over.

Now with XMP all manner of scholarly content – documents, images and other media types – can be properly labelled and many programs (not just Zotero and Papers which she reviews) can directly profit from the richness of semantic web descriptions.