Action Potential

Neuroscience and Web 2.0: Participation may vary

In the last couple of years, after the recent explosion in the number of resources where scientific discussions can take place rapidly and without boundaries (i.e., using the internet), one could easily have predicted that we were on the cusp of a revolution; the way in which scientists communicated with each other regarding data was about to change forever. Although poster session chatter at your favorite scientific meeting was never going to be replaced, now researchers could interact, trade ideas and get feedback from anyone, anywhere, at any time. Sounds pretty good, right? Unfortunately, it doesn’t seem like neuroscientists are taking advantage of these cool new offerings. I could extrapolate to biology in general, but for more simplicity (and other obvious reasons), let’s stick to what we know best.


I have been thinking about these issues for a long time, ever since I commented on the introduction of Nature Precedings last June (which was my first post on this blog, for those of you keeping score). I was skeptical about the speed with which scientists would give up their unpublished data, and at 57 neuroscience papers currently in the archive, this was even a bit slower than I expected (but other subject areas are doing quite well, with Bioinformatics boasting 110 submissions, for example). Shortly thereafter, I discussed commenting forums in general, and felt again that although adoption may be slow, that this medium would eventually become fundamental to the scientific process and discussion. In my opinion, we have a long way to go before we reach that lofty prediction.

My hand was forced by two excellent and recent commentaries (here and here) that provide many of the theories for why scientists do not widely use these new technologies. I would like to focus mainly on the concept of commenting on papers and data, the style embraced by PLoS, PLoS ONE, Neuron, Nature Precedings, and more recently, in the neuroscience journal club started at Nature Network.

So why are neuroscientists not taking advantage of this seemingly golden opportunity to communicate with one another? Granted, some papers are comment magnets (e.g. this or this, but consider the subject matter and/or implications). But most of the time, papers sit with a “Comments” link beside them that links to nothing. What about telling the author your alternative hypothesis? How about suggesting a future experiment? Why not simply ask a very innocuous question about which buffer worked best for the biochemistry experiments? I can’t put actual numbers on it without some painstakingly-boring grunt work, but I spent quite a bit of time clicking on paper “Comments” links at all of these sites that offer such a luxury, and found very few random hits where any comments had been made. I’ve attended enough journal clubs to know that somebody always has an opinion about most papers, so where are those people now? This should be their critical dream-come-true!

For those few publications that did receive comments, it was even rarer for the authors to respond. I took the liberty of commenting on some papers at Precedings within the categories of environmental sciences or evolution, and on only 1 occasion out of 5 did I receive a response. So why do these authors even post their papers if they are not interested in feedback? It is an obvious question for pre-prints, but even for published work, researchers put their efforts out in the public domain to be scrutinized. The authors should then be obliged to answer such scrutiny if and when it does arise. Can you imagine the oddity of a speaker staring out into space without responding during a question-answer session at a meeting? Well, the cyber-equivalent was going on at most places that I looked.

As Anna Kushnir and David Crotty pointed out in their blog commentaries (see above links), one of the main reasons for a lack of initiating comments is the fear of placing one’s name with a criticism. That criticism may be wrong, too abrasive, not well-written, and now it is associated with the submitting party for all of cyber-eternity (or at least until server failure). At a meeting, after asking a really stupid question, we have plausible deniability. Unless somebody was taping the session, there is no hard evidence revealing our misguided thoughts (saying to our friends: “Well, the speaker didn’t understand my question and blah, blah, blah…”). Not the case with these forums. But even that can be circumvented. Although anonymous comments are not really allowed in these forums, what’s to stop people from simply making up a name and providing a Gmail account address like “sciencegeek@gmail.com” to establish legitimacy for the servers (or moderators, for that matter) accepting the comments? Although I disagree with anonymous commenting, I understand that some prefer this incognito approach. But despite the ability to use this cover of darkness, we still haven’t seen a massive adoption of commenting practices!! So perhaps the age-old beast, lack of time, again wins out and should take most of the credit. Although probably true, I find this to be a lame and sad excuse.

During graduate school, we are taught that we learn the most when we present our data to the community, whether it is at a conference or even at lab meeting. We can learn just as much, if not more, from participating in online forums to discuss data, hypotheses, and interpretations. I still firmly believe that commenting forums will indeed become fundamental to practicing and publishing neuroscience in the future, and not just a fad, as it currently looks. So I say to you, readers of Action Potential, let’s not miss out on an excellent chance to be early-adopters, with neuroscientists taking the lead on internet-based dialogue and collaborative thought amongst the biological science disciplines. With so many branches of neuroscience having deep roots in physics, computational methodology, programming and bioinformatics (all fields that are seemingly well-stitched into the fabric of Web 2.0 technology), it is an opportunity that would be shameful to miss.

Comments

  1. Report this comment

    Michael said:

    Good topic. I think there are a number of reasons as to why “Web 2.0” has not played much of a role in discussing Neuroscience research. In no particular order

    1) Inefficiency: If I want to know why somebody used buffer x for their biochemistry experiments, or why they didn’t do control experiment xyz I email the authors, or use the phone if I know them. Why post it in the comment section of their paper, and wait for 5 weeks until they bother to check? And why does everybody else need to know about it?

    2) Lack of dialogue: Commenting forums are poor venues for true dialog. If you analyze the comment sections for the more popular entries, either here or in other venues (e.g. the New York Times) there rarely is a true back-and –forth of ideas. All too often it’s 50 comments trying to be as witty as possible, with few people attempting to follow and respond to what others have written. There are rare cases where a small number of likeminded people post thoughtful comments at just the right rate to allow for a meaningful discussion. But without some type of moderation that will remain the exception. The dynamics of a true group discussion – be it at a Journal Club or at your poster – are often taken for granted, but can’t be easily replicated online.

    3) Who is commenting: It’s fine to have a democracy of opinions: In science, I don’t care so much about it. The people I want to hear/read often choose not to comment whereas others who have nothing to say keep posting.

    4) The Fear Factor: This one is obvious which is why scientists have lab meetings or Journal Clubs. It’s also why attempts to discuss papers online (i.e. the Journal Club here and at Journal of Neuroscience) haven’t quite lived up to expectation. Ideally, we want to be honest in our opinion of a paper, but we are also human and don’t want to suffer the consequences of bruising the ego of a potential reviewer or search committee member. Staying anonymous is not the solution, since that makes it difficult for everybody else to properly evaluate the comment. After all, it does matter who is doing the criticizing.

    5) Speed: Even the liveliest online discussion of a paper will drag on over hours or days. If a paper grabs my attention I will discuss at a lab meeting or Journal Club and over the course of one hour we will have thoroughly dissected it. Our attention span only lasts so long. Again, the online dynamics of who logs in at what time don’t allow for a true discourse that leads to some sort of resolution.

  2. Report this comment

    Anna said:

    Noah, it seems as though we are of like mind on this subject! It is a shame that the vast potential of current web2.0 applications for science is not being fully realized. My only hope is that time and increasing exposure of these applications will change the current status quo of inactivity.

    I also would like to respond to a couple of Michael’s comments. Michael, your first point about the inefficiency of communication in forums – it’s a shame that it takes the authors weeks to reply to comments, but I am not sure it’s because they are not checking for new comments on their articles. I would imagine that most applications notify authors of recent comments. It could be the authors’ lack of initiative in replying to comments in a timely fashion, which is a whole other unpleasant issue. It may be easier to email the authors directly, but if you have a question about buffer X, it is quite likely that someone else out there does as well. Having the question answered in a public forum could help other readers with the same confusions/questions that you have.

    As far as speed of discussion is concerned – yes, I would imagine that it takes longer to get all of your questions and points addressed in an online discussion, but you benefit from collecting the opinions of an entire planet of readers, scientists, and experts. While I greatly respect the people in my lab and department, they are not the end all and be all of scientific discourse. There is far more expertise to be accessed over the web than down the hall. And I completely agree with you on the Fear Factor aspect of commenting. I just wish I could think of a viable solution!

  3. Report this comment

    Bora Zivkovic said:

    Responding to this is not entirely straightforward because everyone working in this field is still figuring out the answers and the way scientists interact with research is evolving and changing daily.

    What I can tell you is that there is a gentle uptick in commenting on PLoS ONE, with which I am quite pleased. As the journal is only one year old (and so is the commenting experiment) it will probably take some more time until we reach a tipping point and commenting by scientists becomes a ‘natural’ thing to do. So far, the reasons people give for not commenting are probably already familiar to you: lack of time, lack of familiarity with a topic, fears for one career, lack of a motivating factor (i.e., ‘how do I put this on my CV?’).

    As everywhere online, e.g., on blogs, some articles do not get any commentary, a few receive one or two, and only occasionally an entire thread develops. On which papers this happens is hard to predict and the usual suspects i.e., folks who feel confident in their writing ability and are naturally outgoing communicators tend to be involved – I’d like to see more people gain in confidence – the question is how do you encourage that when the topic is rarely taught.

    So far, those attracting comments appear to be the papers on topics in which we already have quite a large community within PLoS, e.g., clinical trials, ecosystem ecology, anthropology and visual processing in the brain – these are probably contingent on what papers we published early in our history (which led to submissions of a number of other papers in the same field) and the picture will probably change over time.

    Now that five out of our seven journals have moved onto TOPAZ, it will be interesting to see what level of participation they get in comparison to PLoS ONE – being narrower in scope and run by and for their communities, they are likely to attract people who know each other (and the related topics) much better and may be more likely to move their conference hallway discussions online. We’ll have to wait and see – this is still very much early in the game.

  4. Report this comment

    Noah Gray said:

    I have to admit that although most of what Michael had to say (all great points) I had previously considered, the “speed” factor was completely off my radar. It is indeed a weakness of these discussion forums.

    Bringing non-scientists to blogs written by scientists is a good thing. There is even a good chance that those readers outnumber scientist readers. However, the “fear factor” may even be greater for non-scientist readers, believing that without an established science background, that their comments may sound rudimentary. On the contrary, I have often received some of the best and most challenging “practical” questions regarding my work from non-scientists. In addition, with specialization in science being the norm, most people (scientist or not) are going to be non-experts on a particular topic, so hopefully we can diminish the fear factor through the comfort of being surrounded by other novices. Then everybody can learn together.

  5. Report this comment

    David Crotty said:

    Good thoughts on the question of anonymity. It’s a tough call, in that it’s good as it allows you to say things publicly that might put your career in jeopardy, so it should lead to more commmenting. It’s bad in that it takes away any incentive that might be given to commenting in that no one will ever know you were the clever person who made that observation.

    I thought Michael’s comment’s above were excellent. The speed question is definitely a driving factor for the journal I run, which is all about methods and protocols. Are you really going to postpone your experiments for a few weeks hoping someone will come along and offer advice, or are you going to seek out an expert and ask directly?

  6. Report this comment

    Paolo Avesani said:

    I have a controversial opinion on Web2.0. On one hand I believe it may be a source of useful information/knowledge, on the other hand the ratio signal/noise risks to be too low.

    From the point of view of neuroscience, web2.0 may become the space where can emerge all the lesson’s learned that usually don’t find space in the journal. It is well known that we learn from our error. Journal are a collection of success story “only”. It is very difficult to read an article where someone claims that a certain idea doesn’t work.

  7. Report this comment

    Maxine said:

    Michael’s list is a good one— the “who is commenting” is borne out by our experience at Nature, where for about six months now we’ve had online commenting on our news stories. Some of the people who have been sending us several “ranting” emails a week for years are among the most energetic commenters.

    I wanted to add two points: one has been discussed before, and it is the issue of accreditation. It applies to peer-review more generally than post-publication/public review, which is in effect a subset of the review process and is being discussed here. If we could come up with a way to provide accreditation for “good” reviews (and post-publication comments), that would help.

    My other point is a small one: I take the point about gmail addresses but many academic institutions in countries such as India are using gmail accounts instead of institutional email accounts, for a variety of practical reasons. Therefore we need systems that can distinguish between “on the fly” accounts by the frivolous and the “recreationally outraged” community that you see in many blog commenting threads, even scientific ones, and scientists using these accounts as their main professional account. Moderators of comment threads need to be generally aware of this situation.

  8. Report this comment

    DSK Samways said:

    I also generally concur with Michael’s comments, particularly #4 and #5.

    Maxine said, “I take the point about gmail addresses but many academic institutions in countries such as India are using gmail accounts instead of institutional email accounts”

    Well, as a scientist from the developing world (state of Missouri), I’ve also given up on my institute’s wretched email service in favour of gmail. Actually, many people choose to sign up to forums and blogs with “on the fly” accounts simply to protect their professional accounts from being spammed to death. Privacy not really being much of an option on the internet.

  9. Report this comment

    Hans Ricke said:

    Some points are good. Remembering what happened on Psyche-D of now ASSC before it was closed down, I think though there would be many ways to improve and thus take advantage of web 2.0

    What matters is a good forum software. The one used on Psyche was awful. The one on network.nature.com is not much better. A much better software is used on Richard Dawkins website. Some of the downsides mentioned here, although correct, can be avoided by distributing posting rights and having good moderators.

    Good software and good moderation should cope with points 1 and 2, some of 3 as well. The fear factor as difficult to address. 5 is interesting, because those who already have meetings and a club will not need an online forum that much. But what of those of us who do not? Moreover, what makes web 2.0 so powerful is that it breaks the barriers of continents!

    So my suggestion to the editors here is: keep encouraging the public debates and enhance them. Neuroscience as well as cognitive sciences are still thriving and a world wide science should have advantages over the old-fashioned snail ways.

  10. Report this comment

    mr. gunn said:

    Absolutely, the quality of the “user experience” matters immensely. If you make it difficult(requiring logins, captchas, etc) you’re going to slow people down and turn some off completely. OpenID and the like can potentially be a solution to this, but it’s gonna take some time for that.

    The main thing I think is lacking is that services/sites expect user interaction coming to them, but they don’t do any reaching out to the user in a useful way. Of course, it would be annoying as hell if every site upon which I’ve ever commented bugged me to come back and leave more comments, so don’t even think I’m suggesting something so obnoxious, but if a site like PLOS emailed me to say that there was a answer to my question on a paper, or told me that a follow-up or rebuttal had been posted, or maybe even sent me a list of new papers similar to ones on which I’ve previously commented, that would not only be OK with me, but gladly welcomed. I don’t want to have to manually check back to see if there’s something interesting, and I don’t have time to do that on the hundreds of sites upon which I’ve at some point, commented, so why can’t the site do a little something to reward me for my efforts in leaving a comment?

    I bet it would increase overall user participation, too.

  11. Report this comment

    mr. gunn said:

    In case you missed it, yes, the commenting turn-offs of entering security codes, jumping through flaming hoops, waiting to see if your comment is deemed worthy of posting was directed at the blogs.nature.com commenting system. Handle spam behind the scenes, please.

  12. Report this comment

    Hilary said:

    Hans and mr. gunn—your points about user experience and usability are well taken. As the product manager for Nature Precedings, I wanted to say that we always appreciate suggestions and feedback, particularly on usability and feature enhancements. In response to user feedback, we’ve recently added a comment notification service that authors and commenters can opt-in to in order to receive notification when someone posts a comment on their document, or responds to one of their comments.

    Along those lines, one of my (usability-related) pet peeves is too small comment boxes, which make it difficult to write more than a short paragraph (erhm, I’m not going to name names, but… cough, cough). Another pet peeve is answering a captcha correctly in order to preview one’s comment, and then being immediately presented with yet another captcha for further previewing or posting—if you weren’t a bot on the previous page, why would you be a bot now?

    Bora—I think you make a good point about confidence. For a long time, I was hesitant to leave comments on blogs or post to mailing lists for fear of looking stupid (or at least naive) to someone I’m hoping to impress who runs my name through Google several years down the line . At some point, I realized that most bloggers/posters will say silly things at some point and that if one writes enough it will create a large enough pool of comments/posts such that sampling will turn up some comments/posts that are silly and others that are (hopefully) brilliant.

  13. Report this comment

    Michael said:

    I want to echo a comment by Paolo (see above) and suggest that web2.0 would be a suitable venue to publish important data that would otherwise not see the light of day. For every paper published there is another one lying on the shelf (or sitting on a drive). The critical thing is to find a way to organize the data being uploaded, to have only a minimal (but some type of review) and to index the data somehow so they can be referred to in a paper (or on a CV). I think the last point is important, there has to be some incentive for people to do this.

  14. Report this comment

    David Olmsted said:

    This all boils down to the purpose or goal of the community. If the goal is a specific and time limited problem such as “how does this thing work” or "how to we design this machine to accomplish x,y,z) people will participate with their comments because their ideas will be recognized and the ideas of the community will help them with their own related sub-goals.

    In contrast to that most science is not time limited and occurs with no obvious goal in mind. The goal of most scientists is to get funded which means being recognized by various funding agencies via publications in established journals.

    The only way science would adapt web 2.0 would be the result of a radical restructuring of how science is funded. Instead of funding by ever narrowing specialist disciplines funding would have to be geared towards goals in which ideas and reputation in a forum dedicated to that goal and moderated by the funding agency is just as important as publications.

  15. Report this comment

    Action Potential said:

    Getting into and out of character

    A great discussion over at Nature Network inspired me to initiate a similar conversation here at Action Potential. Corie Lok asked the question “What is fair play in the blogo/commentosphere?” A fair question indeed. The responses have produced some in…

  16. Report this comment

    Londynie said:

    Very interesting post, I think is helpful for many people, especially for me. Thanks for excellent article 🙂

  17. Report this comment

    strony said:

    thanks for really good article. good work 🙂