The new dilemma of online peer review: too many places to post?

As online comments on newly published research become widespread, a new dilemma faces scientists wanting to enter the electronic fray: where to comment, and in what format for maximum impact?

That question faced Kenneth Lee, a researcher in regenerative medicine at the Chinese University of Hong Kong, when he wanted to post up his critique of controversial stem-cell research. Lee’s research group has, like many other scientists, tried and failed to replicate the work, published in Nature at the end of January. The studies are now under investigation, with some of their authors calling for retraction.

Lee had his pick of online fora. He could have posted on a closely watched stem-cell blog by researcher Paul Knoepfler at the University of California, Davis, which has been collecting tales of failed replications. He could have posted on PubPeer.com, a website where people can make anonymous comments about published papers, which has also seen large amounts of traffic discussing problems with images used in the studies. He could have posted on PubMed Commons, an initiative launched last October that allows scientists to comment on published abstracts on the PubMed website. He might have chosen any number of other venues — such as the news articles reporting on the controversy — or even his own website.

Instead, Lee picked ResearchGate, a social network that boasts more than 4 million signed-up researchers. And instead of just adding his comment linked to the publication’s page on the site, Lee posted up a structured mini-review, with sections for ‘methodology’, ‘analyses’, ‘references’, ‘findings’ and ‘conclusions’, and including his own images.

This did not happen by accident. ResearchGate’s managers had noticed that Lee was chattering about his replications on their network, and an employee invited him to be the first to try out their new post-publication review format. “I was very reluctant at first, but she said I keep the copyrights, so I reluctantly agreed,” Lee says. “This is how everything came together. I think it is just fate.”

ResearchGate is calling its structured feedback format Open Review, and the co-founder of the site, Ijad Madisch, says that it is a feature he has long wanted to introduce.

“It looks interesting, and I am a supporter of innovative approaches to facilitate discussions among scientists in real time,” says microbiologist Ferric Fang of the University of Washington in Seattle. “A nice thing about the more structured format is that it encourages reviewers to be more systematic and to support their critiques. Short comments are OK but it is easier to make reckless statements in the absence of structure.” Fang adds that in this particular case, “I don’t expect the open review to have much impact on the paper since questions about its validity have already been raised”.

Asked why researchers should post their reviews on ResearchGate — as opposed to any other website — Madisch points out that his site has a community of verified scientists. “The content is free — anyone can read that from outside — but to contribute, you need to be affiliated with an institution that does research, so the quality is high,” he says. “I think Kenneth decided to publish on ResearchGate because he is part of an engaged community there. He wanted to get his replication out fast in order to warn others, and to get feedback on his work — rather than, say, write a letter to the editor, which can come six months after an article is published, and may be completely detached from the study itself. If there is one central place where people go, post-publication peer review becomes more efficient for everyone,” he says.

Will a few hubs such as ResearchGate or Pubpeer.com dominate post-publication peer review? Or will online comments look more like a scattered hodgepodge of reviews, comments and discussions across websites unlinked to original publications? And if so, can search functions tie the thicket together? To these questions, Madisch has a simple answer: “I don’t know where this will end, but I do know it will be really big.”

Lee says he would still like to publish his results in a journal, so that his students get the credit they deserve for their efforts. He says he doesn’t know whether his work posted on ResearchGate could be considered a citable object in itself. “But it has already been cited on the Wall Street Journal, BBC and Boston Globe, so the impact is really far reaching,” he notes.  “The most important thing is that the finding is fairly and accurately reported so that other researchers can decide whether to use their valuable resources to continue pursuing the study.”

Online post-publication peer review, in the fuller sense that Lee has performed it, is unlikely to be common, says Fang. “Given the amount of time it takes to read and carefully review a paper, I suspect that the papers selected for discussion are going to be limited to very high-profile work about which readers have concerns. After all, there are something like a million new papers published each year and the average scientist reads only about 20–25 papers each month,” he says.

Elizabeth Iorns, chief executive of Science Exchange, and an advocate for efforts to reproduce published scientific research, agrees. She points out a subtlety in the way scientists have rushed to replicate the findings. Rather than, like Lee, acting as post-publication reviewers seeking to check the paper, she says, researchers are instead trying to adopt the method for their own laboratories, and so often are not performing exact replications of the original work.

“What we have learned is that researchers don’t generally want to perform confirmatory replication studies of other researchers’ findings,” she says.

Cutting-edge UK science facilities going unused

Scientific instruments that cost millions of pounds are standing idle in the UK because of a lack of money to run them, a new parliamentary report has revealed. There is a “damaging disconnect” between funding to build new facilities and the funding to actually run them, it concluded. This includes spending nearly £40 million on high performance computers, without budgeting for the electricity they use.

The report’s authors, a cross-party group of politicians in the House of Lords, are demanding that the government review its funding for large scientific infrastructure sites after conducting a wide ranging inquiry into the subject.

The House of Lords Science and Technology Committee cites examples such as the ISIS site in Oxfordshire, which produces beams of neutrons and muons that researchers use to probe the properties of materials. It has recently been operating for 120 days per year, down from a previous level of 180 days, despite only marginal cost saving from such truncated hours.

In evidence to the inquiry John Womersley, chief executive of the Science and Technology Facilities Council which oversees much of the UK’s government-owned science infrastructure, said: “It has been difficult to invest in the routine maintenance and upkeep of existing facilities, because [government] ministers very naturally are interested in new initiatives and transformative change in entirely new projects.”

Another example cited by the committee are the high performance computers at the Hartree Centre near Manchester. These were set up in 2012 with £37.5 million of government funding but the government has not provided enough money to run them, according to the committee’s report. Womersley told the committee that running the new computers at the Hartree Centre had come with “a significant electricity bill that we had not anticipated”.

In their report — released today — the committee says: “There is substantial evidence of a damaging disconnect between capital investment and the funding for operational costs.” They recommend that capital investment and operational funding should be “tied together in one sustainable package”.

The committee says it is broadly positive about the country’s science infrastructure, but its chair Lord Krebs notes that poor long-term planning places at risk the current excellent reputation of sites like ISIS. “The lack of a strategy and an investment plan risks the UK’s place at the forefront of scientific research,” he said in a statement.

Immunologist calls on university to disclose details of misconduct claims

An immunologist accused last year by the National University of Singapore (NUS) of “serious scientific misconduct” relating to 21 research papers says that he refutes the accusations and is calling on the university to make public its report into the matter.

“I categorically deny having been party to any fraudulent or scientific misconduct,” Alirio Melendez, who worked at NUS before joining the University of Glasgow and the University of Liverpool in the UK, wrote on a new website on 16 October, and at the site Retraction Watch, which has been tracking the case.

Melendez has maintained for two years that he is not to blame for the problems found in papers that he co-authored. Yet in December 2012, NUS said that a committee report had found fabrication, falsification or plagiarism associated with 21 papers, and no evidence indicating that other co-authors were involved in the misconduct. Or as Melendez sees it: “without showing any proof whatsoever that I am the guilty party for scientific fraud”.

Thirteen of those papers have now been retracted, and Melendez concedes that as corresponding author he is at fault for signing off the work without overseeing it adequately — a form of misconduct in itself. But in seven of the papers in which NUS found irregularities, he stated last week, he did not contribute data generation, analysis or any part of the manuscript writing.

So far, Melendez’s counterclaims have lacked convincing detail. That is, in part, because neither Melendez nor NUS would provide details of the papers, nor the committee report. Now, Melendez tells Nature that he will shortly post a “paper-by-paper response” on his website, but that it will be his “personal statement” on the papers, not the whole report. “Since this report is confidential I cannot publish it myself without NUS permission,” he claims.

But a spokesperson for NUS told Nature last December that it is “standard procedure” there to keep research-misconduct investigations confidential (although this is not the case at some other universities that have investigated research misconduct). Pressed, she repeated this week that internal inquiries were confidential and did not reply to a query about whether a redacted version of the report might be released.

There is also dispute about whether Melendez’s concerns have been given a fair confidential hearing by  NUS. The university says that it “conducted interviews with as many authors as possible” and that Melendez declined responses when a committee visited the United Kingdom in 2011 (which Melendez puts down to ill health).

Melendez says that last year, he did send two replies to the NUS investigation, but that they did not take these responses into consideration for their final report. The NUS spokesperson agrees, and says that Melendez’s responses in 2012 did not address the irregularities that NUS found and were also not sent in time for the deadlines that the university allowed, as guided by its research integrity code. Therefore, they “were not considered part of the record of the inquiry”. But Melendez says he was never made aware of this.

In addition, the NUS stated:

As a leading research university, NUS is committed to ensuring that all allegations of research misconduct are investigated thoroughly and fairly. This investigation involved a detailed examination of the papers concerned. The University had also liaised with the various institutions involved and conducted interviews with as many authors as possible. NUS offered every opportunity at each stage to Dr Melendez to respond to the Committee’s questions during the period of the investigation. In all, the 21 papers concerned were carefully examined with the journals involved. Since then, several retractions and corrections have been issued by the journals.

Student projects interrupted by US shutdown

It was to have been an exciting three-month research visit. Siddharth Hegde, a PhD student at the Max Planck Institute for Astronomy in Heidelberg, Germany, had lined up a trip to NASA’s Ames Research Center near San Francisco, California. Hegde is an astronomer who models atmospheres on extrasolar planets, and he was planning to study the optical properties of extremophiles — organisms that thrive in extreme environments — during his sojourn.

But then the US government shutdown hit. Hegde, who had carefully nurtured and grown his extremophiles, had to pack up his things and walk out of the Ames lab. Without someone there to oversee the cells and feed them regularly, the extremophile cultures are now dying. (The seed cultures, gathered from hostile environments such as the Atacama and Mojave deserts, remain safe in deep freeze.)

“To go from seed culture to see them grow takes some time,” says Hegde. “Some of these organisms were taking a long time to grow, and if all of these die then I have to start again and wait another month.”

Time is precious because Hegde, an Indian citizen, has a three-month US visa. When that expires at the end of November, he will have to go back to Germany and re-apply if he wants to return — even as other work there requires his attention. “It’s not a question of money right now,” he says. “It’s time. There is no substitute for time.”

For now Hegde is hunkered down at his uncle’s house in northern California, logging into his German research projects and trying to get some work done remotely on those. But he cannot help but think about the organisms languishing in the lab nearby. “The whole field of astrobiology is so hot right now,” he says. “If I don’t finish my work, then that work is lost. Someone else will do it and someone else will get the credit for it.”

Even so, Hegde knows it could be worse. Some of his friends, who are also visiting students at Ames, got kicked out of their NASA dormitory-style housing and struggled to find places to stay, in an unfamiliar city and on short notice, when the dorms closed down. And Hegde’s adviser at Ames, astrobiologist Lynn Rothschild, has found herself at loose ends trying to advise a student team she had mentored for months before the shutdown. Those students, from Stanford and Brown universities, work at the Ames centre in the summer developing a synthetic biology project.

The team competed last weekend in the International Genetically Engineered Machine (iGEM) contest in Toronto, Canada, but Rothschild was technically barred from communicating with them. They did, however, advance to the next round of the iGEM competition.

Court upholds need for export permits for risky flu research

The researcher who created mammalian-transmissible strains of the H5N1 avian flu virus, raising fears they could cause a pandemic, has failed in an attempt to overcome government restrictions on the publication of his papers (see Nature‘s mutant flu special).

Ron Fouchier, of the Erasmus Medical Center in Rotterdam, the Netherlands, lost a court case in which he tried to have publication of papers describing the production of such strains exempted from government export controls, which require researchers to obtain an export permit before publishing.

Fouchier says that he and his colleagues will meet with lawyers next week to decide whether to appeal against the 20 September ruling by the district court of the North Holland region of the Netherlands. They have six weeks in which to act. “We do not want to give up,” says Fouchier, adding that he cannot yet say whether he will pursue an appeal until the meeting with his lawyers.

The Dutch government last year told Fouchier he had to apply for an export permit before being able to publish a paper on his engineered strains in Science. It argued that 2009 European Union (EU) legislation on export controls require an export permit for ‘dual-use’ materials and information that can have both legitimate and malicious uses — including dangerous pathogens such as H5N1.

Fouchier had at first threatened to defy the government and publish the paper without applying for an export permit but reluctantly gave in after the government warned that he could face up to six years in prison, plus fines, for doing so. The government finally granted him a permit in April last year.

Fouchier said at the time that all the paper’s co-authors and the board of directors of their institutions had agreed to seek a permit only “under protest”. At the same time, they initiated a legal challenge to dispute the necessity of them having to apply for one. Fouchier argued that the research fell under two exceptions in EU export control law, including one for “basic scientific research” that was “not primarily directed towards a specific practical aim or objective”. The paper also fell under another exception for information that is in the public domain, he argued, on the grounds that the methods were already well known. Fouchier also argued that the obligation to apply for a permit hampered access to the results of scientific research.

The court noted that the law was intended to counter the proliferation of weapons of mass destruction, including biological weapons, and that H5N1 was among those dangerous organisms specified in the law — akin to the US Commerce Control List. It ruled that because the researchers showed how to convert an avian flu into a mammalian-transmissible virus, that was by definition a practical purpose going beyond basic research.

The court also ruled that the study went beyond known methods, by selecting and detailing the specific changes necessary to obtain mammalian-transmissible strains, so was therefore not already in the public domain. That a top journal such as Science was willing to publish the paper also indicated its novelty, it added.

All this pointed to there being a legal requirement to apply for an export permit so that proliferation risks could be assessed, said the court in its ruling. It argues that the export permit system does not have any impact on most research, and causes only a few weeks delay during review of the application. Any such drawback is outweighed by the need to counter any proliferation of biological weapons, it said. Fouchier says that while the export-control review process is eight weeks, it can be extended, and that it also offers officials the option of blocking publication.

Fouchier says that he finds the court’s arguments “weak”. “They are of the opinion that our work is not basic scientific research,” he says. “They claim our goal was to make a dangerous virus airborne. However, the goal was to increase fundamental understanding of airborne transmission of bird flu virus.”

He asserts that the court’s rejection of his argument (that the methods were already in the public domain because he had only added to them), would mean that only researchers following published procedures would benefit from this exception to the export control rules.

Fouchier was one of 22 flu researchers who in August argued for the need to perform similar experiments to genetically engineer versions of H7N9 — which emerged earlier this year in China — that are more transmissible and pathogenic in mammals (see ‘Handle with care‘). The ruling suggests that any European researchers wanting to publish such work would first need to obtain export permits.

In one sense, the court’s conclusions are a reaffirmation of existing legislation in the face of a challenge, rather than setting any new precedent. Fouchier argues, however, that it “provides an opportunity for European governments to control and censor scientific publications, which in my opinion is unacceptable”.

The court argued that, more broadly, decisions on what is and what is not research with potential consequences for international proliferation cannot be left to individual researchers without compromising states’ obligations under United Nations Resolution 1540, passed in 2004, which requires states to adopt legislation to counter the proliferation of nuclear, chemical and biological weapons. “Apparently, the court thinks that export control officers are in a better position to judge scientific publications,” says Fouchier.

Nobel prize guessing game begins

‘Tis the season: with just about two weeks to go until the winners of the 2013 Nobel prizes are announced, speculation about who will win and who will get snubbed is once again brewing. Thomson Reuters — the firm that maintains the Journal Citation Index — released its predictions today based on, of course, citations.

Anyone who takes issue with the disproportionate attention paid to Nobel prizes or citation counts might want to look away now.

Among the noteworthy names on Thomson Reuters’ list are François Englert and Peter Higgs for their prediction of the Brout–Englert–Higgs particle in physics. In medicine, the list is topped by Adrian Bird, Howard Cedar and Aharon Razin for their discoveries in DNA methylation and gene expression. Chemists on the list include M. G. Finn, Valery Fokin and Barry Sharpless for the development of modular click chemistry.

Thomson Reuters’ full list below names multiple teams tipped each for prize. Since this forecast first began in 2002, 27 of those researchers have eventually gone on to win a Nobel.

If this all seems worthy of some (fake) money, the Nobel Exchange run by the science magazine Nautilus is open for speculation.

Continue reading

UK animal research increases again

UK researchers performed a greater number of experiments involving animals in 2012 compared to the year before, with hundreds of thousands of additional experiments taking the total number of animal procedures to 4.11 million.

Data released today by the UK Home Office show an overall 8% rise — or 317,200 more procedures — that was mainly down to an uptick in breeding of genetically modified (GM) animals and mutant strains of mice.

Since 1986 the United Kingdom has collected data on animal ‘procedures’ rather than actual number of animals used. Breeding a GM animal and then conducting two experiments could therefore be counted three times in these statistics. The Home Office report says the number of procedures “generally corresponds” to the number of animals.

After dropping off from record-high numbers in the 1970s, animal research numbers in the country started increasing again around 2000 (see graph). They have continued to rise year on year since then — a growth generally put down to the increasing utility of GM animals in biology. In 1995 only 12% of procedures were breeding of GM or other ‘harmful mutant’ (HM) animals. In 2012 that number was up to 48%.

In total in 2012, 4.03 million animals were used in procedures for the first time. Mice accounted for 76% of this total.

graph animal 2

There was also an increase in the use of primates. However, the group Understanding Animal Research  — which campaigns in support of animal research — says: “this is partly due to a large drop in primate procedures in 2011 and the figure remains 36% lower than in 2010”.

‘Liberated’ mice from Italian lab now housed in poor conditions

Mice removed by protestors from a Milan lab arw

Mice removed by protestors from a Milan lab appear to be housed in cramped cages.

Two months after animal-rights activists broke into an animal facility at the University of Milan and removed hundreds of animals, photographs of many of the mice have appeared on the Facebook page of one of the protestors’ supporters who uses the pen name Jooleea Carleenee.

The raid took place on 20 April. Researchers at the university said that they lost years of their work along with the animals, most of which were genetically modified mice serving as models for disease. They said that they did not expect mutants that were particularly delicate, or immunosuppressed ‘nude’ mice, to survive outside controlled laboratory conditions.

Carleenee says that she posted the pictures to show that the animals were still alive. But the images of the overcrowded and uncontrolled conditions in which the mice appear to have been kept in her home have fuelled a new row, with scientists posting angry comments, complaining of cruelty.

Daria Giovannoni, president of the pro-science lobby group Pro-Test Italia, says: “If these photos show the actual conditions of the stolen mice, we’re seriously concerned about their well-being and health: we don’t think that these animals are faring better now than when they were in the laboratory.”

The raid on 20 April spurred the nascent Pro-Test Italia — modelled on UK and US Pro-Test organizations — to action. It arranged a series of demonstrations by scientists in defence of their work on animals.

UPDATE: 28 June, 2013 We have been contacted by Jooleea Carleenee who requests that we report that the mice are now being kept in humane conditions and that the pictures showed only a temporary situation when the mice arrived with her.

Strapped Texas cancer centre built pricey office suite for president’s wife — UPDATED

The financially stressed MD Anderson Cancer Center in Houston, Texas, seems to have invested at least US$1.5 million in capital funds in a new ‘corporate’ office suite that will be home to Lynda Chin, the wife of MD Anderson president Ronald DePinho.  The revelations come in an article published today in The Cancer Letter.

Cancerletterpg3

{credit}Cancer Letter{/credit}

The Letter used the Texas Public Information Act to obtain 680 pages of documents that describe the project as “Dr. Chin Office Renovation”.  However, the article argues:

A renovation it was not. The 25,000-square-foot suite, much of it south-facing, is new, located on the sixth floor of the just-constructed South Campus Research Building III.

Chin, who moved to MD Anderson when DePinho became the cancer centre’s president in 2011, is the scientific director of the fledgling Institute for Applied Cancer Science (IACS), a drug-discovery centre, and chair of the cancer centre’s department of genomic medicine.  The office was built to house the genomic medicine department and the IACS, which aims to enlist drug companies in promising collaborations. For instance, the IACS in December announced a partnership with drug-maker GlaxoSmithKline, with the goal of producing cancer-fighting antibodies.

Among the itemized expenses reported in The Cancer Letter article are nearly $28,000 for settees, lounge chairs and occasional tables for the IACS, and about $210,000 for translucent walls in the 2,323-square-metre space. The spending on the walls required a variance, or special permission, from the University of Texas system’s executive vice-chancellor for health affairs, Kenneth Shine. (An architect’s rendering of some of the translucent panels is pictured above.)

In a statement to Nature, MD Anderson defended its actions, noting that the IACS has raised $15 million in donations:

The renovations of space for the Institute for Applied Cancer Science and Department of Genomic Medicine — both new entities for MD Anderson — transformed a traditional academic office suite to a work environment and meeting area for a science/business enterprise, a concept new not only to MD Anderson, but most of academic medicine… The existing space was not configured to support this new concept.

The “redesigned” space, it added, would “create an open environment of communication, provide an appropriate meeting space with high-level industry decision makers and support a new suite in computational biology”.

This is not the first time that Chin has landed in the public spotlight since she arrived at the huge, high-profile centre.  Last year, there was an outcry after an IACS team headed by Chin was awarded an $18-million grant from the Cancer Prevention and Research Institute of Texas (CPRIT) that bypassed scientific peer review.

The 17 May issue of The Cancer Letter (available only to subscribers) reported that DePinho announced austerity measures — such as suspension of merit raises and slowing of hiring — in an e-mail to MD Anderson employees on 15 May. It read, in part:

For most of Fiscal Year 2013 (FY13), our operating expense has exceeded our operating revenue — meaning that we’ve spent more than we’ve made from providing our patient care services. What we’re facing today is much like what you’d face with your own checkbook if you spent more than you were paid each month for several months.

Updated with new comments from MD Anderson.

Gairdner Foundation honours two of three prizewinners for hepatitis C research

As they sipped an Ontario sauvignon blanc and munched on Canadian lobster and tuna tartar, guests gathered last night at the Canadian Embassy in Washington, DC for a celebration of two of the prestigious Canada Gairdner Awards. But one key guest was missing.

In March, for the first time since the first awards were made in 1959, a recipient declined the $100,000 prize. That was Michael Houghton, a virologist at the University of Alberta who is a co-discoverer of the hepatitis C virus. Houghton’s co-winners, Harvey Alter of the National Institutes of Health and Daniel Bradley, formerly of the US Centers for Disease Control and Prevention, were  honored at the embassy last night, in a sixth-floor reception room with nearly-unparalleled views of the US Capitol.

Introducing them, John Dirks, president and scientific director of the Gairdner Foundation, noted wryly, “Michael chose for the first time in our history to decline this award. I won’t go into that. All I can say is that it did give us a lot of publicity. It’s the old story: All news is good news.”

After declining the award in March, Houghton, who in 2000 shared a Lasker award with Alter for his hepatitis C accomplishments, told the Globe and Mail that he “agonized” after accepting the Lasker, because two of his collaborators at the biotechnology company Chiron were not similarly recognized. Houghton worked closely with Qui-Lim Choo and George Kuo to identify and clone the hepatitis C virus, and he determined that he would not in future accept an award that did not also honour them.

The arduous, multi-year quest to identify hepatitis C, which can be transmitted by blood transfusions, laid the foundation not only for a huge improvement in the safety of the blood supply, but for promising hepatitis C therapies that are now close to the market, as Nature reports this week.

The Gairdner Foundation, like the Nobel Foundation, limits to three the number of scientists who can share an award — a limit that, some argue, is outdated in an age of team science.

Alter, when he stepped to the podium last night, implied as much in a speech that likened the work of research to that of making a movie. Beyond the award winners in the limelight at the Academy Awards, he noted, are “many many more supporting actors who played such critical roles.” (The director of his movie, he said, was undoubtedly his wife, Diane Dowling, even if she never set foot in the lab.)

For a cogent, if opinionated, description on what each of the five men contributed in the quest to track down hepatitis C and screen it in the blood supply, see this Scientific Clearinghouse blog.