News blog

Scientists urge broad sampling for US children’s study

A panel of advisers to the US National Children’s Study met at a crucial juncture today, as leaders of the study prepare to choose a sampling strategy for the multibillion-dollar effort to document influences on the health of 100,000 children from before birth to age 21.

Both the panel’s members and the observers who spoke at the day-long meeting in Bethesda, Maryland, argued nearly unanimously for what epidemiologists call a “national probability sample” — a scientifically chosen, geographically distributed group of children who reflect the diversity of the US population and from whom findings can therefore be generalized to all US children.

The study’s leaders at the National Institutes of Health (NIH), which oversees and funds the study, told Congress in this budget document in February that they would be dropping this approach in favour of a “well-described cohort followed longitudinally” that would not be representative of the entire US population.

Study director Steven Hirschfeld told participants at the outset of today’s meeting that he would be convening federal statisticians in a webcast meeting on 29 May, as well as meeting with the study’s principal investigators on Thursday, before developing a final sampling design in consultation with Francis Collins, the NIH director, and Alan Guttmacher, who directs the NIH child-health institute, where the study is housed.

Today’s meeting, Guttmacher added, “gives us extremely important thought and input about what the sampling strategy should look like.”

Included in that input was an 18-page paper circulated at the meeting, signed off by 31 principal investigators representing 28 of the study’s 40 pilot sites, which have been coming online since 2009. Two other principal investigators who attended the meeting said that they agreed with the document’s scientific approach, but had not signed it because of concerns unrelated to its science.

Entitled “A Cost-Effective and Feasible Design for the National Children’s Study Recommendations from the Field“, the document argues strongly for a probability sampling approach and for retention of the 105 counties in 43 US states that have already been designated as study locations. It says that both can be achieved by recruiting from health-care providers’ offices — and all within budget.  (The study, first authorized by Congress in 2000, is budgeted to cost about US$3 billion, but there have been concerns about cost overruns. To date, it has spent more than $750 million, the document notes.)

It was not only the principal investigators at today’s meeting who shied away from giving up on a national probability sample that would be generalizable to the US population.

“I’m greatly concerned about a convenience sample,” said Edward Sondik, an ex officio member of the advisory committee who is director of the National Center for Health Statistics, part of the US Centers for Disease Control and Prevention in Atlanta, Georgia. “About whether it can be generalized, and how — whether the science community would accept a convenience sample in a study of this magnitude.”

Joseph Konstan, a professor of computer science and engineering at the University of Minnesota in Minneapolis, described the conundrum that is before the study’s managers and advisers succinctly: they are looking, he said, for a study that is “cheap, representative, large and deep. I’m starting with the assumption that we can’t get more money, and so we are trading off among the other three.”

Asked whether cost concerns will force the study’s leaders to abandon a sample including all 105 designated US counties, regardless of what sample design they adopt, Hirschfeld said: “We hope that we’re guided by science and data and that the final determinant isn’t finances… With that principle, our response remains that we don’t know yet what locations will be in the sampling frame or not… What we have to convey to all the locations is: ‘Stay tuned.’ ”

Hirschfeld said at the meeting that a final design may be proposed as soon as the advisory committee’s next meeting, which is in July.

The National Children’s Study has been buffeted by controversy since its leaders proposed dropping national probability sampling in February.

Correction: An earlier version of this blog incorrectly stated that a paper circulated at the meeting was signed off by principal investigators at 31 rather than 28 pilot sites.


  1. Report this comment

    dennis long said:

    Any decision to abandon the original 105 county probability sampling scheme will indeed undermine scientific rigor of the NCS, and will effectively trash hundreds of million of dollars of existing physical infrastructure, intellectual capital, and community-level good will established over the course of a decade by the Vanguard institutions. While statements from the NCS Program Office (PO) may seem ‘contradictory’, or ’perplexing’, especially in view of the FAC comments/recommendations describe above, in reality there is a method to the madness. To understand it you have to realize that the PO some time ago actually DECLARED WAR AGAINST THE NCS, AS IT WAS ORIGINALLY ENVISIONED BY CONGRESS AND INDEED BY THE PREVIOS NIH DIRECTOR. In short the NCS is being hi-jacked, in order be transformed into a sibling, if not a replica of, the UK Bio-bank study.
    The first casualty of all wars is of course truth, and as in all wars decision-makers operate under the assumption that they are unequivocally correct, that God is on their side, and that a great deal of collateral damage (in this case, most if not all of the previous investment in infrastructure ) is quite acceptable in order to achieve the larger, final, unquestionably righteous and exceedingly important goal .
    I will leave it to the reader and to the reporter to inform themselves about the nature of the UK Biobank study, and to read the various, relatively recent published articles by Francis Collins, Alan Guttmacher, and/or Steven Hirschfeld. In doing so it will become pretty clear that 2+2+2= 6 . There, among other things, the argument is made that : (a) in cohort studies like the UK Biobank, convenience rather than probability samples are a ‘better way’ , offering more advantages than disadvantages; (b) issues of response rates, retention and bias are relatively unimportant as long as the resulting sample sizes are large enough, and the objective is to look at relationships between genes and disease (as opposed, for example, to estimating incidence and prevalence of morbidity); and (3) the importance of specifying a-priori hypotheses about the relationships between genes, environmental exposure and morbidity has been given more merit than it deserves by ‘most’ scientists/epidemiologists — especially when cohort studies resembling the UK Biobank is the real objective
    In short, with regard to the issues of sampling frame/strategies, the case is in fact closed and the FAC committee meeting was pure and simple a cover-your-butt exercise designed to give the appearance that the PO is open to all ideas about the future course of the study, including the sample design. I can tell you as someone who has listened in on virtually of the conference calls among the NCS PI’s (but not as a PI myself) that they now understand all of this quite well. The document they have written and made available at the FAC meeting quite convincingly argues that the original 105 county-based probability sample does indeed represent a viable strategy to build upon, in terms of moving forward with the NCS and its original objectives. It did not represent , as the NCS PO would like everyone believe, an attempt by the current PI’s to continue lining their pockets and preserve their funding. That the game is over for them in that respect is clearly understood. The purpose of the document that they created and their testimonies at the FAC meeting was to preserve their credibility and their legacy – that is, to go on the record that they are opposed to the stealth, and ingeniously deceptive hi-jacking of the NCS ; that the justification for the recent ‘change in direction’ by the NCS PO is completely baseless on any scientific grounds; and that waste of money (approx $800 million so far) represented by the trashing of the existing Vanguard infrastructure is unconscionable. Perhaps more importantly it is also to state that the senseless and incomprehensible incompetence of the NCS PO reflected in the unending and senseless changes in study protocols and procedures that were forced upon the PI’s in the past portends a massive train-wreck, the responsibility for which they do not want to be associated. .The trainwreck will come in form of years of miscalculation and inevitable missteps that have been the trademarks of an arrogant NCS PO completely in over their heads, unwilling to accept criticism or suggestion from respected and accomplished scientists in the field, hampered by bureaucratic paralyses, and driven by political and ideological concerns – which all by their very nature will require a diligent conviction to avoid transparency.

Comments are closed.