The impact of the Impact Factor

No scientist would be telling the truth, if at some point or other they checked the impact factor (IF) of journals before submitting an article, and simultaneously declaring the IF being a flawed system. A recent article in BMJ, highlights another aspect of the IF; differences in levels of IF of published studies between industry funded versus government funded research. Essentially, publication in “prestigious journals” was found to be associated with industry funding in studies assessing the effects of influenza vaccines. In other words, industry funded research ended up in journals with a higher impact factor.

However, ask any scientist how this system is calculated and how it is regulated and you be met with largely blank stares. So what is the IF and what does it really mean?

The idea of the IF was first proposed in 1955 by Eugene Garfield as a quantitative means to assess the impact of a particular article in the scientific literature on the scientific community as a whole, or in other the IF measures the popularity of a paper. This system however, was then adapted to a means of scoring a scientific journal as a whole. The IF for a journal is calculated, (for example for 2007) by:

IF =

Citations in 2007 of articles published in 2005-2006

——————————————————————————

Citable items published in 2005-2006

On this basis, here are the top 10 journals for 2007:

1 CA-A Cancer Journal for Clinicians 69.026

2 New England Journal of Medicine 52.589

3 Annual Review of Immunology 47.981

4 Reviews of Modern Physics 38.403

5 Nature Reviews Molecular Cell Biology 31.921

6 Annual Review of Biochemistry 31.190

7 Cell 29.887

8 Physiological Reviews 29.600

9 Nature Reviews Cancer 29.190

10 Nature 28.751

A quick look at this list and 7 of the top ten journals publish review articles only. So therefore how can review articles without any original scientific content have a higher impact than original scientific articles in Nature, Science (14th in the list), PNAS and JBC (both not even in top 100)?

Alone this suggests there are some limitations to the IF. It can be deduced from this that a review journal with a small number of articles per issue is more likely to gain a higher citation rate than that of a journal publishing numerous primary science articles of a diverse nature.

There are numerous other criticisms of the IF system including:

• Differences in citation rates between scientific fields

• Inclusion of letters and commentaries in the citation count, but not in the count of citable items

• The bias of self-citations by authors

• The date of the publication in the year

• Skewed citation data from “landmark” papers (it is estimated that ~20% of citations in a journal may account for 80% of citations)

• Short time-frame of calculating the citation level of publications

• Citations of flawed studies

As this system is used in some cases to assess an individual for funding, promotions and research institutes as a whole, what alternatives are out there? Other models of assessment include the Hirsh h-index (an assessment of the individual scientist), SCImago Journal Rank, the Prestige factor and the PageRank. In the modern information driven age, the use of the Google PageRank system is an interesting approach which evaluates not only the number of hyperlinks (popularity), but the quality of the referring sites (prestige). In their article on this, Bollen and colleagues (Scientometrics, 2006) demonstrated by PageRank that a very different “top 10” could be generated from that of the IF list. Taking this one step further they plotted the impact factor score against PageRank (the Y-Factor) and came up with another “top ten” journal list:

1. Nature

2. Science

3. New Engl J Med

4. Cell

5. PNAS

6. J Biol Chem

7. JAMA

8. Lancet

9. Nat Genet

10. Phys Rev Lett

This combined approach seems to result in a list most similar to the perceived top scientific journals. Although devising an ideal system to measure the quality of an individual scientist’s work is difficult, this seems like a reasonable approach by combing both the IF and PageRank. However, one major flaw in the IF system is the fact that the IF data is generated and owned by a private company (Thompson Scientific). In the same realms of making databases of genomes freely available to all, a similar approach is needed for assessing publication / journal / individual scientist status. Perhaps it is up to scientist themselves to solve this problem and form an international committee / set of standards. Although the suggestion of Dr Garfield himself (Chairman Emeritus, Thompson Scientific) about the use of the IF is that “there is nothing better and it has the advantage of already being in existence”, scientists should be aware that not only are there other measures of quality, but the limits of the impact factor.

Finally, another science sitcom…

Tonight I just caught the pilot episode on ABC of the new comedy series “Better off Ted”. Those of you who didn’t have the chance to see this may want to check out this link to catch some clips.

Essentially, Better off Ted is a comedy show about a soulless R&D company, where no idea is a bad idea if it means a profit. The main straight man character is the R&D head (Ted) and the laugh track actually comes from the scientists.

We start with a R&D meeting prompted by Ted’s boss who demands “We want to weaponize a pumpkin”. The roundtable discussion decides to mutate pumpkin fungus into human eating fungus; “spores in uniforms”.

The show is interspersed with faux adverts for the R&D company (Veridian Dynamics) on their products; a focus on food which boasts of producing “lemon flavored fish”, “chicken that lay 16 eggs per day” and “organic vegetable, chock full of anti-depressants”. In an obvious nod to the whole in vitro meat idea, the company decides to focus on growing meat in the lab. Unfortunately, apart from the emotional attachment of the lab scientists to the chunk of meat they name “blobby”, the company’s food taster classifies the lab meat as tasting like “despair.”

Whilst interpreted by the media as a workplace sitcom, this is definitely one for the scientists out there. Talk about connecting science to the masses…

The evolution of science journalism

A blog article discussing the arguments about how science is communicated to the masses and the importance of the internet in this process, is akin to preaching to the converted here on Nature Network. There has been much written recently (here, here and here) on how conventional scientific journalism is fading faster than the ink on last weeks newspapers, and what is replacing this. The rapid rise and popularity of blogs and other electronic formats of information are touted as replacing the print form, but many argue that these merely supplement rather than substitute conventional journalistic reporting.

I was therefore interested to hear what the opinions were of a range of scientific editors and writers at a recent panel discussion hosted by the Columbia Biological Society and Columbia Science Review. Although the aim of the meeting was to feature a panel discussion about careers in science journalism, the discussion evolved into a broader theme of the changing nature of communicating science and the role of science and its impact on society.

The panel consisted of five experienced writers / editorial including;

Ingrid Wickelgren – an editor and writer at Scientific American Mind who develops articles on psychology and neuroscience. Ingrid started as a freelance writer for numerous publications including Popular Science, Business Week and the New York Times, before joining Scientific American Mind.

Davide Castelvecchi – studied mathematics at the University of Rome and left academia to study science writing at the University of California at Santa Cruz before eventually becoming an editor at Scientific American

Claudia Dreifus – an Adjunct Assistant Professor of International and Public Affairs at Columbia University and a regular contributor to the Science section of the New York Times. She interviews and writes about prominent scientists in the column “A Conversation With…”.

George Musser – a editor and writer for Scientific American who developed journalism and writing as a hobby before accidentally getting into science writing though a desire to escape the cold winters of upstate New York.

Noah Gray – editor for Nature, neuroscientist, blogger and twitterer.

After an individual introduction (condensed above) about how each person found their way into science writing and editing, a panel and audience discussion commenced. What became apparent from the panel was that the outlook for traditional science journalism as a career move is bleak. Not new information for anyone following the recent news of layoffs and altogether abolishment of science departments in the mass media. The role of blogs and the internet in replacing the traditional print form drew some apprehension from a number of the panel, who thought the print form was still preferred through in part its role in supporting the source. Some championed the role of blogs are other online forums for science as a way to bridge the broader reporting of science to interest people to seek out and find more detailed online information.

The questions then led onto why this change is occurring and whether the state of scientific literacy in America may be a factor in this. A range of interesting points were raised by the panel to explain the lack of interest from large parts of the population in science. One such point is that the perception of science and scientist have changed, and not necessarily for the good. Blame the media for one was a point; the perception of the “mad scientist” and their Armageddon devices in movies and the dividing issue such as stem cells and evolution. One of the main problems perceived was that scientists are not getting their message out clearly and can be perceived as elitist. Claudia Dreifus pointed out that as children we are naturally curious about the world around us, but through our development into adults, most of us lose our interests in science.

So what has changed and what can we do about this? Isn’t it the role of “science” journalism to get the message out? Are we failing as scientists to help with this? Perhaps, the evolving role of the science journalist could be something altogether different if we are to keep the interest of a greater percentage of the general public, and explain what it is we do. It seem that for most we have already lost them from and interest or understanding of science during childhood, and no matter what is tried this will not change. Therefore, could the evolving role of the science journalist be to learn how to maintain and develop the curiosity of children in the world around us?

New York in the Hudson?

Although the an increasing number of Americans (41% ) believe the threat of climate change to be exaggerated, a range of recent reports make Hollywood-style imagery such as The Day After Tomorrow_, seem like a possible future event. One such recent headline illustrates this with the title "_Wall Street set for another big dive"

Highlighted this week by a study in Nature Geoscience, researchers from Florida State University, University of Illinois and the National Oceanic and Atmospheric Administration suggest the effects may be more serious than predictions in 2007 by the U.N.’s climate change panel. Earlier predictions, had suggested that low-lying areas such as Florida and Western Europe would be at most risk. However, the effects of climate change on the Atlantic meridional overturning circulation (AMOC) could result in higher sea levels and adverse weather effects across the North-East coast of the US. Using a range of climate change models, the authors predicted that weakened North Atlantic currents combined with global sea level rises could be a greater threat to cities of the North-East, including New York, Boston, Baltimore and Washington.

The study comes after release of a report by the New York City Panel on Climate Change (NPCC) last month. Formed by New York City Mayor Michael Bloomberg, the panel’s aims are to identify hazards to New York and its infrastructure and prepare for these eventualities. Even over a short time period as the next decade, a potential 3 degree Fahrenheit increase in temperature and 5 inch increase in sea level in the New York region are predicted. Also, these changes would include more frequent weather events such as longer and more frequent heat waves, heavy rainstorms and coastal flooding.

The panel concluded that by the end of the century New York City’s this could equate to an increase in temperature by 7.5 degree Fahrenheit, annual precipitation by 10 percent, and sea levels to rise by 23 to as much as 55 inches. If you’re interested how this increase in sea level would look like, here as an indicator.

The report is a sobering read of how over the last century the changes that have occurred in the New York area and the vast effects on the city these ongoing effects would have. The next steps with this report will be its use by the Climate Change Adaptation Task Force, which is a combination of City and federal agencies, and private companies that are involved in critical infrastructure in New York.

At the report’s release, Mayor Michael Bloomberg said “Climate change is real and could have serious consequences for New York if we don’t take action”. Perhaps Mr Major, swimming lessons could be tax deductible from next year in New York?

New York Biotech boom time?

I recently attended a meeting organized by the Science and Technology Ventures at Columbia University, on a presentation on the plans for the East River Science Park. Those of you not familiar with this, the idea is to build a state-of-the-art science park in the heart of New York City (a 3.7-acre complex off the FDR Drive between East 28th and East 30th streets). At the meeting, presentations were made by Alexandria Real Estate Equities; a company responsible for the development bioscience clusters in San Francisco, Cambridge Seattle and San Diego.

The project is a partnership between Alexandria, New York City ($13.4 million were pledged by Major Bloomberg at the outset) and other private and federal sources.

In Alexandria’s own words:

New York City has it all: location, science, capital and talent. Not only is the NYC metro region a thriving epicenter of the world economy, but the intellectual and cultural capacity and resources are unparalleled on a global scale. Each diverse component that NYC has to offer helps to support the growth and limitless potential of a bioscience and technology industry cluster.

The long-term aims are not only to provide space for New York-based bioscience businesses, but the creation of ~2,000 jobs for researchers, technicians and other bioscience professionals.

The Science Park itself is being modeled to have the benefits of a campus feeling with on site amenities including shared support services, conference facilities, on-site building support, and a host of additional specialty services.

The plan is to open the first part of this site in 2010. Those curious how the construction is coming along can see it here

Further to the posts recently by Caryn on career planning and Ian on alternative paths, this provides perhaps something else for New York scientists to think about in the not too distant future.

Podcasts versus Professors

Those out there wanting to spruce up their teaching statement with methods of technological advancement, take note of the results of a recent study published in Computers and Education.

In this study by McKinney, Dyck and Luber of the Department of Psychology at State University of New York, the authors tested whether students learn more from a traditional style lecture versus a podcast of the same lecture.

Using two groups (32 per group), students were then tested on the material of the lecture, with the podcasters averaging higher scores that those who actually attended the lecture.

The authors strongly point out that this should not mean the replacement of lecturers with podcasts, or an excuse to skip classes. The main advantage they point out gained by the podcasters, was when they re-listened to the lecture multiple times.

The study does raise some interesting points regarding the future of teaching. Could podcasts provide a means to provide lectures from the very best teachers for all, or just an excuse to play with your iPod or iPhone too much?

Those of you interested in checking out available online courses available through the iTunes U, click here

Healthy New York

Ask your average New York inhabitant what the New York City Department of Health and Mental Hygiene does, and most of us will think of officials closing down restaurants containing too many cockroaches and assorted critters.

In fact this department led by commissioner, Thomas R. Frieden (also known as the “health czar”), has spearheaded a number of city-wide health campaigns that have targeted us all. With a general campaign to lower the incidence of heart disease and cancer in the City, the Health Department are behind such campaigns as the smoking ban in all workplaces (yes this is the one affecting bars and restaurants) and the ban of trans fats in restaurant food.

The most recent measure by the Department of Health is the enforcement of chain restaurants to display calorie information on their menus or menu boards. Considering that New Yorkers get an estimated third or more of their daily intake of calories away from home, knowing what’s in your meals is imperative. Subway riders may be familiar with this campaign from the colorful posters screaming “2000 calories a day is all adults should eat”, with a picture of some 1200 calorie ridden cheese-oozing burger below the title. Taking a look at around the City at some of these new calorie displays, I was shocked at the actual calorific content of some menu items. Therefore, it was not surprising to learn that in a number of studies, where less-healthy food items were involved, it was found that people generally underestimated the calorie content of food by around 50%. The estimated effects of this campaign are if people are aware of the nutritional composition of their food, that less intake through this could mean over the next 5 years, ~150,000 fewer obese New Yorkers and a reduction of ~30,000 cases of diabetes.

So what’s the next campaign for the DOHMH? Salt. Don’t worry, this doesn’t mean a ban on salt, but the reduction of the amount of salt consumed (a so-called “Salt Assault”). The idea is to get makers of processed food to reduce the salt content by 40-50% over the next 10 years. Apparently in America, we consume on average twice the recommended amount of salt per day and 80% of this is in the food we buy. It is well-known from many epidemiological studies that reducing salt intake reduces blood pressure. An equally accepted fact is that lowering blood pressure also reduces the incidence of heart attacks and strokes. Therefore, it is generally accepted that reducing salt intake reduces risk of cardiovascular disease. Generally, but not everyone.

Like many scientific fields, there are those with opposing opinions. Highlighted by an Op-ed in the New York Times by Dr Michael Alderman, from the Department of Epidemiology & Population Health at the Albert Einstein College of Medicine, he suggests reducing salt intake may in fact be detrimental to health. In a number of studies by Dr Alderman, the complete opposite was seen between salt consumption and heart disease; an inverse relationship between salt and cause of death by cardiovascular disease. As you can imagine, this has caused some heated debate, including critiques of the design of these studies and the methods of analysis. So what’s the real message? It seems that both sides may be right depending on the individual. Recent studies have shown that genetic differences may cause people to react different to sodium, where some individuals are sensitive to high levels and vice versa.

So what do these people agree on? Reduce dietary intake as a whole and hence this could be a way of reducing salt intake. As for the campaign itself, it seems reducing salt levels in food will leave a bitter taste in the mouth for some.

Sweet memories

Thinking of having another chocolate bar tonight, lying on the couch and skipping the gym? Well, you should think again according to a recent study published in December’s issue of the Annals of Neurology.

The main findings from a team at Columbia University were that high blood sugar levels in the normal aging population are associated with a reduction in cognitive function. It has been known for some time that people with diabetes, where period of high blood sugar levels are the norm, have a higher risk of developing various forms of dementia, including Alzheimer’s disease. Therefore, the current study suggests its not just diabetics who should worry about their sugar levels.

Using high-resolution functional magnetic resonance imaging (MRI) the researchers studied the region of the brain known as the dentate gyrus. The dentate gyrus, for those of you not intimately familiar with the structure of the brain, is located in the hippocampus and is involved in neurogenesis (in simple terms the birth of neurons). Previous studies have implicated this region of the brain in normal age-related memory decline. Studying both human subjects with and without diabetes, they discovered that in this region of the brain, reduced activity correlated with higher blood sugar levels. The same pattern was also seen in mice and rhesus monkeys.

So, what’s the magic pill the authors suggest to counter the effects of sugar levels on age-related mental decline? For most people, the answer is simply taking regular exercise and eating the right diet. Therefore, the message here, forget those cravings for sweet delights if you want to hang onto your sweet memories.

New York, New Blog

When you think about New York, perhaps science would not be the first thought for most people. The self-appointed “World’s Capital” is home to over 8 million people, the same population as the entirety of the neighboring state of New Jersey. Like a massive scale scientific experiment in human sociology, New York squashes people from all corners of globe together into a space of just over 450 square miles. Resembling a giant living organism, subway tunnels and pipes feed the city like giant arteries and veins, with people and traffic like blood cells flowing through the streets.

With the city also being the financial and cultural center of the U.S and perhaps the world, what is the role of science and scientists in this metropolis? New York isn’t lacking when it comes to science, boasting 15 nationally leading medical research institutions and medical centers in addition to over 50 bioscience companies. These institutes include Columbia University, Rockefeller University, Beth Israel Medical Center, Weill Cornell Medical College, Mount Sinai Medical Center, Memorial Sloan-Kettering Cancer Center, New York University and the Albert Einstein College of Medicine and boast an impressive combined 127 Nobel laureates. However, in such a vast research environment it is easy for especially early career scientists to get lost in this expanse. To address this, Nature Network has launched this New York hub with the aim to provide a forum for scientists to come together from the diverse scientific community in New York.

So, you may ask, what is the point of participating in and joining another online network? The difference with this network is that this is specifically for you, the New York scientist. This is maintained and run by local New York scientists with the specific aim to promote local science and to bring together the community within the city. With your participation, we can build a stronger community of scientists and create a network of contacts whether it is for social events or potential collaborations. Have a read of Caryn’s blog for a more detailed overview of the New York site including the forum, events and people sections.

Welcome to the New York Nature Network!