Talk about Citations
Last night the British Library held the first of their Talk Science events. The topic of the evening was 'Citation in Science - don't quote me on that' and it was hosted by Tim Birkhead, is Professor of Evolutionary Biology at the University of Sheffield. Back in January Tim wrote a piece for the Times Higher Education suppliment
discussing citation in science, and specifically mis-citation in science. This formed the core of the
discussion last night. Tim chatted about his views on the topic for about 20 minuets and then opened it up to the floor. There were a great diversity of people present, from publishers, funders, senior academics, journalists, PhD students
to library staff. There may even have been some elusive members of the genreal public (if so they were keeping quiet.) The discussions mainly revolved around the burden of perr review, the way that citation metrics change the practice of science, and how citation should be viewed and practised. I found the discussions very informative. It's a big topic, and one that is bound to get most academics animated, one way or antoher. One area the we have discussed on this blog from time to time is how people can get credit for work outside of the traditional peer reviewed literature, and this didn't really
come up last night, but that again is a topic worth spending more than a little time discussing. They also served some free booze, and we all got a choclate bar (which I am polishing off as I type this). The discusison has rolled over to Nature Network so you can head over and join in if you wish.
Comments
Speaking of citations, did you guys see C&EN recent article on the subject:
http://pubs.acs.org/cen/science/86/8621sci1.html
Posted by: Mitch | May 28, 2008 09:44 PM
Hi Mitch,
I'd not seen that article before, it's a nice overview of many of the topics that surround citations. One of the very interesting things about citations is that they are value. A citation doesn't tell you whether the author agreed or disagreed with the paper being cited. This opening paragraph of the article you point to really nails that.
All though citations are a directed graph, I guess one needs a weighted directed graph to take that into account, and furthermore one would need a way to determine whether the author was in agreement or disagreement with the citation.
Perhaps semantic markup could help in the future?
In the mean time, there are a number of people who have been working on measuring the emotional temperature of the internet by tracking the use of emotional terms in blogs to see whether people are happy or sad.
We Feel Fine is a really pretty example of this kind of work. Wouldn't it be great if you could do something similar with scientific disciplines?
(We Feel Fine allows you to drill down by weather, gender, location, age, emotion, I just checked up on the home country, and got the following response:
'No feelings found from people aged 20 to 29 in 2008 when it is sunny in Ireland', I guess the weather has been pretty bad :( )
Posted by: Ian Mulvany | May 29, 2008 04:21 AM
A semantic solution is of course the most elegant, but I would doubt you could convince authors to rate each of their references positively or negatively. I suspect an algorithm to access the author's intent would be more difficult in science; when I'm critical of a paper I often write my criticisms subtly.
I think people need to target their metric more specifically to what they want answered. I know committees love to have one simple number to access scientific merit, but I'd argue that it isn't quantifiable within the same order of magnitude.
Posted by: Mitch | May 30, 2008 04:37 AM
That reminds me of someone who was telling me about work they were doing to judge how well received various conference were looking for comments about the conferences on blogs. The system they built had a hard time dealing with sarcasm, which is not a big surprise.
I think it's a fair point, scientists are usually polite in their criticisms. Nature China is running a site on which readers can up or down vote papers you can see that the vast majority of votes are positive votes.
Metrics are also tricky. The debate the british library are hosting is getting a . It seems that there is a feeling that the metrics that are used now for evaluation are poor at this job. I think there is a case for looking at different types of metrics, but for reasons that go beyond evaluation, for instance to help with literature searches, or to help find emerging topics, or to help people move across into a new field.
Posted by: Ian Mulvany | May 30, 2008 04:48 AM