A step in the right direction. PLoS makes a move

Neil Saunders is known to be a man who speaks his mind, and usually very wisely. Here is an example

Yes! Finally, someone with the balls to take affirmative action.

Whose balls was Neil referring to? None other than a favorite in these parts, The Public Library of Science. The comment came on Friendfeed, in a thread about a wonderful blog post, Mark Patterson writes

In 2009, in this online world, how do most scientists and medics find the articles they need to read? The answer for the content published by PLoS (and no doubt by many other publishers) is via one of the now ubiquitous search engines, be it one that only searches the scientific literature, or more likely, one that searches the entire web. Given that readers tend to navigate directly to the articles that are relevant to them, regardless of the journal they were published in, why then do researchers and their paymasters remain wedded to assessing individual articles by using a metric (the impact factor) that attempts to measure the average citations to a whole journal?

Like PLoS, myself and many others that I hang around with have said that it doesn’t make sense, but unfortunately no alternatives have been presented (and in our world you always need some metrics). Well, that’s changing. I have always liked the approach of Eigenfactor but they are measuring a different metric. For publication metrics, PLoS is presenting a concept called Article Level Metrics. While it’s difficult for me to say that a single metric is a perfect indicator, PLoS has done several things right. First and foremost is the fact that the metric is the article, which is the very fabric of the web. It’s what makes search engines like Google work. It also overcomes one of my biggest issues with the Impact Factor, that it is about the source, rather than the actual content.

The second thing that PLoS has done right is go beyond the world of the Web of Science and count as citations social bookmarking, blog coverage, etc. The sources of those metrics aren’t perfect yet, but this is a great start and, self plug here, the core of my first Ignite Seattle talk. One could say that this system could be gamed, but the safe assumption is that the chances of marketers trying to game a metric for scholarly publication is low.

Another advantage of this approach is the temporal changes. While appropriate citation will always make sure that an original, seminal, paper in a field has a high metric, more recent, more relevant papers will also score high.

Is the system perfect? No, it probably needs to be refined and perhaps continuously so. But PLoS seem to realize that. Is this the only relevant metric? No, I’d like to see author level metrics, and metrics normalized within a field, something Eigenfactor tends to skew towards (and more traditional search engines). I do wonder if the metric would benefit from some more extensive web crawling and indexing, but that’s an expensive process.

The last paragraph in the post says it all

Article-level metrics and indicators will become powerful additions to the tools for the assessment and filtering of research outputs, and we look forward to working with the research community, publishers, funders and institutions to develop and hone these ideas. As for the impact factor, the 2008 numbers were released last month. But rather than updating the PLoS Journal sites with the new numbers, we’ve decided to stop promoting journal impact factors on our sites all together. It’s time to move on, and focus efforts on more sophisticated, flexible and meaningful measures.

It’s a small step. It may not work, but in my opinion, it’s a key step, because someone’s actually doing something. Now it’s up to the community to support this. Perhaps they will. That the Editor-in-Chief of Nature seems to have a similar viewpoint is encouraging.

I will add one thing though. If you want article level metrics, you need to be web native. You need to be able to follow the links. So please, publish journals not as pdf versions of print, but as first class web citizens.

Related articles by Zemanta

• What’s wrong with scholarly publishing today? (slideshare.net)
• Michael Nielsen: Scientific Publishing will be disrupted (lib.uiowa.edu)
• Science moves from the stacks to the Web; print too pricey (arstechnica.com)