As we build collective intelligence applications, which are the heart of Web 2.0, we need to make sure that their inner workings remain open and transparent, or we may fall into the same trap that ended up bedeviling Wall Street in this past year, in which no one understood any longer just how the machine they’d built was going to perform, and the Golem was out of control.
The above quote comes from a post by Tim O’Reilly where he discusses politics and Warren Buffet. It struck me as telling that the peer review process and the academic model also forms an analogous “machine”. I am not an academic, so I could be wrong, but from what I read, hear, and from my memories as a graduate student, there appears to be a lack of transparency in many processes. What are the criteria for tenure? What types of publications are exptected? What are the metrics behind peer review?
I fear that once science loses credibility, getting it back is going to be very very hard. We don’t want that do we.
People have started speaking out quite a bit. This was one of the more contentious topics at Scifoo. I have a feeling that there is going to be a change of the guard over the next decade and perhaps a change in how we bring transparency to science, academic or otherwise.
If things were transparent, IMO, we'd be less concerned about conflicts of interest, etc etc. It's a question of trust, perceived or otherwise.
Yes it appears that quite a bit of these processes are undocumented. For instance, when is it necessary to do a postdoc versus not? Well the typical answer is that it depends on what science you are in? If its a popular physical science such as biology you may have to do three postdocs at the least versus a rather novel science such as bioinformatics where you might not even have to do a postdoc at all.
Then there is the issue of peer review. I have no experience with this process myself either but have heard of the complications associated with them. For instance there's the issue of competitive bias. If you submit something for publication and one of the reviewers is studying the exact same concept the temptation of unethical behavior occurs.
Long story short, I agree with what you are saying and feel that its something that needs to be spoken about more.
Required: A transparent science machine
I fear that once science loses credibility, getting it back is going to be very very hard. We don’t want that do we.
Technorati Tags: Open Science