At Accelrys, a lot of the software I managed was in-licensed from academia. That approach allowed the company to tap into the intellectual resources of some of the smartest academic researchers in the world, but it also created a problem. One was the difference in software development practices. Some of the academic code barely had version control. But that’s the obvious one. In a new post at Computing at Scale, Bill McColl writes about Domain-specific parallel programming. Translating code parallelized for an academic setting, often under the assumption that huge clusters might be available, to an industrial setting where scaling and fault tolerance become critical, where resource availability varies widely, and speed is critical, is always a challenge. This is especially true when you’re trying to shrink wrap software and building interactive interfaces.
So in an era with more scale available, clouds to tap into, accelerators, and new data and distribution models, are we going to see a shift? I still feel that the underlying scientific research has to come from academia. They have the resources, time and incentive to do so, but I think industrial think tanks and expertise can contribute back by working with academia on advanced problems of relevance, e.g. in the area of computing. Will we tap into some of the new domain specific development being done today as a scientific community? It can’t be done by one side or the other. But rather we need to identify approaches as a community and understand what works best, without trying to duplicate efforts. Of course, we need people who understand these new methods and paradigms to implement them.
There will always be a tension between academic research efforts and commercial need. In the life sciences it is especially tough for industry specific apps to be developed from an economic point of view, which is why I believe it will have to be a joint effort.
Your thoughts?
Image via Wikipedia
Technorati Tags: High Performance Computing, Parallel Programming
4 Comments
You're doing the morphic resonance thing again Deepak. Was just talking to someone this morning about the issues that seem to be universal with building the instrument control and data reduction software for large user facilities like mine. Do you let the instrument scientists, who are usually competent coders, but not software engineers, just get on and do it (the traditional approach) or bring in software people are are a) much more expensive and b) don't know the user experience but may be equipped to build something that is more easily extendable (and might be documented)
At my first company, a startup, we had two distinct groups, the scientific programmers (my group) which interfaced very closely with the engineering group, which then put those into production (which is kinda what Accelrys did, but much more tightly coupled). The challenge doesn't go away though if the two sides at least don't have some overlap.
You're doing the morphic resonance thing again Deepak. Was just talking to someone this morning about the issues that seem to be universal with building the instrument control and data reduction software for large user facilities like mine. Do you let the instrument scientists, who are usually competent coders, but not software engineers, just get on and do it (the traditional approach) or bring in software people are are a) much more expensive and b) don't know the user experience but may be equipped to build something that is more easily extendable (and might be documented)
At my first company, a startup, we had two distinct groups, the scientific programmers (my group) which interfaced very closely with the engineering group, which then put those into production (which is kinda what Accelrys did, but much more tightly coupled). The challenge doesn't go away though if the two sides at least don't have some overlap.