Freebase – the scientists perspective

The blogosphere is abuzz about Freebase today. Tim O’Reilly has a great post on the subject, where he explains what the early alpha product is all about.

So what is Freebase? Well, it comes from a company called Metaweb Technologies, a spin off from Applied Minds. According to the Metaweb FAQ, Freebase “is home to a global knowledge base: a structured, searchable, writeable and editable database built by a community of contributors, and open to everyone. It could be described as a data commons.”

That definition alone is likely to attract the attention of bbgm readers. It should come as little surprise that Freebase content is licensed under a Creative Commons license. There isn’t much info on the site, so I am going to base my writeup on the O’Reilly article and others like the one on TechCrunch.

Apparently, Metaweb has gathered content from many of the web’s free databases, such as wikipedia, etc. Then users add metadata that makes connections between the different types of data. Using the metadata, a whole additional set of information gets pulled in, including various empty fields associated with the data type that is recognized. Users can then add additional information (does that information get pulled in automatically if it is there in Wikipedia?). The built in data structures can recognize various information types and will create appropriate pages and data fields. The approach that Metaweb is taking is different from how the W3C is organizing the semantic web, which is based on controlled vocabularies. What we have here is a bit of a self-contained beast, which can grow on its own as users add data and categories and connections. The following quote from Tim O’Reilly is compelling.

If Metaweb gets this right, this bottom up approach will build new connections between data, new categories and ways of thinking. It will likely be messy and contradictory for a while, but as I told John Markoff for the story on Metaweb that he was preparing for the New York Times tonight, they are building new synapses for the global brain.

As Michael Arrington notes, the Freebase mission is not unlike Google, albeit with a different twist (I still think that the generalized, unstructured approach that Google takes is necessary to find data). There are some who are uncomfortable about all our data being centralized in one place, but is it really centralized? I wonder. After all the sources reside all over the web, plus developers are allowed to access Freebase via APIs and use the data where required (since it is all CC licensed).

The concept definitely sounds cool. I am quite enamored by the idea of using structure to bring together data that is distributed all over the web. For scientific information, where a lot of structure is built into various databases already, I think this concept might work quite well. In fact there is one area where I believe the concept is dynamite – pathways. Using the capabilities of the Metaweb engine, where users can construct associations and linkages, maybe we can build a CC-licensed database of networks and pathways. If the published literature were searchable, that would even be more powerful. I wonder what others think about using this approach for scientific information?

Now of course Freebase might go down in flames, but it is definitely an idea worth following, and yes, I signed up.

Update: An update from Tim O’Reilly on his “outdated” view of the Semantic Web.  I believe the key is openness.  With smart people like the metaweb folks, the W3C, and the user community, the semantic web has a good chance to succeed.  To succeed for the sciences, it is important that the Semantic Web be universally accepted and understood.

Technorati Tags: Freebase, Metaweb, Semantic Web, Data, Startup, Pathways