Wolfram|Alpha should have launched officially by the time you read this, though it has been live since Friday evening. The execution is slick. The different result visualizations are a great idea. It’s loaded up with cool widgets and APIs. Most of the time the servers don’t fall over (despite some glaring security holes). To quote FriendFeeder Iddo Friedberg it’s “a free, somewhat simple interface to Mathematica”. Free for personal, non-commercial use, anyway. If you’ve got any questions about the GDP of Singapore then wolframalpha.com is the place to go.
I think that it’s a very interesting project and that it’s important to bear in mind that as the homepage says:
Today’s Wolfram|Alpha is the first step in an ambitious, long-term project to make all systematic knowledge immediately computable by anyone
(emphasis mine)
WA certainly has lots of potential but was anybody who used it over the weekend not left mildly let down? You’d have thought that we’d all have learned not to believe interweb hype after the Powerset and Cuil launches but even if you took all the pre-launch media guff with a liberal sprinkling of salt it was hard not to expect much from Alpha. A breathless Andrew Johnson suggested that it was “the biggest internet revolution for a generation” in The Independent: “Wolfram Alpha has the potential to become one of the biggest names on the planet”.
Personally I was disappointed because I’d been expecting the wrong thing. I’d assumed that WA was akin to Cyc, which is a computational engine that takes a large manually curated database of “common sense” facts and relations and uses it to infer new knowledge. For example: searching photos for “someone at risk for skin cancer” might return a photo captioned “girl reclining on a beach”. Reclining at the beach implies suntanning and suntanning implies a risk of skin cancer.
A few years back a Paul Allen venture called Project Halo took the engine behind Cyc and taught it facts and rules from chemistry textbooks; it took a lot of time and money but the resulting system had a good go at answering college level chemistry exam questions.
It turns out that WA doesn’t do anything like this. One of the most interesting posts about the system that I’ve read comes from Doug Lenat who perhaps not coincidentally is the founder of Cyc. Lenat was impressed by WA but notes that it’s a different beast altogether:
It does not have an ontology, so what it knows about, say, GDP, or population, or stock price, is no more nor less than the equations that involve that term"… [it’s] able to report the number of cattle in Chicago but not (even a lower bound on) the number of mammals because it doesn’t know taxonomy and reason that way
If a connection isn’t represented by a manually curated equation it isn’t represented at all. Apparently the Mathematica theorem prover is currently turned off as it’s too computationally expensive.
One example of this is: “How old was Obama when Mitterrand was elected president of France?” It can tell you demographic information about Obama, if you ask, and it can tell you information about Mitterrand (including his ruleStartDate), but doesn’t make or execute the plan to calculate a person’s age on a certain date given his birth date, which is what is being asked for in this query.
It might seem harsh to criticize WA for not being what people (OK, I) wanted it to be but bear in mind that Wolfram’s About and FAQ pages suggest that WA is an amazing leap forward that brings “expert level knowledge” to everybody and “implements every known model, method, and algorithm” – it’s not like they were managing expectations particularly well.
Even if the computational inference part is lacking the system is still potentially useful as a well presented structured data almanac – but I’m not convinced that it’s a winner for life sciences data.
Wolfram|Alpha for genetics questions
If I search for “DISC1” I get back information about the human gene (genetics coverage in WA is lacking, despite Stephen Wolfram using a sequence search in the video demo. Only the human genome is available). It tells me the transcripts, reference sequence, the coordinates of DISC1, protein functions and a list of nearby genes.
That data is useless without proper citations, though. What genome assembly release are the gene coordinates on? Are the “nearby genes” nearby on the same assembly, or do they come from a different source? Who and what predicted the transcripts, and what data did they use? Were the protein functions confirmed by work in the lab or just predicted by algorithm (if so, what’s the confidence score)?
The “sources” link at the bottom provides a bunch of high level papers describing different genome databases but doesn’t specifically match these to elements of data on the page: furthermore there’s a disclaimer suggesting that actually the data could be from somewhere else entirely that isn’t listed. Not much help.
What happens with contradictory data? The GDP of North Korea varies depending on who I ask. How does WA – or rather whoever curates that data for WA – decide which version of the answer to show?
I’m also worried about how current the data is. Lenat mentions that:
In a small number of cases, he also connects via API to third party information, but mostly for realtime data such as a current stock price or current temperature. Rather than connecting to and relying on the current or future Semantic Web, Alpha computes its answers primarily from [Wolfram’s] own curated data to the extent possible; [Stephen Wolfram] sees Alpha as the home for almost all the information it needs, and will use to answer users’ queries.
I can see why you wouldn’t want to rely on connections to third party data sources for anything that looks like a search engine; users expect a quick response. But in fast moving scientific fields the systematic knowledge that’s useful to researchers isn’t static like dates of birth or melting points – datapoints get updated, corrected and deleted all the time. Does Wolfram bulk import whole datasets regularly? If I correct an error in a record at the NCBI when will Wolfram pick it up?
Can a monolithic, generalized datastore run by Wolfram staff work as well as smaller specialized databases run by experts? What’s the incentive for the specialized databases to release data to Wolfram in the first place, given that WA will be a commercial product?
(for more science tinged coverage there’s lots of Wolfram|Alpha chatter on Friendfeed, a new room dedicated to collecting life sciences feedback for Wolfram and Deepak has a good blog post out)
Web publishing as a discipline has few tenets but I think release early, release often and don’t be afraid to fail are pretty sound. That was the philosophy behind 
Web publishing as a discipline has few tenets but I think release early, release often and don’t be afraid to fail are pretty sound. That was the philosophy behind 
Wolfram|Alpha should have launched officially by the time you read this, though it has been live since Friday evening. The execution is slick. The different result visualizations are a great idea. It’s loaded up with cool widgets and APIs. Most of the time the servers don’t fall over (despite some