Home > Uncategorized > Watson: Are We in Jeopardy?

Watson: Are We in Jeopardy?

February 17, 2011

I was very interested to see the part 1 of the IBM Challenge special on Jeopardy. It didn’t meet my initial expectations, but that’s a good thing: From the way I’d heard it described, it seemed like it was going to be some kind of John Henry, man-versus-machine thing.

If you’re looking at it as a competition to try to prove whether computers are better than human players, that’s an exercise in futility. Computers don’t think anything like we do, so it’s an apples-to-oranges kind of comparison.

On the other hand, if your point is, “Hey, look at this awesome computer program we wrote,” then I have to agree. That’s pretty awesome.

Watching Watson play was pretty fascinating. When it was on a roll, it just steamrolled through questions. When it got something wrong, though, it was sometimes hilarious and always interesting to find out why.

For instance, when asked about… Sorry, this is Jeopardy. When given an answer about an olympic gold medalist’s anatomical oddity, Watson’s response was “What is leg?” But the correct response was that he was missing a leg. Watson basically came up with the right answer (sorry, question), but didn’t know how to phrase it.

I’m no software engineer, but here’s why I think Watson missed that one: It doesn’t really understand what it’s saying. It just knows which words are associated with which other words. It knew there was a connection between the athlete in question (whose name escapes me) and “leg,” but couldn’t quite figure out that “leg” needed to have some other words with it in order to make it the right answer. It looks to me like Watson understands each word individually, unless it’s a specific multi-word phrase that it treats as a single word.

That Watson doesn’t understand the meanings of words in the way we do also accounts for a hilarious mistake it made in an earlier challenge: It thought milk was a non-dairy creamer. It didn’t know that “milk” was the opposite of “non-dairy.” It just knew that “milk” was strongly associated with the phrase “non-dairy creamer.”

For a better explanation of Watson’s algorithms, check out Engadget’s interview with IBM’s Dave Gondek. It’s pretty fascinating. (Tip of the hat to Jake Kline for that link.) What I’d like to know is just where Watson gets its information if it’s not allowed to connect to the Internet.

All things considered, I’m not impressed with the idea of computers as game-show contestants, or more broadly, of computers rivaling human intelligence. But I’m very impressed with the work the IBM programmers have done. I look forward to seeing the recordings of parts 2 and 3. (Thank goodness for DVRs!)

%d bloggers like this: