Jeopardy!, Errors, and Artificial Intelligence

Thinking about the man vs. machine competition that is going on this week on Jeopardy! The challenge is to get computers to interact in a way that accounts for the sophistication of actual intelligent human discourse replete with allusions, puns, and plays on words. The concept behind it is a particular picture of artificial intelligence: achievement. A computer displays intelligence when it can do the sort of tasks that humans can do. This is different from the approach offer by Alan Turing who offered an experiential criterion, that is, if you cannot tell whether you are interacting with a person or a computer, then the computer is displaying intelligence.

But last year, when my personal hero Doug Hofstadter (how cool am I, I call him Doug -- he's actually a really nice guy) came to campus, he took the question in a different direction. He said that if we want to think of a computer as intelligent, we shouldn't look at what it does well and see if it is the same sort of thing we do well, we should go in the other direction. Ask whether the computer makes mistakes in the same sorts of ways we make mistakes. Humans make interesting and peculiar kinds of errors and the question is whether we see computer errors of the same sort.

In the match, the computer did answer some questions wrong and in ways that were similar to wrong answers given by its human counterparts. Which is interesting. But I want to think of the sort of errors the human mind makes.

There are Oliver Sacks style wiring problems. My favorite of which is Autosomal Dominant Compelling Helioophthalmic Outburst syndrome or A.C.H.O.O. which is a common condition in which exposure to sudden bright sunlight causes sneezing. The nerves from the eyes and nose are close together and in many people strong signals from the eyes gets misinterpreted by the brain as coming from a disturbance in the nose triggering the sneezing reflex.

But Hofstadter's interest is more in cognitive errors. The sort he presented are typified by something TheWife said a while back when we were having trouble with our well. She needed me to go out and open up the metal plate covering the opening and clearly couldn't decide whether to say "lid" or "cover" and so asked me to go out and "open the liver."

A similar sort is confusing words that are somewhat close in meaning and in sound. Last night at a little league meeting, the head of the league was explaining the means of drafting teams which is a cooperative approach. He meant to say that the coaches confer on decisions about which player to draft, but he repeatedly said that they commiserate about the choices. While I am sure that there are some players (or parents) about which coaches can commiserate, that is clearly not what he meant. But both words begin with the prefix "co" and refer to situations in which people come together verbally.

But then there are other sorts. Some deal with linguistic expectations. Yesterday, TheWife had rung in on skype just as the chair of our Women, Gender, and Sexuality program walked into my office for a meeting. I told TheWife, "Hang on a sec, Sweetie Pie. I have to talk with someone for a couple minutes. I'll call you back." Seeing that I was on the computer and hearing "Hang on a sec, Sweetie Pie," my colleague responded "o.k." and ducked out of the office. I was horrified for a moment thinking that the head of our women's study program thinks I just called her "Sweetie Pie," but, of course, after about 5 seconds, she realized the full context, that she had misinterpreted the utterance and found it amusing. Had she processed the entire utterance, that it was not an attempt to communicate with her would have been clear -- I am quite friendly with my colleague, but not THAT friendly -- and on a second processing, it was. But the original interpretation came from assessing not only the utterance, but the context which led to certain expectations which were partially fulfilled by the utterance. This is a common kind of error for humans we hear what we expect to hear, not what was said.

What are other human errors that would be interesting to find in computer behavior?