Top A.I. Program Fails an 8th-Grade Science Test

Artificial intelligence may one day steal your job as a nurse, gamer, or global tyrant, but first it'll have to learn how to pass an eighth-grade science test. 

As MIT Technology Review reports, the winning program at a contest organized by the Allen Institute for Artificial Intelligence (AI2) correctly answered just 59 percent of a middle school science test's questions, which were modified to make them easier for a computer to understand (diagrams and questions that weren't multiple choice were removed). The contest's winner, Chaim Linhart, trained his computer by giving it thousands and thousands of questions paired with the correct answers; for his efforts, he received a $50,000 prize.

While perhaps disappointing for your average 14-year-old, the failing score was heralded by Technology Review as a "remarkable" achievement. Oren Etzioni, AI2's director, told the Review that he hopes the results lead to a more complex test with a larger prize, which would hopefully encourage more programmers to compete. “I do think that a longer contest and ‘deeper’ AI will be required to get from 60 percent to 80 percent or more,” Etzioni said. “It’s our hypothesis that you can’t do this with cheap tricks—that you have to do something smarter.”

Even if artificial intelligence continues to struggle with science, it's already proficient in other areas: Last September, an A.I. system called GeoS took the math section of the SAT, matching the average score for an 11th grader; and just last month, researchers revealed an A.I. program that can write semi-coherent political speeches.