AI system achieves 49% score on official SAT questions

Artificial Intelligence (AI) is a field of study that most of us are most fascinated with. It continues to reach new milestones and the future shown in sci-fi movies may become reality very soon.

One such breakthrough in AI research was reported on Monday by UW Today. The Allen Institute for Artificial Intelligence (AI2) and University of Washington researchers have developed an AI system that can solve SAT geometry question at par with the average American 11th grade student.

The system, called GeoS, is an automated system that combines diagram and text interpretation to solve geometry problems. It uses a combination of computer vision to interpret diagrams, natural language processing to read and understand text and a geometric solver to attain 49 percent accuracy on official SAT test questions. If these results were extrapolated to the entire Math SAT test, the system would achieve an SAT score of 500 (out of 800) approximately, the average test score for 2015, the post said.

“Unlike the Turing Test, standardized tests such as the SAT provide us today with a way to measure a machine’s ability to reason and to compare its abilities with that of a human,” said Oren Etzioni, CEO of AI2. “Much of what we understand from text and graphics is not explicitly stated, and requires far more knowledge than we appreciate. Creating a system to be able to successfully take these tests is challenging, and we are proud to achieve these unprecedented results.”

GeoS is the first end-to-end system that solves SAT plane geometry problems. Future work includes expanding the geometry language and the reasoning to address a broader set of geometry questions, reducing the amount of supervision, learning the relevant geometry knowledge, and scaling up the dataset.

According to Fortune, GeoS is actually a respectable start to AI research and that’s because “while so many of the latest artificial intelligence advances are essentially advances in detecting patterns (Siri’s voice recognition, Google’s facial recognition for photos and Cornell’s anticipation engine for cars that can predict what a driver does next, for example), GeoS is actually trying to make sense of the data it’s fed.”