MIT Data Science Machine May Replace Human Intuition with Computers

MIT may soon be replacing human intuition with algorithms. Scientists have created   a new big-data analysis system that outperforms 615 of 906 human teams.

Big-data analysis consists of searching for buried patterns that have some kind of predictive power. But choosing which "features" of the data to analysis usually requires some human intuition. In this case, though, the researchers tested their first prototype system that analysis big-data against human teams in three data science competitions. The machine managed to finish ahead of a total of 615 human teams.

In fact, in two of the three competitions, the predictions made by the Data Science Machine were 94 percent and 96 percent as accurate as the winning submissions. In the third, it was a more modest 87 percent. But while the human teams labored over their prediction algorithms for months, the Data Science machine took between two and 12 hours to produce each of its entries.

"We view the Data Science Machine as a natural complement to human intelligence," said Max Kanter, one of the researchers, in a news release. "There's so much data out there to be analyzed. And right now it's just sitting there not doing anything. So maybe we can come up with a solution that will at least get us started on it, at least get us moving."

The system exploits structural relationships inherent in database design; databases typically store different types of data in different tables, indicating the correlations between them using numerical identifiers. The Data Science Machine tracks these correlations, using them as a cue to feature construction.

The findings could be huge when it comes to analyzing big data. It could be used as a way to get started on multiple datasets.

31200, 31107

Related Stories

Quantum Mechanics: Scientists May 'Paint' Quantum Electronics with Light Beams

How Robots May Mimic Lizards and Snakes When Traveling on Sand

For more great science stories and general news, please visit our sister site, Headlines and Global News (HNGN).