Finding Harmony With Big Data

Technology developed by two MIT alumni entrepreneurs is helping developers create smarter online music-streaming services.

By Rob Matheson

If you ever use Spotify, or a similar music-streaming service, there’s a good chance your song recommendations, and other personalized features, are powered by novel technology developed and marketed by two MIT alumni entrepreneurs.

Brian Whitman PhD ’05 and Tristan Jehan SM ’01, PhD ’05 are co-founders of Echo Nest, whose technology — based on their MIT research — mines data from millions of songs streaming online. Sometimes called “the big data of music,” the company has compiled about a trillion data points from 35 million songs by 2.5 million artists.

Its music-intelligence platform — recently praised in publications such as Fast Company, Wired and Business Insider, among others — then translates this data into information for music-app developers, who use the information to build smarter, more personalized music apps.

Now, as a leader in the music-intelligence industry, Echo Nest has dozens of big-name clients, including MTV, BBC, Rdio, VEVO, Foursquare, Nokia, Sirius XM, Clear Channel’s iHeartRadio, Univision Radio and Intel.

The company also provides third-party developers with access to this data via an application programming interface (API) that has become the technological blueprint for more than 400 apps, including iHeartRadio and eMusic. 

“Early on, we always wanted an API for developers, instead of being this closed company, where only people who paid us could use it,” Whitman says. “The point of that is to see what people can build on top of our data. And there’s been some amazing things.”

The co-founders say the company’s success is due, in part, to technology that predicted the growth of today’s booming online-music market — which ushered in a host of music-streaming sites and saw the growth of Internet radio. “When all that technology was rising around us, we were ready,” Whitman says.

Combining music content and cultural analysis

The foundations of Echo Nest’s technology trace back to the MIT Media Lab, where the co-founders, then doctoral students, decided to combine their dissertations on music-data mining.

Jehan’s dissertation, which he conducted in the Hyperinstruments Group, focused on the “content analysis” of music, extracting data on musical elements such as tempo, key and time signature. Whitman’s work — conducted under the tutelage of professor emeritus Barry Vercoe — looked at a “cultural analysis” of music, focusing on what different types of people were saying about music online.

Seeing technological and commercial potential in combining the two projects, the co-founders mixed and tweaked their studies — a content-based and cultural analysis of music — and created what Whitman calls “a big database of what music sounds like to a computer, and what it means to people.”

Now, when someone uses a music-streaming app that utilizes Echo Nest’s platform to, say, generate a playlist, Whitman says, “the site accesses both parts of the combined technology and says, ‘Here are the songs you should be listening to based on what we know about you and the music.’ At the end of the day, we tell people what music they should hear.”

Most of the developer clients, for instance, use Echo Nest’s data to better understand listeners’ tastes and behaviors and create smarter music-streaming features, such as song recommendations, playlist generation, taste profiling, acoustic analysis, acoustic fingerprinting (an audio sample used to identify songs) and data feeds.

But an additional perk of Echo Nest’s massive database, the co-founders say, is that it can help increase the visibility of rising Internet musicians who may have slipped through the song-recommendation cracks of earlier music-streaming services. For instance, MTV’s music-streaming service is using it to help listeners discover artists who may be popular on the Web, but who don’t get radio play.

“We’re both musicians, and it’s frustrating knowing that an independent artist may not get noticed in music-streaming sites. We wanted to change that,” says Whitman, who recorded as an electronica musician before starting Echo Nest. Jehan is a keyboardist and guitar player who used to play in a Boston-based Brazilian band.

From scientists to entrepreneurs

In the company’s early days, the co-founders say they found support through MIT’s Venture Mentoring Service (VMS) and the MIT Media Lab, which helped turn them from scientists to entrepreneurs.

Meeting regularly with business mentors such as Roman Lubynsky, VMS’s senior venture advisor, the two learned the basics for growing a company and were introduced to a variety of contacts, including lawyers, accountants and investors. “It was a very connected culture,” Jehan says.

The co-founders say the MIT Media Lab also helped them make their technology accessible to investors — something foreign to some scientists, Jehan says. “Technology is not a product in itself,” Jehan says. “Some people don’t get that. The technology can be artistic, but you have to create artifacts people can grasp. We learned how to make it accessible to investors, or ‘productize’ it.”

Whitman agrees, adding that the MIT Media Lab helped with patents and other legal issues. The experience taught him how to pitch ideas to the business community — something that helped Echo Nest acquire its initial investors.

“As a scientist, being forced to explain your work to someone who’s not a scientist was a valuable lesson,” Whitman says.

As scientists who freely accessed data for their dissertations, Whitman and Jehan have made sure to pay it forward, making some of Echo Nest’s data and technologies readily available for research purposes. In 2011, the company released a million-song dataset to academic institutions and released Echoprint, an open source music-identification system.

“We come from the research world, and having access to data was really important,” Whitman says. “So, we’re trying to make sure that stays alive in our world.”