The Government Is Making a Robot That Can Play Music for an Unexpected Reason

The Department of Defense has found a new way to create an artificially intelligent being: jazz music.

Kelland Thomas, a jazz musician and associate professor at the University of Arizona School of Music, was recently granted funding to build a system capable of musically improvising with a human player. Thomas' program, called MUSICA, for Music Improvising Collaborative Agent, will be far more advanced than the other virtual musicians out there. 

Instead of following chord progressions programmed into it, MUSICA will take cues from a long history of jazz royalty, including the great geniuses of improvisation, Miles Davis, Charlie Parker and John Coltrane. The software will play collaboratively, as if it were a human musician jamming at an open mic.

"Rather than a human saying 'Here's what I want you to do' and programming the system, it has its own knowledge base and can make its own decisions," Thomas told Mic about his software. "We're trying to build something that communicates with humans and doesn't just wait for the human to tell it what to do."

This isn't a cool, futuristic robot designed to rock out live, like the all-robot band Compressorhead. Actually, MUSICA is going to look pretty boring. 

In the beginning, it will be a computer with a huge musical library and a microphone, with the ability to "listen" to what a human musician is playing and repeat it back. Using the microphone, it'll listen to the live musicians jamming with it, then play complementary solos based on classic riffs by jazz legends.

"DARPA's interested in a program that will interact with humans."

The biggest element is the data. "We'll mine all of [those transcriptions] for patterns that the system can use to have an idea of what it'll play next," Thomas said.

There's a reason why the Department of Defense's advanced-research wing is throwing money at Thomas and his team. As it turns out, MUSICA is more than a futuristic practice tool for aspiring jazz musicians. 

"I think DARPA's interested in a program that will interact with humans," Thomas said. "Right now, a computer waits for us to tell it what to do. The idea [with MUSICA] is that the computer can communicate with us the same way we communicate with it. It might be able to anticipate needs or ask us for clarification on a goal, then help us plan that goal for the future."

The collaboration is with an initiative of DARPA project manager Paul Cohen, called Communications With Computers. Why base the system on music instead of another, more straightforward medium? Because the creativity of music could inform new ways to approach artificial intelligence.

"We're trying to understand human creativity and simulate that."

"It's called computational creativity," Thomas said. "We think of creativity as a human trait. [But] can computer systems do what a human might recognize as creative in some way? We're trying to understand human creativity and simulate that."

The broad strokes of the CWC program, according to its website, are to develop a library of elementary ideas, derive algorithms for assembling complex ideas based on elementary language and context and create algorithms for figuring out what to do or say during communication.

Thomas' program would use recorded improvisation from throughout history (for example, Davis' "So What") to piece together spontaneous riffing, then play it in the context of jamming.

In that sense, MUSICA fits the program tighter than the 9/8 time signature in Dave Brubeck's "Blue Rondo á la Turk."

What the government can do with AI that can improvise: Think about the implications of a computer program that can anticipate human needs. It could open myriad doors for a government research department. For one, software like MUSICA could read through libraries of medical information and highlight and parse out the more relevant information, anticipating what would be helpful for researchers. 

In fact, Cohen requested a similar concept from leading researchers at DARPA's Biology Is Technology conference in June.

Computational creativity doesn't stop at reading, however. The program could create communicative robots to brief and debrief soldiers or researchers on missions, or ask for help when something goes haywire in an unprecedented situation. Basically, it would write its own scripts and adapt when it runs into problems.

The program "would make computers more of an equal collaborator," Thomas said. "I see a system aware of weather and traffic that knows you have to catch a flight, and asks to set an earlier alarm for you based on the traffic. This is interpreting what you need. This is symmetric communication."

While jazz musicians will be pouring into a studio in Arizona to help Thomas teach a robot to wail, they're really finding new ways to teach computers how to be more human, helping usher us further down the path toward artificial intelligence. It's all thanks to a guy with a saxophone and a government research grant.

At that idea, Thomas laughed and said, "We may be the first musicians ever funded by DARPA."