How good does AI music get?

AIVA are using machine learning to compose what they call “artificial soundtrack music” - the kind of thing you would hear in the background of a video, and never even realise it’s computer-generated! CEO Pierre Barreau joined Chris Smith...

Pierre - So basically AIVA works in two steps. The first one is it looks at large amounts of scores, thousands of scores written by different composers in order to infer some rules about how music is composed, and analyses patterns in the melody and the harmony and the structure of the scores and the instrumentation and so on. And then using those rules, it's able to create whole new pieces of music. The second step is basically to convert these pieces of written scores into audio that any consumer can listen to and enjoy. And in order to do this, we basically sample and digitalise instrument recordings and stitch them together in order to produce a believable recording of the scores that were written by AIVA.

Chris - How does it, sort of, know when you're teaching it, how does it know what it should be producing with what sorts of vocabularies, what sorts of musical instruments, what sorts of notations? How does it actually learn and then produce something which is an accurate facsimile of what you've taught it to produce?

Pierre - Right, so we actually classify the data that AIVA sees into different categories. For example, on the broad level, we start with style, but it also knows about other categories like the pacing of a piece of music. So is it a slow piece of music? Is it a fast piece of music? It also knows about the instrumentation and the different instruments that are used and it knows things like the structure of the piece. So using all of these categories, it actually uses that in the composition process in a way that the users of our product can actually say, "I want a piece of music that's three minutes long, that's written for a symphonic orchestra in a cinematic style, and that feels slow-paced".

Chris - There's still a lot of human input to this though, isn't there, where you've got that coding being done by the human in the first place. Is that just because at the end of the day, we know what it is to be human and the computer doesn't. And so therefore you need that kind of human input in order to give the data the structure, but once it's got that -you're off, you're away.

Pierre - Yeah I would say the human input is necessary and it's a bit like when I started learning about music, I had some teachers that were teaching me and in the same way, humans are always going to be here to give additional guidance to computers in order to perfect the learning and make it more effective.

Chris - So put simply then, if I want to reproduce something that sounds like, say Mozart or Beethoven could have composed it, then I would basically educate the system with enormous amounts of the work of those composers and get it to then learn from them. And then I, having coded that input up appropriately, I could get it to reproduce something that sounds like a reasonable facsimile of what one of those composers would produce.

Pierre - Absolutely. That's one way to do it. Another way is to, for example, give to AIVA a very specific piece of music by Mozart, if we're trying to recreate Mozart. And we give to AIVA one specific piece of music, it's able to sort of analyse the database and to say, here are all the compositions that I've looked at that sound similar to this one, and use the specific material and patterns found in those compositions in order to recreate something that's very specifically in the same style as one composition from Mozart.

Chris - Now, given it is Beethoven's birthday, it's only fair to see what AIVE made of Beethoven. So you've fed AIVA a significant amount of Beethoven's music, and this is what Beethoven came up with when asked, can we have some Beethoven, please?

Pierre - The introduction is very reminiscent of Beethoven's fifth, the sort of romantic melodies. Also the choices in harmony are very Beethoven-like.

Chris - And if you asked it tomorrow to produce me some Beethoven, would you get a completely different result or are you going to get pretty much the same thing again and again and again? Is it a bit limited in its vocabulary in that respect?

Pierre - No, usually we get very different results. Of course it depends how we define Beethoven. Like if we say I want something in the style of the fifth symphony, we're going to get very specific compositions that sound like the fifth symphony. But if we say I want something that sounds broadly like Beethoven, we can get widely varying outputs from AIVA.

Chris - And what do audiences make of it when you play them this? Have you done the sort of study where you take something you've made and subject it to audience scrutiny and say who wrote that?

Pierre - Absolutely. So actually earlier this year we were commissioned to write a piece in the style of Mozart. And this piece was performed in Japan, in Tokyo, and the orchestra basically performed one piece by Mozart and the piece that was commissioned by AIVA in the style of a Mozart. And they were asked to tell which one was written by AI and which one was written by Mozart. And it turned out that 60% of people thought that the AI composed tune was a Mozart tune. You know, basically, AIVA won the Turing test. I would add an asterix here because it all depends on the audience's familiarity with Mozart's works. And I think at the end of the day, what matters is that those people went into the concert, enjoyed the music and had a good time. You know, sometimes people say, when will AI compose better music? I think that's besides the point. The real point, just like Rick said, is how useful is it going to be as a tool to help creators, and are people going to enjoy the music, rather than ‘when are computers going to replace humans’.

Chris - And you bring up Rick, I mean, should he worry? Should he be watching his back? Is he out of a job soon do you think?

Pierre - Absolutely not. I think that even if AIs objectively get better at composing music than humans, I think one crucial element that humans bring to the table is meaning in what they do. And an AI could come up with a new style of music, totally crazy style of music, but if there's no creative intention that can be explained, I think it's very hard for an audience to really connect. And for this reason, I think that humans will always be the best at writing music, at least for other people.