Science Interviews

Interview

Tue, 8th Dec 2015

Gesture controlled electronic music

Adam Stark, Mi.Mu Gloves

Listen Now    Download as mp3 from the show Music Technology: Do or Die?

So far weíve heard how technology can influence the creative process, but what Mi.Mu Glovesabout the world of live music? How are artists integrating technology into their acts? One musician, Imogen Heap, has been collaborating with textile designers, engineers and computer scientists on a project called Mi.Mu gloves. Connie Orbach went to meet Adam Stark who is one of the team...

Adam - My name is Adam Stark and I am a software developer and Iíve been working on the Mi.Mu gloves now for about four years trying to turn all the rich data we get about your hand into music.  At the moment, if you want to make electronic music youíre normally presented with a midi keyboard, or some buttons, or nobs, or faders, but theyíre not very expressive ways of performing electronic music. The thing thatís really, really expressive about people, more than anything, is their hands and so we wanted to make a device that catches the expressiveness of your hands and uses that as a controller for making electronic music.

Connie - I feel like the best way to explain this is if you let me have a go.  Is that possible?

Adam - Sure, Iíll talk you through it.

Connie - Okay.  Fantastic, what do we need to do first?

Adam - So, the first thing we need to put the gloves on and then we need to calibrate them for your hands because everyoneís hands are different.

Connie - The gloves are black, thin and feel very fragile, but actually theyíre incredibly sturdy and designed specifically to be strong, flexible and unobtrusive.  Theyíre wireless and even fingerless so you can play other instruments whilst you wear them but, most importantly, theyíre covered in electronics.

Adam - Through your fingers, there are either one or two bend sensors, depending on the finger and this detects the bend of your knuckles.

Connie - Okay.

Adam - So if you just feel the end of your fingers youíll feel something hard there.

Connie - Itís like a kind of ribbon running down my fingers and thatís a flex ven sensor.

Adam - Yes, thatís a ven sensor.  This part here, this little bud on the wrist detects the orientation of your hands - well itís actually of your wrist - so it detects the roll of your wrist, and if you move your wrist up and down or from left to right.

Connie - And thatís all of the sensors on the glove?

Adam - So, in terms of the information going into the those are the two things; the bend of your fingers and orientation of your wrist, but we have a couple of pieces of feedback as well.  So we have an LED here which will light up; we can programme that to tell you different things about the software so you really donít have to look at the screen.  And on the wrist here we also have some vibration measures that buzz.  We can also use those to tell you that certain things have happened or not happened.

Connie - After a bit more fiddling to get the gloves in place it was time to configure the software to my hand.

Adam - The first thing Iím going to do; Iím going to press calibrate and youíre just going to move your hand open and closed and make sure youíre moving all of your fingers including your thumb from fully bend toÖ.

Connie - Okay, so clench them like a fist and then open them back out.

Adam - Itís now finding all of the extreme points of your hand and where your hand moves to and what the greatest extent and minimum extent of it is.

Connie - Once the software knew the extent of the movement in my hand we could programme it to gestures.  So Adam got me to make a few different movements, like a fist or a pointed finger, and the software remembered them so it could later match them to particular musical commands.

Adam - So what Iíve set up here is a way for you to play chords using the gloves.  If you make a fist with your right hand that will play a certain chord and if you let go, it will stop.

Connie - Okay.

Adam - Now if you move it to a different direction, make a fist, it will play a different chord...

Connie - Ah, this is so exciting.  So Iím moving my hand say in front of me, above my head, to the side and down.

Adam - There should be one over this side as well, just to the left.

Connie - To my left as opposed to my right.

Adam - So if you make this one again in front of you.  Now move on your left hand.  So the right hand here is now making a fist.  So on the left hand if you roll your wrist while you make a sound.  You control the tone of it there.

Connie - Wow!  I have so much power in my hands right now.  I had so much fun playing with gloves. I canít imagine what someone with actual musical abilities might be able to do.

Connie - Of course, I was far too distracted to manage an actual interview so sadly the time came when I had to take the gloves off and get back to work.

Adam - We allow you to combine together different movements.  You might say, if Iím making a fist and Iím rolling my wrist from left to right, that particular movement Iím going to connect to a certain musical control perimeter.  So, when I say that, what I mean is we connect it to some piece of music software, maybe itís the volume fade or maybe itís the panning around the room or itís the amount the reverb.

Connie - So there were originally, I guess, a tool for Imogen but you have many other people using these gloves as well.  Is that right?

Adam - Thatís right.  We realised there was something really, really powerful in these gloves and that every time we gave them to somebody different to put them on, they used them in a completely different way.  We were really interested in how other people might use them and so weíve now got about 20 different users around the world and theyíre using it for all kinds of different things.  Weíve got film composers, weíve got Ariana Grunday whoís a sort of pop star, and weíve got a charity called Drake Music who work with musicians with disabilities that are barriers to them making music. So they use technology to try and break down those barriers, and weíre working with a musician called Chris Halpen and heís been making huge use of the gloves and gigging all over the place. He has cerebral palsy and heís had a journey through dealing with his condition and trying to find ways around his condition to make music, and I think the gloves for him have been somewhat of an emancipation.

Connie - Thereís a lot of performance involved in wearing these gloves.  You really have to be willing to throw your hands around and play with it, so I guess thereís only a certain type of performer that would be happy doing this as well?

Adam - I think thatís a really good question.  Iím a guitarist and I play in band where you normally tend to look at your guitar pedals and that kind of thing, and suddenly youíre not looking at your guitar pedals, youíre looking up at the audience and youíre moving your hands around, and itís a very different experience.  Sometimes I really love it and sometimes I feel a bit like a lemon.  It does take a certain personality,  but it does communicate to people better, I think, than almost anything else.

Connie - Iím not sure that Iíd feel comfortable flailing my arms around on stage but maybe thatís why Iím not a performer.  That, and the whole musical ability thing but, with these gloves, there seems little doubt that technology and creativity are hand in hand.

Multimedia

Subscribe Free

Related Content

Comments

Make a comment

See the whole discussion | Make a comment

Not working please enable javascript
EPSRC
Powered by UKfast
STFC
Genetics Society
ipDTL