Paul Rinne, Imperial College, Blake Hannaford, Washington University and Alex Russomanno, University of Michigan
Our current crop of virtual reality relies heavily on visual and audio, but could we ever create virtual touch? This is whatís known among scientists as haptics and Graihagh Jackson made it her mission at the to find out at the Eurohaptics conference which took place this weekÖ First up though - to re-create touch, you need to know what it is and how it works and thatís where Imperial's Paul Rinne helped Graihagh out...
Paul - From my medical background, touch is incredibly important. So simple, make a cup of tea. If youíre not able to sense the heat of the cup, if youíre not able to sense the pressure of how you hold the cup, you canít perform these daily tasks.
Graihagh - So I suppose an example is: I have a handbag full of rubbish and Iím trying to find my keys, and Iím rummaging around in that bag. So what is it about feeling those keys that makes me able to identify that they are in fact keys?
Paul - Our sort of interface with the world is our skin, and within our skin we have a number of different sensors and receptors. These can range from pressure receptors to thermal, so cool or hot, stretch receptors, for example. And itís actually these receptors when we start touching objects and we start interacting with them that they send signals up through our nerves, through our spine to our brain, to an area called the somatosensory cortex, and itís up in that area of the brain that all this information is processed. And weíre able to suddenly distinguish, for example, between different temperatures of objects, different edges of objects, how hard an object is, how soft it is, and thatís where weíre able to start distinguishing these things.
Graihagh - So my snotty tissues from my keys?
Paul - So youíre snotty tissues from your keys, yes!
Graihagh - So weíve managed to hack, if you like, or mimic vision and sound so why is it so hard to then mimic touch?
Paul - Itís an incredibly complicated process. It all stems down to the neuroscience of it; we need to try and work out the processes of how these receptors work, how the nerves work and, ultimately, when we go up into the brain the processes of what is firing, what signals are being sent to different parts of the brain.
Graihagh - Thereís a reason why neuroscience is called the last frontier. Thereís more we donít know about the brain than we currently do know but this hasnít hampered hatics. In fact, the conference was littered with demos so the University of Washington's Blake Hannaford and I, we sat down to try out the latest electronic braille with Michigan's Alex Russomanno...
Alex - A large reason for this is that for blind people, itís very difficult for them to access digital content and the way they read is real braille and they way they can interact with any graphics has to be via touch. And so what weíre trying to do is create a large display, where that display has pixels but those pixels are features that raise up and down on a surface that you can feel.
Graihagh - Alex had a small square prototype - 8 pins across by 7, and these pins they pop up and down kind of like whacamole. So, in theory, this could kind of be like the future of Kindle but for the blind. To demonstrate its versatility though, Alex didnít show Blake and I a series of braille letters but, actually, a game of snake.
Alex - I donít know if you remember the original Nokia cellphones used to have that snake gameÖ
Graihagh - Except with Alexís game, instead of seeing the snake eat the apple, you feel it. And this is hard, actually really hard and I was completely useless. Blake fared a little bit better thoughÖ
Blake - My eyes are closed..
Alex - And itís all restarted.
Blake - Iím kinda getting it now.
Graihagh - Now that youíre playing it as a blind person would, I canít actually see if youíre winning or losing, so you know you could just beÖ
Blake - Iím sort of giving random inputs to the game.
Graihagh - So this is one way that haptics could work in that you are physically moving things underneath your fingertips but how else might you recreate a sensation?
Blake - Well thereís tactile, which is like feeling textures and small shapes and edges as they deform the skin of your fingers and weíre very sensitive. Somebody just said today that thereís 2,000 tactile sensors in each fingertip that we have and thatís a lot, and another is temperature displays. Suppose you had three objects that are the same shape and they weigh the same but theyíre made of different materials. You can still tell because a metal object will feel colder than a wood object even if theyíre at the same temperature. The metal absorbs heat faster out of your fingers so people are studying that phenomenon to see if that can give more realism to a touch sensation.
Graihagh - Thatís pretty cool, hey?
Blake - Yes, thatís why weíre all here. We all think itís cool and thatís why weíre crazy enough to work on this for 25 years.
Graihagh - What do you think is going to be the most exciting developments that you think are going to happen in the next few years?
Blake - Well, you know weíre at at a very exciting period right now for virtual reality. Thereís this whole second wave of startups: the Oculus Rift, the Vibe, the Samsung, all these companies and when you interact with a nice visual, immersive virtual reality you really become aware that you lack the haptics. You see cool objects out there, you reach out and your hand goes right through it and thatís not reality and so I see a lot of rapid progress sooner. The full holodeck that would fool you into believing that itís real, that will be further.