Reading the brain to decode speech

We all do it. Whether it’s pumping the air as we yell “YEEESSS” when a favourite team scores a goal, or waving our hands expressively in front of us as we talk politics with a friend, gesticulating appears to be part and parcel of how humans talk. But what underlies this? Well perhaps a new discovery sheds some light on the question. As he explains to Chris Smith, working with patients paralysed by spinal injuries, Sergey Stavisky has been looking at recordings from the part of the brain’s motor cortex that normally moves the hand. When his subjects spoke, neurones in this area fired up too. But why?

Sergey - The opportunity that we had is we were recording in people with paralysis as part of a clinical trial and we're recording from an area where previously speech was not thought to be located, but no one had actually directly tested for this. And so this gave us this great chance to go and look. So we have electrodes that are placed in the so-called hand-knob area of motor cortex and specifically it's in an area that's well implicated in generating our movements. Traditionally, our group and many others have been decoding hand and arm movements, and because there's a need for people who can't move their arms and hands, to restore that ability. The clinical trial, which is called brain gate, was set up to put electrodes into this arm and hand area of the cortex, read out attempted arm and hand movements and do things like move a computer cursor so someone can use a computer or type or move a robotic arm. It turns out we found that there's also speech related activity there, but initially the arrays, these electrodes, were put in for the main goal of supporting the arm and hand activities of this trial.

Chris - What did you see then in that area when your subjects were speaking?

Sergey - We wanted to test for the speech activity and so we had them do a very simple task. They sat in a room, they were plugged in, we were recording from their brain and we just asked them to listen, hear a word or a syllable like ba, ga, da and just say that back. This, let us record both what they were saying and exactly when they were saying and what the brain cells around these electrodes were doing. What we found was that whenever they were talking, many neurons lit up and not only were they active during speaking, but they were differently active for speaking different words.

Chris - And to be clear the same cells, if you asked them to imagine moving the arm that that part of the brain would originally have controlled, those same cells would respond to that stimulus as well?

Sergey - Yes. So it's not that this area is no longer involved in our movement. So one might think that because these are people with paralysis, so they can't move their arm, but that wasn't the case: if we asked them to try to move their hands or try to move their arm, this part of the brain is very, these cells are much more active than during speaking. It just happens that also if we ask them to talk, the same cells are involved in that and it's not just speaking if we ask them to make the muscle movements that you would need to speak - so let's say open your mouth, move your lips, move your tongue - the same cells are also active. So it's, I want to be clear, we don't think that this is related to language in sort of a high level abstract sense. It's really the movements underlying speech. So in the similar way that we know these cells are responsible for generating the muscles to move the arm and the hand, we think they're also active during the production of speech. The other thing I want to clarify is, we don't think these are causally involved - so we don't think these cells are driving the speech. It's just that, during speaking, they're also active. So it could be that they're getting some sort of copy signal for coordinating arm and hand movements, or just generally keeping track of what's going on kind of in the rest of the movement system.

Chris - Now when I'm talking to you, I'm waving my hands around. Do you think then that actually what you've uncovered here is the connection across the motor cortex that underpins our tendency to just gesticulate.

Sergey - So you've hit upon a very relevant observation. Almost all people like to move their hands when we talk. And there's a rich literature of connections between the hand and the face; some evolutionary theories that speech sort of evolved on top of hand gestures for communication. It's certainly intriguing and quite possible that what we've uncovered is related to that. But I want to be careful because we don't have direct evidence for that.

Chris - If one looks at how the motor cortex is organised, it's almost like a map of the body there, with a person with their legs dangling down on the inside surface of the brain and then stretched out, head pointing downwards, down towards where their ear is. Now the hand area is actually quite close to the head area, isn't it? So do you think what you're picking up here is just spillover: when you activate this very big territory that corresponds to moving your mouth, do you think it could just be that the facilitation that's making that possible, is just spilling over a bit into the hand area and you're just eavesdropping on a bit of that?

Sergey - I don't think it's spillover in the sense of uh, you know if you're having a loud conversation in the other room I can hear a bit of it cause it's just going through the wall. It's not that kind of signal spillover in a transmission sense. Now is it that a large area of the brain nearby is very active during speaking and it's sending some connections to this area and there's some back and forth. Spillover in that sense is very much how I am looking at this and that's what we think is going on. But these are neurons in this area that are actually firing and doing their neuron thing during speaking.

Chris - We do know of course that when people are subject to spinal injuries that there is a degree of cortical reorganisation. So to what extent do you think what you're seeing here is an artefact of the fact that the people you are studying are tetraplegic, and they've been tetraplegic for a while, and therefore they could have remapped how their cortex is wired up and were you to go and look in me, for example, you wouldn't see this relationship?

Sergey - That is a very nagging question that I wish I had the answer to. There are a couple of reasons we don't think it's remapping. If we ask the person to move their arm and hands, we see lots of activity and it looks very much like able-bodied arm and hand movement activity that we would observe in an animal model. So that's point one. Point two is if we asked the person to try to move their foot, we also see activity here and you might think, well they can't move their arm any more and they can talk. So maybe part of the brain devoted to speaking will have expanded to sort of make use of this idle brain area, but that wouldn't make sense for moving their foot because their foot is also paralysed. So, together, these things are more pointing to the hypothesis that really multiple body parts are represented across motor cortex. But you are right that we cannot rule out remapping. And because of the nature of these measurements that you have to undergo a brain surgery to have these electrodes put in, we're not going to be able to answer this in someone who is completely healthy anytime soon.