Augmented Reality in Space

Ben -   Now, when something needs changing or repairing on the International Space Station, astronauts need to open a printed manual or look at a laptop to find out what it is that they need to do.  But that's not necessarily easy to do when everything is floating about or when you're working in a confined space.  So what's the solution?  Well, the European Space Agency are developing a system called WEAR that's short for WEarable Augmented Reality.  It's a headset that superimposes instructions and information onto the thing you're looking at.  Luis Arguello is one of the principal investigators behind the new system, and he joins us on the line now.  Hi, Luis.

Luis -   Hello.  Good evening.

Ben -   So, what's main point behind the WEAR system?

Luis -    Well, this is just a tool to help the astronauts to perform their activities onboard with more accuracy and using less time.  The thing is, as you mentioned before, the astronauts are floating in space.  They have to perform different tasks, experiments, maintenance, and they have instructions for things which have been there for a long time - they use paper.  And for once in a time task, they use the laptop to see what they have to do.  And I don't know if you've seen people working in space, but when they move around, they are floating and they have to help themselves to move around with their hands, and restrain with their feet to the floor.  So, it's a bit difficult to look at the laptop or hold the papers.

So the idea is that you have this system that you put on your head and is connected to the computer, and then you talk with it while you perform your task, and then it gives you the instructions of what you are supposed to do.   What we were trying to do is to help the astronauts to identify the small elements within a very complex structure.  So, he doesn't lose too much time, trying to find the little valve he has to open or close or the thing he has to replace.  And so, he can do things more precisely and also saving some time.

Ben -   So, in order to build this, did you need to build special equipment for it or can you actually use kit off the shelf?

Luis -   We did use off the shelf elements, but it was mainly by accident.  I've been collaborating with Frank De Winne, the commander of the space station, for six months.  He's been working with us, with our section in the Space Agency for many years, and I invited him to view the requirements for the first WEAR tool we were developing.  Well, he told us, "I'm going onboard and I'd like to take one with me."  So, usually you develop custom equipment to fly onboard according to the standard safety and to make it more operable onboard.  In this case, we didn't have the time, so we built this prototype, this is only a prototype, to show how helpful it would be to have this kind of system onboard, and also to assess the usability.  We had very little time but we had to do the whole development, safety assessment with NASA, and all of the process of integration testing.  And so, it was very challenging project.

Ben -   So, once they actually put it on and presumably calibrated it for each person, did they find it really helped?

Luis -   The main problem, as you mentioned, is the calibration.  Because the system was off the shelf, we couldn't think much about the adjustment to make it very simple. So it takes a bit of time to get the overlaying of the images according to what you're looking at.  But after the first calibration is done, they find it very convenient to use, and easy to use.  Well, you need some voice training because the system is guided by voice.  We made an assessment, we gave them a questionnaire and this was one of the points -  usability - and he was happy.

Ben -   So, it sounds like there's quite a long way to go.  This was just a prototype with off the shelf kit, but still very promising.  What's your next step?

Luis -   Well, our friend, Frank, the commander of ISS says he's happy with it and he  would like to have a second version.  So, we have more people, more colleagues working on human interfaces onboard.  We work together with NASA because on ISS, we have many partners.  We have Americans, Russians, European, Japanese, so if you want to put something on board, it has to be agreed with all of the people working on ISS.   So now, if we want to use it for good, then we need to improve the next phase of size, calibration, usability, and to try to integrate it more with other systems onboard the ISS - the inventory, management system and user interfaces we have onboard.

Ben -   So, there is definitely a lot of work to do.  Well thank you ever so much for joining us and for sort of filling us in on what could be a very nice new way to stop bits of paper and Haynes manuals essentially floating around the air.  So, thank you very much, Luis.

Luis -   Okay.  Thank you for the invitation.

Ben -   That was Luis Arguello from the ESA, explaining how Wearable Augmented Reality Systems could greatly improve efficiency on the international space station, eventually doing away with the need to have paper manuals floating around.