What's in a face?

A team of scientists have successfully harnessed the power of computer face recognition technology to turn it into a diagnostic tool so that individuals with very rare syndromes that are usually associated with characteristic facial features can receive accurate diagnosis.  Christopher Nellaker, from the University of Oxford, spoke with Chris Smith...

Christopher -   Some rare diseases are so rare that a single clinician might not ever see them in their entire career and that their reliance has been on identifying multiple people with the same disease when there might be less than 10 people across the entire world alive at any one point.  So, we tried to create a computer vision approach to teach a computer to analyse faces in photographs to try and find and cluster these patients by which disease they might have.

Chris -   Was this literally going up to people who have diagnoses, taking their photo and then training the computer"this is what condition X looks like", or can you be clever at them at and go to online databases where there might be many, many images of these people and build your tool that way?

Christopher -   So, we went online and collected publicly-available photographs of people with well-established and relatively common rare diseases that we could then collect lot of examples from. Because we have machine learning and computer vision, you're reliant on many examples to get accuracy.  So, we got around about 100 to 200 images for 8 different syndromes.  We use those to train the computer what to look for.

Chris -   And these are all disorders that manifest with changes to the way a person's face makes them look.

Christopher -   So, not all rare diseases will have changes in their facial structure, but in the development of the head and face, a large fraction of our genes in our genome are used.  So, it's quite likely that a mutation might disrupt some part of development for the facial structure as well.  So, it's been estimated that maybe 40% to 50% of rare diseases at least have some sort of manifestation in face or cranial structure.

Chris -   And so, - what - you would present your system with a snapshot, it would draw out these features by analysing the face and then it would compare its snapshot readings with your database of many different faces from individuals with known diagnoses and see which featured it shared most in common with those faces with those diagnoses and then return you, this is likely to be condition "X".

Christopher -   Yes, so it could be used in that way.  But the current approach has just been a proof of principle.  So, it's not sufficiently accurate at the moment to be considered a diagnostic tool.  We consider it more as a way of narrowing the search space for possible diagnoses.  But it does this by comparing between people it had seen before.

Chris -   What about the fact that you're just using 2-dimensional photographs?  Could more accuracy be delivered by looking at 3-dimensional images if you were to take stereoscopic pictures of an individual and analyse that?

Christopher -   Absolutely and so, there has been a lot of work into analysing rare syndromes with 3-D imaging previously and they get very detailed data.  But our approach tries to expand into more accessible data because everyone has access to a camera or scanner, or a digital camera.  And so, it is easier and more accessible in the future to try and use normal 2-D photographs.  And so, we don't have as high accuracy as for 3-D imaging, but with the machine learning we've been using, we've been approaching the discriminatorial classification accuracies of 3-D imaging for the 8 syndromes we've been using.

Chris -   So, how would you see this being deployed in the future?  Would this be a tool that say, a hospital doctor could take a snapshot of a new born baby and feed it into your system which would remotely analyse it and then return a possible diagnosis or a list of diagnoses for individuals that clearly do have a problem but the doctor may not know what they are?

Christopher -   Yes.  So, that's precisely how we see this being deployed in the future through normal healthcare services as an expert tool for clinicians to narrow diagnosis when a rare disease is already suspected.