What does your face say about you?

Helen of Troy apparently had a face that launched a thousand ships. And scientists say that we form an impression - by looking at the face - of what someone is like in less than a tenth of a second. But what features do we base these opinions on? By studying thousands of faces and peoples' reactions to them, scientists at York University have now produced a computer programme that can tell us how another person will respond when they see us. Which apparently could come in very handy in job interviews or even on dating websites! Tom Hartley led the research and spoke to Chris Smith...

Tom -  When people look at a face, they start to form an impression about the person that they're looking at.  So, they'll first of all be able to judge things like age and sex, maybe more objective features.  But also, they form an impression about more subjective qualities.  So, things like whether somebody is approachable or confident, healthy, aggressive - people can guess who's going to win the election based on what their face looks like.  Another example would be court cases.  So, there's been some work to show that the result of a court case can be determined by the face of the person that their looking at.

Chris -  So, how did you try to approach this?  What did you do to try and find out what the key features were that people, when assessing a face are actually looking for?

Tom -  Well, what we had was a large collection of faces, just ordinary images drawn from the internet.  We looked at all of these images and we went through them very carefully and placed 179 points all around the face to sort of form a join-the-dots-like picture that describes the whole face.  

For the same pictures, we also asked judges to look at these pictures and tell us what do they think about the person so we would say, "Is this person approachable?  Is this person aggressive?"  We got 16 different ratings for each one of the thousand faces.  And then what we really wanted to do was look at the relationship between the join-the-dots picture of the face on the one hand and the social impressions on the other hand.  So, to do this, we trained a computer model to guess what somebody would say about the character based only on information about the shape of the face.  We're able to show that just by training the computer, we could get fairly accurate guesses about what people would then say about the person's character.

Chris -  Does it work?  Can you take a person standing in front of you and get the computer to make a prediction about what the general public would say about that person?

Tom -  Yes, as far as we can tell, it works.  I mean, we know that it will work quite well from this, what's called a cross validation procedure where we train it on one set of data and then test it on another set.  It does seem to work when we try out on our own pitches, we get ratings that accord with our own subjective judgement.  And for future work obviously, it will be great to demonstrate this by using a whole new set of faces and finding out exactly whether the model's predictions accord with those that we have.

Chris -  Do you think that it would be possible to take this sort of readout and use it to give feedback to a person because obviously, it's very difficult to change the face you have?  People often say, "I've got a perfect face for radio."  Perhaps we won't go there.  But the point is, could I be given some feedback by your system that would help me to adjust the way in which I put a facial expression or the expression I tend to adopt to encourage me to generate greater sensations of trustworthiness amongst the people I deal with or a better friendly interaction with people for example?

Tom -  Yeah, that's absolutely right.  I mean, that's one of the most important implications of the work really.  But one of the interesting things about what we found is that many of the features that vary and contribute to our social impressions are features that will not be the same in one image in the next - for example, approachability or trustworthiness, being signalled by a warm smile.  Well, a warm smile is something that might be there in one photograph and not in another photograph.  So, the first step if you want to appear to be more trustworthy is to have a photograph with a warm smile.  The model can tell you exactly which photographs look the warmest, the most approachable, which evoke which impressions of dominance, which evoke impressions of attractiveness.  And so, you can select images that either maximise all of those things at the same time which would be fantastic I guess or maybe you can pick images that are right for the particular purpose that you have at the moment.  

So, if you were going to submit your picture for an online dating agency, you might be more concerned about appearing to be youthfully attractive.  But if you're applying for a job and you're attaching your picture to a CV, maybe dominance and trustworthiness are more important.  And so, for different situations, you might want to use a different picture to portray a different social impression.  We now have a really good handle on what it is about these images that create those impressions.  So, we've put this kind of instinctive knowledge onto a more scientific footing.

Chris -  Tom Hartley from the University of York.  He published that story this week in the journal PNAS.