To live and lie in LA: the science of lying

In recent memory I can’t remember wanting and needing to pay so much attention to the details of a game as I do LA Noire. During interviews and interrogation sequences I stare daggers into the characters on screen. Still, I make mistakes — select doubt when it should have been truth, because I think I see a diverted gaze… I select truth when I should have caught a lie because I missed a curl at the edge of a liar’s mouth. I begin to second guess my ability to read faces, but I restart and try again. This has led me to become quite critical of the game’s main mechanic, and while it’s implementation is solid for a initial effort, I wonder how well the game accurately simulates the mannerisms human expression (lying in particular).

Micro-Expressions

Anyone that has seen the production videos for LA Noire knows that painstaking effort was taken to capture the micro-expressions of hundreds of actors, which were then faithfully recreated in the game. If you’ve ever studied the work of Paul Ekman, or watched a couple of episodes of Lie to Me (based on Ekman), you know that micro-expressions occur at a reactionary level, faster than our conscious minds can control. Our faces betray us when we lie, but also reinforce us when we tell the truth. It is not so much lying as it is a unconscious expression of emotion. Interestingly Ekman confirmed that certain expressions are universal; meaning that tribes of people who have no contact with the modern world both understand and form the same expressions to convey certain emotions. He identifies seven universal emotions – anger, fear, disgust, contempt, surprise, sadness, and happiness. These expressions can occur very quickly, in the moment, and can occur in reaction to external stimuli.

And that’s part of what makes LA Noire confusing. We only see the face of the person we interview when they are speaking, and we also see a prolonged reaction to their own statement while we make our determination of the statement. It is in those moments after the statement is made that we most likely see fear, or contempt, maybe even pride, and that’s what we use to determine Truth or Doubt. This distinction is much more difficult than the Lie condition, which is contingent upon having a piece of evidence — but isn’t it also possible to know when someone is lying even without evidence?

In what would make the game infinitely more difficult to play and design, the player should be able to see the face of the interviewee as you are speaking to them, and much more emphasis should be given to the expressions they make when they provide a statement, rather than the shifty-eyed gaze that occurs while you are selecting. I realize now, because of a conversation with NA guest, Ryan, how hard this might be for the player, because LA Noire does very little in the way of providing practice with feedback for lie detection. There is an intro mission, but no real learning opportunities with feedback for failures — feedback is the most critical element of learning with practice. You know that you get things wrong, but you never know why.

Faces can say a lot, even when we don't

What about other signals?

I also wonder how much attention was given in the game to other forms of communication. Lie detection is not restricted to micro-expressions. The language we use can also reveal our meaning — as in “I did not have sex with that woman.” The phrase ‘that woman’ is used (as opposed to ‘her’ or ‘Miss Lewinsky’) to create distance between people. When we lie or get into trouble, we use language like this that we wouldn’t use in our everyday conversations. If we make up a story we know is not true, we often cannot repeat it event for event; or better yet we would not be able to repeat those events in reverse order. I wonder if the writing of LA Noire fully accounts for these nuances — although I do not remember any of these contextual clues jumping out after playing about 1/2 of the game so far.

Body language is also very relevant to this issue. When we try to remember things we tend to stare off into empty space, as if to retreat into our brains to recall events. A liar might stare directly at another person, because the lie they tell does not come from a memory. They might divert their gaze after the fact because the confidence cannot be maintained. When a lie is told we also tend to unconsciously cover our mouths, or pull on our ear lobes. Humans  unconsciously detect other emotions from body language as well — a puffed chest, a clenched fist, crossed arms, or sweat on the brow all have potential meaning.

A combination of communication signals

Because of the emphasis on facial expression capture in LA Noire, I do not know whether or not these other clues were woven into the fabric of the game. However, lying or not, communication between individuals is not just localized to speech, or even facial expression, but a combination of many factors including body posture and social distancing.

Sometimes however… a cigar is just a cigar.

I don’t mean to knock LA Noire in any way, it’s a great display of technology. In 10+ hours I never once had an Uncanny Valley moment.  However, I feel that sometimes the NPCs give mixed subconscious signals (e.g. mismatched word selection and body language) — because they are not actual human beings. I assume this is why is is often so difficult to decide between Truth and Doubt. However, the effort put forth to create this type of realism in LA Noire makes me excited for what the future of games like these might hold.

Looking toward the future

I believe the next big innovation in gaming is not 3D, not even gesture based first person shooters, rather the implementation of near-real human expression in our avatars and characters. We’ve seen how sophisticated human modeling technology has been used in Heavy Rain, and at the micro-level of LA Noire, but this is just the tip of the iceberg. Perhaps someday technology will allow us to catalog a database of human expression (like FACS) that can be accurately implemented without the need to capture performances from actors. If this is possible, then these expressions can be trigged on the fly, which could lead to richer, branching story experiences. I’m not gonna lie… it’s all very exciting stuff and I can’t wait to see what comes next in this vein of story-based gameplay.

More on Paul Ekman Micro-Expressions

Lie to Me