Why Computer Scientists Are Studying Tom Hanks's Face

Computer scientists at the University of Washington are trying to figure out just what makes Tom Hanks look like Tom Hanks. Using the thousands of photos taken of the actor over the course of his lengthy career, they’ve mapped out his face, and created a digital replica that can mimic his expressions. It’s all part of a larger project to create highly accurate 3D simulations of human faces.

Hanks makes an especially great subject because there are so many pictures of him out there—and because his facial expressions are so distinct (as The Atlantic notes, he's got an "essential Tom Hanksiness").

In their recently published paper, appropriately titled “What Makes Tom Hanks Look Like Tom Hanks” [PDF], the University of Washington scientists wrote, “Tom Hanks has appeared in many acting roles over the years. He’s played young and old, smart and simple, characters with a wide variety of temperaments and personalities. Yet, we always recognize him as Tom Hanks. Why? Is it his shape? His appearance? The way he moves?”

It’s a surprisingly existential question for a group of computer scientists to ask, but its answer may hold the key to some pretty significant advances in virtual reality and filmmaking. Already, computer scientists have developed an algorithm that charts changes in facial expression using 49 pre-defined points on a person’s face. Using photographs of Tom Hanks and other celebrities, they’ve built realistic facial simulations, whose movements eerily mimic those of their real-life counterparts.

The Atlantic explains that once the technology is more advanced, it will it make it easier to portray real people’s faces in virtual reality and movies. Most older forms of facial mapping have involved the laborious and complex process of photographing a subject from all angles in a highly controlled setting. By contrast, the new process doesn't even require the person you're simulating to be present—all you need is a few good photos.

“In the optimal setup, you’d say, ‘Let’s go to a lab, put 20 cameras around the room, decide on some lighting, and constrain all sorts of environmental conditions,” Ira Kemelmacher-Shlizerman, one of the researchers on the study, told The Atlantic. “The big breakthrough in our research is we’re doing it in completely unconstrained environments unlike other research in this space.”

Check out examples of the researchers' strange and fascinating digital ventriloquism below: