Technology Can Put Someone Else’s Expressions on Your Face

In the future, we might be able to command other people’s smiles. In a paper presented at the 2015 SIGGRAPH Asia conference, researchers from the University of Erlangen-Nuremberg, the Max Planck Institute for Informatics, and Stanford University demonstrated the ability to transfer facial expressions from one person in a video to another in real time [PDF]. In other words, the technology would allow my smile to appear on a CGI representation of your face, at what the researchers call “previously unseen visual realism.”

A sensor analyzes faces in terms of their geometry and the reflectance of different regions of the skin. Essentially, it creates virtual models of the faces of both participants and the lighting in the video, so that the facial expression of one person can be pasted over the face of another. The technology computes the difference between the first person’s expression and the target's face, and renders them together. The goal is to only manipulate the parts of the face necessary for that particular facial expression, rather than changing the target’s entire face. 

What non-criminal purpose could this serve, you ask? The researchers suggest that this technology could be a way for freelancers working from home to never, ever have to put on pants:

Imagine a multilingual video-conferencing setup in which the video of one participant could be altered in real time to photo-realistically reenact the facial expression and mouth motion of a real-time translator. Or imagine another setting in which you could reenact a professionally captured video of somebody in business attire with a new realtime face capture of yourself sitting in casual clothing on your sofa.

Plus, the prank possibilities seem endless. Imagine if SNL could transfer Kate McKinnon’s facial expressions onto what looks like Hillary Clinton’s actual face.