In the Next Mission: Impossible, Tom Cruise No Longer Needs a Human Assistant, AI Is Enough

—— By Chao Shen Ke

Disguise is common in movies and novels. For example, in the hit movie "Mission Impossible 6", Benji (Tom Cruise's henchman) used a fake face to reveal the villain's true identity.

Looking at the Mission Impossible series, mask-changing can be considered a standard move.

AI magician who disguises characters in videos

In reality, it is not difficult to change one's face. Makeup or customized facial masks can make the fake look real. In addition, there is the "photo-cheating" technology that everyone is flocking to.

However, changing the faces of characters in video works is still a big challenge, because changing the faces of dynamic character images requires strong image processing and replacement capabilities.

When a work is halfway through filming and actors have to be replaced, they usually change the script or find a stand-in or add special effects (there is also the practice of editing out the character's scenes), but these methods are costly and may also result in a loss of quality.

For example, the accidental death of Paul Walker was compensated for in "Fast & Furious 7" in several ways, and the complexity of filming increased several times.

If it is possible to "change the face" of the characters in the video, this problem can be easily solved.

DeepFakes, a face-changing program based on machine learning algorithms, is such an AI face-changing magician. At the end of last year, DeepFakes was born. Through it, users can replace the characters in the movie with their favorite "idols" without any sense of disobedience.

However, due to the powerful magic of this program, it has been often used in face-changing in pornographic films since its launch. People use some programs to change the protagonists of pornographic films into people they like to see. emmm…

Its abuse soon led to issues of infringement, privacy, and security. In the end, it was found to have illegally synthesized pornographic videos, seriously infringing on the right to portrait and reputation. Websites such as Twitter, PornHub, and Reddit all distanced themselves from it and announced that they would ban the release of content synthesized by DeepFakes.

It may seem like it has been relegated to the cold palace, but the story is far from over.

DeepFakes Get a New lease of life

Although DeepFakes has been completely banned by several major social networking sites, it has not been wiped out.

What gave it a new lease of life was the huge driving force of the porn industry.

Naughty America, a pornographic film and television production company in California, recently launched a private customization service through DeepFakes. Users can change the faces of characters in AV movies by paying.

Naughty America's paid service includes changing the face of the character and the background of the video. Shorter video conversions only cost a few hundred dollars, while more difficult ones cost a lot of money, for example, some cost up to thousands of dollars.

In addition to paying for the processor's computing costs, this money will also be used to purchase the right to use the character's portrait. Therefore, Naughty America has solved the infringement problem, but the limitation is that it can only be converted into a licensed "face".

In terms of conversion effect, Naughty America has upgraded DeepFakes, which has a good fusion effect when converting the face and background, making it almost indistinguishable. However, in order to convert to the face you want, users need to upload a large number of expression photos and videos in advance (of course, they do not involve infringing content) as a training set for DeepFakes.

To ensure security, Naughty America adds watermarks to the videos, indicating that they are processed "fake" videos.

So how does this AI magician achieve his "face-changing" effect? We just explored it out of curiosity about the technology. "Justice Face.jpg"

DeepFakes tactics dismantled

The core of DeepFakes is a deep neural network model. It mainly includes an encoder and a decoder. When performing facial transformation, the encoder encodes the image and then outputs a new image through the decoder.

In the application process, the quality of image conversion must be guaranteed, and the object of conversion must be accurately identified. The AI model will encode and decode the face of the person in the original video and the face of the replacement person respectively, so that when the face is converted, the expressions of the two can correspond one to one.

Let's use an example to illustrate. Here, we take American talk show hosts Jimmy Fallon and John Oliver as the subjects and briefly explain how to replace Fallon's face with Oliver's.

The whole process is divided into three steps.

The same encoder is used for both faces, with decoder A outputting Fallon and decoder B outputting Oliver.

1) Encode and decode Fallon and Oliver respectively, so that the model can recognize the facial expressions of Fallon and Oliver from complex images and mark the facial expressions;

2) Repeat step 1) until the two decoders can restore the faces of the two people respectively, and the encoder can distinguish Fallon and Oliver from the key facial information through learning;

3) After completing the above steps, re-encode the photo output by decoder A and directly use decoder B to replace Fallon's face with Oliver's face according to the facial expression features.

In this way, Fallon's face can be perfectly replaced by Oliver's. It seems incredible, but with the powerful data processing capabilities of neural networks, it is achievable as long as the hardware does not fail. However, due to the limited training set data, the fusion effect of simple expressions is much better than that of complex expressions.

The Weakness of DeepFakes

Although the fusion effect of DeepFakes is very good, it seems that there are some limitations in the specific practice process.

First, DeepFakes requires a lot of images to train

In a video, if you want to change the face of a person in the video, you need 300 to 2,000 facial images. Moreover, the number required will vary depending on the complexity of the facial expression and the degree of match between the new person and the original video.

In this way, the current face-changing is only suitable for people who have a large number of photos online, such as celebrities who appear in many movies. Therefore, it is difficult for DeepFakes to change the face of everyone.

Secondly, the images used for training should have characteristics

AI models are dependent on training photos. In the face-changing example above, DeepFakes cannot restore Oliver's side profile because the dataset lacks a side profile photo of Oliver. Therefore, if you want a certain facial expression, the training set of pictures must contain a large number of such expressions.

Finally, training the face-changing model is time-consuming and labor-intensive.

With current technology, the process of changing faces in videos is quite costly.

It takes about 48 hours to complete a normal face-changing on DeepFakes, while a more detailed disguise requires about 72 hours of training.

In addition, model training has high hardware requirements. If you rent a GPU at $0.5 per hour, it will cost $36 just to swap the faces of characters A and B, not including broadband fees, CPU and I/O costs. The cost is not low.

However, even though DeepFakes is quite a luxury to use, it is said that many people on Naughty America still pay for it. Emmm... It seems that the driving force of "sexual interest" is still amazing.

The future of face-changing technology for video content

If technology can reduce costs, this face-changing magician AI will be very useful. Although the porn industry has driven its development, it is not just a tool for porn companies to make money.

In addition to the film and television character replacement mentioned initially, DeepFakes can also play a role in recording memoirs or documentaries. It uses photo information to restore the appearance of deceased people, which will be impossible for special effects and makeup artists to do.

There are also articles that mention that it can also be used in the advertising industry, and celebrities can use it to authorize some boring commercials.

Of course, before these applications are put into use, there are many issues that need to be addressed one by one, such as infringement, privacy, security, and abuse.

So the question is, if you could change your face, who would you want to be?