In the Indian Parliamentary Election, Candidates Used DeepFake to Fake Dialect Videos to Canvass Votes

In the recent Delhi Assembly election in India, a candidate used DeepFake technology to generate promotional videos in other languages to win votes for his team. Although this method achieved good publicity and won some votes for him, the final election results were unexpected.

Recently, DeepFake appeared in the Indian election for the first time and was used by candidates in campaign materials.

In the Delhi state assembly election that just concluded in India, Manoj Tiwari, one of the candidates of the Bharatiya Janata Party (BJP), used video fraud technology to "speak" a language he did not know in order to win over voters of minority languages.

Unfortunately, fate played tricks on them. Even with the help of AI, the propaganda achieved a certain boost, but the BJP still lost the election in the end, winning only 8 (out of a total of 70) parliamentary seats in the state.

Speaking fluently in dialect helps candidates gain support

On February 7, the day before the Delhi assembly election, Manoj Tiwari of the Bharatiya Janata Party shared two short videos on social media, appearing in publicity for his party.

First video (English)

Second video (Haryana)

The two videos, one in English and the other in Haryanvi (an Indian dialect), mainly criticized the current Delhi Chief Minister Arvind Kejriwal and encouraged voters to vote for the Bharatiya Janata Party.

In the video, Tiwari, who is not good at Haryanvi, spoke fluent Haryanvi and called on everyone to vote for the BJP. This move aroused enthusiastic response among people who speak the language, and the video was widely circulated on WhatsApp.

The head of the BJP claimed in the media that this move had achieved good results. Many housewives said that they felt very close to the candidates speaking in their own dialects, which also influenced some people's voting decisions.

But then some careful people discovered something strange in the video.

PR firms use DeepFakes at the behest of political parties

By carefully observing Tiwari's video, netizens pointed out that the character's mouth movements have an awkward feeling and do not seem to be recorded naturally.

Later, researchers from the famous technology media Vice and the Rochester Institute of Technology (RIT) in New York also noticed abnormalities in the video.

Vice later verified the matter and uncovered the secret behind the video.

It has been confirmed that the second Haryana video was not actually shot, but a DeepFake product created by the BJP’s IT department in collaboration with the public relations company The Ideaz Factory.

Allegedly, in order to enable candidates to win over voters of different languages, the BJP and The Ideaz Factory have joined hands to make Tiwari speak in dialect and "appear" on the screen.

Bakshi, the BJP's head of media and technology for Delhi state, told Vice that candidate Tiwari's fake video was aimed at Delhi's large number of Haryana-speaking migrant workers to prevent them from casting their votes for his rival.

Bakshi also proudly stated that through the use of DeepFake technology, the benign communication of the campaign has been expanded unprecedentedly.

India has 30 languages and 2,000 dialects

There is a reason why the candidate team racked their brains to make DeepFake speak in dialect.

There are many spoken languages among different ethnic groups in India. According to current statistics, there are at least 30 different languages and 2,000 dialects.

The Indian Constitution stipulates that Hindi, Sanskrit and English are the official languages of the Indian government, but there are 14 other popular languages (with a population of more than 5 million). Haryana (usually regarded as a subtype of Hindi) mentioned in the article is one of them.

Statistics show that this short video in the Haryanvi language was shared in 5,800 WhatsApp groups and was viewed by around 15 million people.

Facts have proved that DeepFake has made great contributions to its dissemination effect.

Using algorithms to add “fake voices” to videos

How is this technology implemented?

The Ideaz Factory claimed that they had carried out the voice matching process in other languages based on the Tiwari video.

Specifically, a Deepfake algorithm with lip movement technology was used, trained with Tiwari’s voice, and finally the external audio and mouth movements were fused.

After getting the trained model, the company hired a voice actor to imitate Tiwari reading the written lines in Haryanvi and finally injected the voice into the video.

In other words, the video of Tiwari speaking in dialect is a real image of himself with someone else's voice added, and the lip movements are modified to make it look real.

The company also claimed that the current technology is not mature enough to directly generate the voice of the target person using an algorithm. However, it is confident in the technology and is planning to promote it more widely, such as in the upcoming Bihar general election and the 2020 US election.

Although The Ideaz Factory refused to share details of the technology used, RIT researchers still guessed from some clues that they may have used Nvidia's vid2vid model.

Unfortunately, DeepFake did not help them win the final victory

This is not the first time DeepFake has been used in election campaigns, but it is an unprecedented first in India, where the situation is relatively complicated.

Previously, there have been incidents of fake videos during the election process in the UK and the US. For this reason, many regions have introduced stricter legal regulatory measures.

Even on social media platforms such as Twitter, Facebook, and Reddit, strict regulations have been formulated to crack down on the use of DeepFake in election campaigns.

However, there are no relevant legal provisions for DeepFakes in Indian elections. Some even believe that it will be impossible to enforce it in the short term because it will not be publicly supported by the ruling party.

It is worth mentioning that despite using DeepFake technology, BJP still suffered a crushing defeat in this election.

On February 11, the Delhi election results came out, with the BJP winning only eight of the 70 seats, while the AAP (Common Man Party) won the remaining 62 seats.

So as of now, it seems that India is unlikely to consider dealing with the impact of DeepFake.

References:
https://www.vice.com/en_in/article/jgedjb/the-first-use-of-deepfakes-in-indian-election-by-bjp?utm_source=reddit.com
http://www.fx361.com/page/2020/0210/6345716.shtml

-- over--