India uses deepfakes and other AI technologies in election campaigning

Experts fear that these methods will soon change elections around the world.

The elections to the lower house of the Indian parliament, which began on April 19, could be the largest in the world: the potential number of voters is 970 million people, or about 12% of the entire population of the planet. Voting is held in seven stages and will end on June 1, with the results due to be announced on June 4. Critics of the current head of government Narendra Modi argue that the victory of his party, the Bharatiya Janata Party, in a third consecutive election could lead to a crisis of Indian democracy. The opposition accuses the prime minister of inciting discord and increasing repression. The tense situation in the elections is also due to the fact that competing parties are actively using artificial intelligence for campaigning and propaganda for the first time. Meduza explains why experts believe that deepfakes and other AI technologies are a threat to electoral procedures not only in India, but throughout the world.

AI technologies have been used for campaigning in previous elections in India. But not on such a scale as now.

The Bharatiya Janata Party (BJP) first used AI tools to generate personalized messages to voters back in 2014. Since then, the technology has become a key part of Indian parties’ campaigning arsenal and a key element of their propaganda discourse, according to Patharlapati Nagaraju, a researcher at Osmania University in India. Deepfake, a digital identity replacement, first became a political tool in the 2020 Delhi assembly elections, when two AI-generated videos of local BJP chief Manoj Tiwari surfaced online. In one video, the politician addressed potential voters in English, while in the other, he addressed them in Haryanvi, a language spoken in Delhi and some other parts of the country.

The BJP justified its use of AI by saying that it was not to discredit its rivals, but simply to convey its leader’s message to as wide an audience as possible. However, the very fact that such videos appeared has caused much controversy in India, with journalists and politicians suggesting that deepfakes could be used in the future to spread false information and manipulate voters.

Doubts about the admissibility of using AI in campaigning grew after the 2019 parliamentary elections, during which different parties actively used social media to spread fake news about their competitors. A group of researchers from Microsoft Research India concluded that the campaigns of almost all major parties “included elements of online disinformation, lies about their opponents, and propaganda.”

Researchers at the Digital Forensic Research Lab found a surge of bots trying to push hashtags and “manipulate traffic” in favor of various rivals on Twitter in the final weeks before the 2019 elections. The surge was also seen on Facebook, where the two largest parties, the BJP and the Indian National Congress, were allegedly spreading edited photos and fake videos. The campaign was also carried out on WhatsApp, where bots were used to mass-forward messages calling for votes for certain candidates.

But in 2024, the manipulation of information and the production of fake news in Indian politics have reached a new level, thanks to the development of generative artificial intelligence . “Generative AI itself is no more dangerous than bots,” explains computational social scientist Kathleen Carley. “But now we have both bots and generative AI.”

Carly said generative AI makes it easier to program bots, allowing them to reach a larger audience and evade blocking for longer. And Pratik Wagre, executive director of the India-based Internet Freedom Foundation, which advocates for digital rights, suggested that with AI technology unregulated, this year’s election “will be a Wild West.”

Even Bollywood stars are becoming characters in election deepfakes. Politicians are also trying to generate dirt on their opponents using AI

In early March 2024, an image of Indian Prime Minister Narendra Modi stylized as Bhishma, one of the main characters in the Indian epic Mahabharata and the earthly incarnation of the god of heaven, Dyaus, was shared on Instagram. The AI-generated image was supposed to symbolize the role that Modi’s supporters envision for him: an outstanding warrior and defender of his country from external threats. Thanks to social media promotion, the image reached an audience of approximately 35,000 users in two days.

According to Meta data obtained by Al Jazeera, between February 27 and March 21, the Modi-focused Instagram account Hokage Modi Sama shared more than 50 posts depicting the prime minister as a deity, military hero, or religious hero. “Through these images, [Modi supporters] are trying to project an image of a sage and warrior — an indefatigable, invincible, and flawless leader worthy of our unconditional loyalty,” explains Amogh Dhar Sharma, a political communications researcher at Oxford. “This is the latest in the BJP’s long-term strategy to cultivate a cult of Modi as a wise and insightful ruler.”

According to Al Jazeera, one of Modi’s main rivals, Rahul Gandhi, a member of the Indian National Congress, also used AI technologies for campaigning. His supporters circulated images on Instagram and Facebook of the politician, allegedly posing with children and farmers. The latter represent the main opposition electorate – in the past, they often protested against Modi’s market reforms. The caption under the AI-generated image said that “Rahul Gandhi empathizes with farmers, promises them unconditional support and readiness to pay close attention to the problems of each.”

While images of the prime minister as a folk hero or campaign art by his opponent's supporters may not amount to manipulation of facts, experts fear that other campaign materials generated using similar technologies could be used to spread false information and deceive voters.

Another example of how AI is used in election campaigning in India is deepfakes. One of the most popular characters in such videos is the same Modi: on the eve of the start of voting, videos with a digital avatar of the prime minister were distributed on WhatsApp, which directly addresses the voter by name and reminds them of how the authorities have improved their lives in recent years. Similar videos with other members of the ruling party are also appearing. According to The New York Times, video messages are automatically translated into any of the dozens of languages ​​spoken in India. Chatbots operate on the same principle, calling voters and asking them for support in the elections in the voices of political leaders.

Devendra Singh Jadoon, a leading Indian deepfake expert who specializes in creating advertisements and scenes for Bollywood films, said hundreds of politicians approached him ahead of the election. According to Jadoon, more than half of them asked for “unethical” things, such as “creating” phone calls in which rivals allegedly admit to breaking the law and accepting bribes. Others wanted to add their opponents’ faces to pornographic images. Some asked Jadoon to specifically create a low-quality deepfake featuring their party’s candidate, so that if a fake incriminating evidence appeared, it would be easier to expose them. Jadoon explained that he refuses such work only on personal principles. He added that there are currently no technical ways to stop the flow of fake videos, images, and audio recordings.

According to Jadoon, of the 200 requests he has received since the beginning of 2024, he and his team of nine have taken on only 14. However, not all AI experts are so principled. In the midst of the election campaign in India, a deepfake video of Bollywood star Aamir Khan went viral . In the video, Khan’s digital double recalled the BJP’s failed promise from a decade ago to ensure that every Indian would have 1.5 million rupees (or $18,000) in their bank account. The deepfake ends by urging his compatriots to vote for the opposition Indian National Congress. The day after the video appeared online, Khan issued a denial, calling the video “an absolute lie.”

According to Nikhil Pakwa, editor of online publication MediaNama, which specializes in digital media analysis, such electoral manipulations can be effective in India because of its deep-rooted culture of celebrity worship. “India is a country where people love to take pictures with celebrity lookalikes,” Pakwa explained . “If, for example, the prime minister calls them and talks as if he knows their name, where they live, and what problems they face, they are almost certain to be delighted.”

“I’m seeing more political deepfakes this year than last year,” said Hani Farid, a computer science professor at the University of California, Berkeley. “They’re more complex and realistic.” AI expert Devendra Singh Jadoon agrees. According to him, the content itself has become better, and the time it takes to produce it has been reduced many times over: while it used to take eight to 10 days to create a believable deepfake, even with powerful graphics processors, it now takes just a few minutes.

Parties justify the use of AI by citing cost savings and convenience, but competitors are accused of manipulation and fakes. Controlling the flow of generated images and videos has not yet been achieved

In discussing how AI could influence elections and voter attitudes, Oxford political communications researcher Amogh Dhar Sharma notes that parties are using deepfakes not only to discredit opponents, but also to advance political narratives. “AI is not necessarily tricking voters into believing something that is not true,” Sharma explains. “AI is also enabling more creative content with lots of cultural references. This leads to more entertaining propaganda that is more likely to be shared, which in turn helps it spread.”

Indian politicians accuse opponents of trying to gain an advantage through AI, but their own parties defend it . They say they use such technologies simply because they are fast and convenient, not out of a desire to deceive voters or defame opponents.

According to Indian political scientist and communications specialist Sagar Vishnoy, “one of the clear benefits of implementing AI in election campaigns is that it allows for significant cost reduction, time savings, and increased efficiency.” Previously, parties relied on call centers with many employees and volunteers to interact with the electorate — AI has allowed for a reduction in costs in this area by several dozen times and a better structured communication with voters. For example, thanks to call technologies, surveys have become faster — now parties receive a volume of data in a day that used to take months to collect.

One proponent of AI in election campaigning, 33-year-old BJP member Shakti Singh Rathore, says he would never have been able to spread Modi’s message to so many people if he had not created a digital twin. “AI is great because it allows us to move forward,” Rathore says .

However, advocates of AI restrictions believe that the potential dangers of using deepfakes and other similar technologies outweigh the benefits. Experts point to cases where artificial intelligence could give one side an unfair advantage, such as the video that went viral during the 2023 Telangana state assembly elections in India, showing the state’s Chief Minister K. Chandrashekar Rao and other BJP members urging people to vote for their Congress rivals.

The negative impact of AI in political struggles is also being observed in some other countries – for example, in Bangladesh, opposition representative Rumin Farhana fell victim to a smear campaign last year, when unknown persons distributed fake photos of her in a swimsuit, which outraged many conservative voters. In Moldova, opponents of the head of state Maia Sandu generated deepfake videos in which the president allegedly asks citizens to unite around the Party of Socialists of Moldova, which has close ties to United Russia. And in South Africa, ahead of the May elections, an edited video was published in which rapper Eminem called for support for the opposition party.

In their report on the use of AI technologies, Microsoft analysts said that many political deepfakes in different countries are backed by companies associated with the Chinese government. Presumably, this is how Xi Jinping's regime tries to promote "its" candidates. According to Microsoft, in 2024, China may be spreading election fakes in India, South Korea, and the United States.

In late March, Narendra Modi proposed labeling AI-generated content “to avoid confusing voters.” However, India, like most other countries, has no legal framework to effectively curb the flow of images and videos created by artificial intelligence. As The Washington Post writes , “law enforcement agencies rarely keep up with fakes that can spread quickly on social media and in group chats.” Indian political strategist Ankit Lal agrees that India “has no effective way to monitor AI-generated content.” This makes it virtually impossible to prevent election manipulation. And it is up to individuals, not regulators, to decide how ethical it is to use AI in political campaigning.

"Jellyfish"