Malevolent Manipulation through Deepfakes: A Tale of Terrifying Misrepresentation on Halloween
In the rapidly evolving world of technology, deepfakes have become a significant concern for individuals and societies alike. These AI-generated media can replicate the pitch, timbre, and individual sounds of any person's voice, raising alarm bells about their potential misuse.
The consequences of deepfake pornography can be devastating, leading to depression, self-injury, and, in some cases, suicide. To combat this issue, the U.S. federal government enacted the TAKE IT DOWN Act on May 19, 2025. This legislation makes it a crime to post or threaten to post non-consensual intimate imagery, including AI-generated realistic pornographic deepfakes, of identifiable real people. Social media companies are required to establish processes to remove such content within 48 hours of a victim’s report, with a one-year deadline for implementation.
At the state level, laws vary, but places like Tennessee, Iowa, and New Jersey have felony provisions with significant fines and prison terms for sharing or creating deepfake sexual content without consent. Michigan is also considering bills that would criminalize and permit civil actions against creators and distributors of deepfake pornography.
Romance scams and voice deepfakes are also a growing concern. While specific laws explicitly naming these issues may not exist, existing fraud and impersonation statutes generally apply. Many states require political deepfakes to be clearly labeled to prevent misinformation, which may inform future regulation of deepfakes used in scams or other malicious contexts.
Deepfakes are used in romance scams to create fake personas for deceiving victims into forming emotional connections online, with the goal of manipulating them into sending money or divulging personal information. Voice deepfakes are used to impersonate someone’s voice, often convincing victims to reveal sensitive information or send funds.
Examining images thoroughly can help in recognizing deepfakes by looking for wrong directions of light and shadows, inconsistencies in facial features, blurriness or distortions, noise, unnatural backgrounds, etc. Voice deepfakes, however, can be more challenging to detect.
To limit the risk of deepfake abuse, it's crucial to be cautious about sharing personal photos online and to thoroughly read the terms and conditions of AI apps. Depending on the nature of the content, contacting local law enforcement agencies may be necessary when photos or videos have been stolen and used inappropriately.
Deepfakes are not limited to pornography and romance scams. They are used to manipulate public opinion and spread disinformation, often during critical moments such as elections and armed conflict. They are also used in advertising to insert celebrities' faces onto the bodies of models. In the film industry, deepfakes can be used to recreate younger versions of actors or bring deceased actors back to the screen.
However, the fight against deepfakes is not just a technological one. It requires vigilance from individuals and societies to recognise and report such content, and for laws to evolve to address the unique challenges posed by deepfakes. As the use of deepfakes continues to grow, so too must our efforts to combat them.
References: [1] Article on TAKE IT DOWN Act [2] Article on state laws against deepfake pornography [3] Article on romance scams and voice deepfakes [4] Article on deepfakes in advertising [5] Article on deepfakes in the film industry
- The TAKE IT DOWN Act, enacted in 2025, makes it a crime to post non-consensual intimate imagery, including AI-generated deepfakes, on social media, requiring companies to remove such content within 48 hours of a victim's report.
- Besides pornography and romance scams, deepfakes are used in various contexts, such as manipulating public opinion, spreading disinformation, and even promoting products or reviving deceased actors in the film industry.
- To combat deepfake abuse, individuals should be cautious about sharing personal photos online, read AI app terms and conditions, and report any inappropriate use of their photos to local law enforcement agencies. Social vigilance and evolving laws are essential in addressing the unique challenges posed by deepfakes.
