A team of researchers from the Department of Information Security at Royal Holloway, University of London have highlighted major privacy risks in technologies designed to help people permanently remove image-based sexual abuse (IBSA) material—such as non-consensual intimate images—from the Internet.
These findings, published in IEEE Security & Privacy, reveal how the techniques currently used to identify and remove abusive content can be attacked with generative AI, potentially putting vulnerable users at risk.
The research team focused on “perceptual hashing”: a method that creates “digital fingerprints” of images to detect harmful content without storing and distributing the original files. Most online platforms (particularly social media sites) keep a list of hashes of known abusive images, enabling the detection and prevention of re-uploaded copies.
Additionally, tools such as “Take It Down” operated by the National Center for Missing and Exploited Children (NCMEC) enable users to self-report IBSA in one place. For this, users can upload perceptual hashes of images, which are then shared with partner platforms such as Facebook and OnlyFans.
However, the recently published paper demonstrates that perceptual hashes are not as irreversible as expected, undermining the privacy assurances claimed by IBSA removal tools on their FAQ pages.
Led by Sophie Hawkes, Ph.D. researcher from the Department of Information Security, the research team examined four widely-used perceptual hash functions, including Facebook’s PDQ Hash (used by “Take It Down”) and Apple’s NeuralHash, and found that all of them are vulnerable to reversal attacks.
More specifically, it became clear that adversarial use of generative AI could approximately reconstruct the original image material. Hawkes explains, “Our findings challenge the assumption that perceptual hashes alone are enough to ensure image privacy, but rather perceptual hashes should be treated as securely as the original images.”
This is particularly concerning given the sensitive nature of IBSA content and the vulnerable user groups these tools aim to protect. Co-authors Dr. Maryam Mehrnezhad (Royal Holloway) and Dr. Teresa Almeida (University of Lisbon) say, “The harms of modern technologies can unfold in complex ways. While IBSA risks are not limited to any demographics, certain groups such as children can be at a greater risk including psychological damage and danger to their safety. Hence, designing secure and safe tools is essential when addressing these risks.”
The researchers argue that the current design of services like “Take It Down” is insufficient and emphasize the need for stronger data protection measures, for example using cryptographic protocols like private set intersection (PSI). By using PSI, it would be possible to enable secure hash matching without exposing sensitive data. This would ensure a more privacy-focused solution for removing harmful content, protecting vulnerable users.
However, presently, the researchers advise users to carefully consider the risks of perceptual hashing and to make an informed decision when submitting a report. In particular, users should take into account both the risk of images being posted online and the risk of the images being reconstructed from reporting hash values.
While there might be no significant loss in privacy when reporting hashes of images that are already shared online, proactive reporting of images might be a concern.
Following responsible disclosure procedures, the researchers have alerted NCMEC to their findings, urging service providers to prioritize the implementation of more secure solutions to ensure user privacy.
Additionally, the researchers advocate for greater transparency, so that users can make an informed decision about the trade-off between their privacy and safety when deciding whether or not to use perceptual hash-based IBSA reporting tools.
Co-author Dr. Christian Weinert, from the Department of Information Security concludes, “Future work in this space will require collaborative efforts involving technology designers, policymakers, law enforcement, educators, and, most importantly, victims and survivors of IBSA to create better solutions for all.”
More information:
Sophie Hawkes et al, Perceptual Hash Inversion Attacks on Image-Based Sexual Abuse Removal Tools, IEEE Security & Privacy (2024). DOI: 10.1109/MSEC.2024.3485497
Royal Holloway, University of London
Citation:
Image-based sexual abuse removal tools are vulnerable to generative AI attacks, research reveals (2024, December 3)
retrieved 3 December 2024
from https://techxplore.com/news/2024-12-image-based-sexual-abuse-tools.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
