An unrelenting, ravenous appetite for more and more data may be artificial intelligence’s fatal flaw; or at least the fastest way for “poison” to seep in.
Cyber attackers sneak small doses of “poisoned data,” in the form of false or misleading information, into all-important AI training sets. The mission: Sabotage once-reliable models to skew them in a completely different direction.
The majority of AI systems we encounter today—from ChatGPT to Netflix’s personalized recommendations—are only “intelligent” enough to pull off such impressive feats because of the extensive amounts of text, imagery, speech and other data they are trained on. If this rich treasure trove becomes tainted, the model’s behavior can become erratic.
Real-world ramifications go far beyond a chatbot speaking gibberish or text-to-image generators producing an image of a plane when asked for a bird. Groups of bad actors could potentially cause a self-driving car to ignore red stop lights, or on a much larger scale, trigger power grid disruptions and outages.
To defend against the threat of various data poisoning attacks, a team of FIU cybersecurity researchers has combined two emerging technologies—federated learning and blockchain—to more securely train AI. According to a study appearing in IEEE Access, the team’s innovative approach successfully detected and removed dishonest data before it could compromise training datasets.
“We’ve built a method that can have many applications for critical infrastructure resilience, transportation cybersecurity, health care and more,” said Hadi Amini, lead researcher and FIU assistant professor in the Knight Foundation School of Computing and Information Sciences.
The first part of the team’s new approach involves federated learning. This unique way of training AI uses a mini version of a training model that learns directly on your device and only shares updates (not your personal data) with the global model on a company’s server. While privacy-preserving, this technique still remains vulnerable to data poisoning attacks.
“Verifying whether a user’s data is honest or dishonest before it gets to the model is a challenge for federated learning,” explains Ervin Moore, a Ph.D. candidate in Amini’s lab and lead author of the study. “So, we started thinking about blockchain to mitigate this flaw.”
Popularized for its role in cryptocurrency, such as Bitcoin, blockchain is a shared database that’s distributed across a network of computers. Data is stored in—you guessed it—blocks linked chronologically on a chain. Each one has its own fingerprint, as well as the fingerprint of the previous block, making it virtually tamper-proof.
The entire chain adheres to a certain structure (how the data is packaged or layered within the blocks). This is like a vetting process to ensure that random blocks aren’t added. Think of it like a checklist for admittance.
The researchers used this to their advantage when building their model. It compared block updates, calculating whether outlier updates were potentially poisonous. Potentially poisonous updates were recorded then discarded from network aggregation.
“Our team is now working closely with collaborators from the National Center for Transportation Cybersecurity and Resiliency to leverage cutting-edge quantum encryption for protecting the data and systems,” said Amini, who also leads FIU’s team of cybersecurity and AI experts investigating secure AI for connected and autonomous transportation systems. “Our goal is to ensure the safety and security of America’s transportation infrastructure while harnessing the power of advanced AI to enhance transportation systems.”
Moore will continue this research as part of his ongoing research on developing secure AI algorithms that can be used for critical infrastructure security.
More information:
Luiz Manella Pereira et al, A Survey on Optimal Transport for Machine Learning: Theory and Applications, IEEE Access (2025). DOI: 10.1109/ACCESS.2025.3539926
Florida International University
Citation:
‘Poisoned’ AI models can unleash real-world chaos; study shows how these attacks could be prevented (2025, April 24)
retrieved 24 April 2025
from https://techxplore.com/news/2025-04-poisoned-ai-unleash-real-world.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
