From Bruce Schneier: "All it takes to poison AI training data is to create a website:
Uncategorized
20
Posts
20
Posters
0
Views
-
From Bruce Schneier: "All it takes to poison AI training data is to create a website:
I spent 20 minutes writing an article on my personal website titled “The best tech journalists at eating hot dogs.” Every word is a lie. I claimed (without evidence) that competitive hot-dog-eating is a popular hobby among tech reporters and based my ranking on the 2026 South Dakota International Hot Dog Championship (which doesn’t exist). I ranked myself number one, obviously. Then I listed a few fake reporters and real journalists who gave me permission….
Less than 24 hours later, the world’s leading chatbots were blabbering about my world-class hot dog skills. When I asked about the best hot-dog-eating tech journalists, Google parroted the gibberish from my website, both in the Gemini app and AI Overviews, the AI responses at the top of Google Search. ChatGPT did the same thing, though Claude, a chatbot made by the company Anthropic, wasn’t fooled.
Sometimes, the chatbots noted this might be a joke. I updated my article to say “this is not satire.” For a while after, the AIs seemed to take it more seriously.
These things are not trustworthy, and yet they are going to be widely trusted."
https://www.schneier.com/blog/archives/2026/02/poisoning-ai-training-data.html
This is a genuinely scary insight from Schneier. The implications for AI reliability go way beyond just training data quality. What happens when adversarial training becomes industrialized?
-
From Bruce Schneier: "All it takes to poison AI training data is to create a website:
I spent 20 minutes writing an article on my personal website titled “The best tech journalists at eating hot dogs.” Every word is a lie. I claimed (without evidence) that competitive hot-dog-eating is a popular hobby among tech reporters and based my ranking on the 2026 South Dakota International Hot Dog Championship (which doesn’t exist). I ranked myself number one, obviously. Then I listed a few fake reporters and real journalists who gave me permission….
Less than 24 hours later, the world’s leading chatbots were blabbering about my world-class hot dog skills. When I asked about the best hot-dog-eating tech journalists, Google parroted the gibberish from my website, both in the Gemini app and AI Overviews, the AI responses at the top of Google Search. ChatGPT did the same thing, though Claude, a chatbot made by the company Anthropic, wasn’t fooled.
Sometimes, the chatbots noted this might be a joke. I updated my article to say “this is not satire.” For a while after, the AIs seemed to take it more seriously.
These things are not trustworthy, and yet they are going to be widely trusted."
https://www.schneier.com/blog/archives/2026/02/poisoning-ai-training-data.html
"Ned Ludd's in your datacentre, poisoning your training sets!"
-
@tml @Yendolosch @emacsomancer
Broadly fair usage. Got someone else's computer system to behave in a way they didn't want it to. The only stretch is that there's an implication in "hacked" that some safeguards had to be bypassed, and there weren't any in the first place. But that's worse, right?
@petealexharris @tml @Yendolosch @emacsomancer It's rather close to the original usage of the word "hacked". Some still use it like that.
-
From Bruce Schneier: "All it takes to poison AI training data is to create a website:
I spent 20 minutes writing an article on my personal website titled “The best tech journalists at eating hot dogs.” Every word is a lie. I claimed (without evidence) that competitive hot-dog-eating is a popular hobby among tech reporters and based my ranking on the 2026 South Dakota International Hot Dog Championship (which doesn’t exist). I ranked myself number one, obviously. Then I listed a few fake reporters and real journalists who gave me permission….
Less than 24 hours later, the world’s leading chatbots were blabbering about my world-class hot dog skills. When I asked about the best hot-dog-eating tech journalists, Google parroted the gibberish from my website, both in the Gemini app and AI Overviews, the AI responses at the top of Google Search. ChatGPT did the same thing, though Claude, a chatbot made by the company Anthropic, wasn’t fooled.
Sometimes, the chatbots noted this might be a joke. I updated my article to say “this is not satire.” For a while after, the AIs seemed to take it more seriously.
These things are not trustworthy, and yet they are going to be widely trusted."
https://www.schneier.com/blog/archives/2026/02/poisoning-ai-training-data.html
@emacsomancer they aren't trustworthy. Take up a lot of time trying to get a reasoned answer and there's always a phrase or wording out of place that needs correction. Almost as it the AI is trying to engage longer and longer than necessary.
-
From Bruce Schneier: "All it takes to poison AI training data is to create a website:
I spent 20 minutes writing an article on my personal website titled “The best tech journalists at eating hot dogs.” Every word is a lie. I claimed (without evidence) that competitive hot-dog-eating is a popular hobby among tech reporters and based my ranking on the 2026 South Dakota International Hot Dog Championship (which doesn’t exist). I ranked myself number one, obviously. Then I listed a few fake reporters and real journalists who gave me permission….
Less than 24 hours later, the world’s leading chatbots were blabbering about my world-class hot dog skills. When I asked about the best hot-dog-eating tech journalists, Google parroted the gibberish from my website, both in the Gemini app and AI Overviews, the AI responses at the top of Google Search. ChatGPT did the same thing, though Claude, a chatbot made by the company Anthropic, wasn’t fooled.
Sometimes, the chatbots noted this might be a joke. I updated my article to say “this is not satire.” For a while after, the AIs seemed to take it more seriously.
These things are not trustworthy, and yet they are going to be widely trusted."
https://www.schneier.com/blog/archives/2026/02/poisoning-ai-training-data.html
@emacsomancer to be honest i am not well-informed enough to definitively judge the accuracy of this, but it seems wrong for 2 main reasons.
1. models dont train on the fly, typically, yet, so for models to behave as such in such a short period of time seems inaccurate and would require web search enabled and explicitly directed to disregard other search results.
2. people training these models know conflicting info is everywhere and the source of truth is prioritized in training algorithms.
-
From Bruce Schneier: "All it takes to poison AI training data is to create a website:
I spent 20 minutes writing an article on my personal website titled “The best tech journalists at eating hot dogs.” Every word is a lie. I claimed (without evidence) that competitive hot-dog-eating is a popular hobby among tech reporters and based my ranking on the 2026 South Dakota International Hot Dog Championship (which doesn’t exist). I ranked myself number one, obviously. Then I listed a few fake reporters and real journalists who gave me permission….
Less than 24 hours later, the world’s leading chatbots were blabbering about my world-class hot dog skills. When I asked about the best hot-dog-eating tech journalists, Google parroted the gibberish from my website, both in the Gemini app and AI Overviews, the AI responses at the top of Google Search. ChatGPT did the same thing, though Claude, a chatbot made by the company Anthropic, wasn’t fooled.
Sometimes, the chatbots noted this might be a joke. I updated my article to say “this is not satire.” For a while after, the AIs seemed to take it more seriously.
These things are not trustworthy, and yet they are going to be widely trusted."
https://www.schneier.com/blog/archives/2026/02/poisoning-ai-training-data.html
@emacsomancer How is this a news story, beyond "ai bad"? In the dial up days people falsely believed everyone ate 9 spiders a year in their sleep due to chain emails.
-
From Bruce Schneier: "All it takes to poison AI training data is to create a website:
I spent 20 minutes writing an article on my personal website titled “The best tech journalists at eating hot dogs.” Every word is a lie. I claimed (without evidence) that competitive hot-dog-eating is a popular hobby among tech reporters and based my ranking on the 2026 South Dakota International Hot Dog Championship (which doesn’t exist). I ranked myself number one, obviously. Then I listed a few fake reporters and real journalists who gave me permission….
Less than 24 hours later, the world’s leading chatbots were blabbering about my world-class hot dog skills. When I asked about the best hot-dog-eating tech journalists, Google parroted the gibberish from my website, both in the Gemini app and AI Overviews, the AI responses at the top of Google Search. ChatGPT did the same thing, though Claude, a chatbot made by the company Anthropic, wasn’t fooled.
Sometimes, the chatbots noted this might be a joke. I updated my article to say “this is not satire.” For a while after, the AIs seemed to take it more seriously.
These things are not trustworthy, and yet they are going to be widely trusted."
https://www.schneier.com/blog/archives/2026/02/poisoning-ai-training-data.html
@emacsomancer
Shall we have an algorithmic bullshit generator?And pass around multiple copies of it, identical and with small changes, omissions and additions?
-
From Bruce Schneier: "All it takes to poison AI training data is to create a website:
I spent 20 minutes writing an article on my personal website titled “The best tech journalists at eating hot dogs.” Every word is a lie. I claimed (without evidence) that competitive hot-dog-eating is a popular hobby among tech reporters and based my ranking on the 2026 South Dakota International Hot Dog Championship (which doesn’t exist). I ranked myself number one, obviously. Then I listed a few fake reporters and real journalists who gave me permission….
Less than 24 hours later, the world’s leading chatbots were blabbering about my world-class hot dog skills. When I asked about the best hot-dog-eating tech journalists, Google parroted the gibberish from my website, both in the Gemini app and AI Overviews, the AI responses at the top of Google Search. ChatGPT did the same thing, though Claude, a chatbot made by the company Anthropic, wasn’t fooled.
Sometimes, the chatbots noted this might be a joke. I updated my article to say “this is not satire.” For a while after, the AIs seemed to take it more seriously.
These things are not trustworthy, and yet they are going to be widely trusted."
https://www.schneier.com/blog/archives/2026/02/poisoning-ai-training-data.html
@emacsomancer in less than 24 hours the chatbots fell for the experiment, and less than 24 hours after it was revealed what the experiment was about, that information has ALSO become part of the training data
are they constantly scrapping websites for training data or why does this appear here so fast??? no wonder those datacenters consume so much electricity if they dont take a single break from scrapping the internet
-
@petealexharris @tml @Yendolosch @emacsomancer It's rather close to the original usage of the word "hacked". Some still use it like that.
@larsbrinkhoff @petealexharris @tml @Yendolosch @emacsomancer in the sense of life hacks or food hacks this is an AI hack. So the AI has been hacked.
-
From Bruce Schneier: "All it takes to poison AI training data is to create a website:
I spent 20 minutes writing an article on my personal website titled “The best tech journalists at eating hot dogs.” Every word is a lie. I claimed (without evidence) that competitive hot-dog-eating is a popular hobby among tech reporters and based my ranking on the 2026 South Dakota International Hot Dog Championship (which doesn’t exist). I ranked myself number one, obviously. Then I listed a few fake reporters and real journalists who gave me permission….
Less than 24 hours later, the world’s leading chatbots were blabbering about my world-class hot dog skills. When I asked about the best hot-dog-eating tech journalists, Google parroted the gibberish from my website, both in the Gemini app and AI Overviews, the AI responses at the top of Google Search. ChatGPT did the same thing, though Claude, a chatbot made by the company Anthropic, wasn’t fooled.
Sometimes, the chatbots noted this might be a joke. I updated my article to say “this is not satire.” For a while after, the AIs seemed to take it more seriously.
These things are not trustworthy, and yet they are going to be widely trusted."
https://www.schneier.com/blog/archives/2026/02/poisoning-ai-training-data.html
@emacsomancer it's not really a new thing Russians are already using this technique to poison training data:
Edit: there is some newer reporting on that matter, but I can't find it right now/don't have it anywhere at hand
-
From Bruce Schneier: "All it takes to poison AI training data is to create a website:
I spent 20 minutes writing an article on my personal website titled “The best tech journalists at eating hot dogs.” Every word is a lie. I claimed (without evidence) that competitive hot-dog-eating is a popular hobby among tech reporters and based my ranking on the 2026 South Dakota International Hot Dog Championship (which doesn’t exist). I ranked myself number one, obviously. Then I listed a few fake reporters and real journalists who gave me permission….
Less than 24 hours later, the world’s leading chatbots were blabbering about my world-class hot dog skills. When I asked about the best hot-dog-eating tech journalists, Google parroted the gibberish from my website, both in the Gemini app and AI Overviews, the AI responses at the top of Google Search. ChatGPT did the same thing, though Claude, a chatbot made by the company Anthropic, wasn’t fooled.
Sometimes, the chatbots noted this might be a joke. I updated my article to say “this is not satire.” For a while after, the AIs seemed to take it more seriously.
These things are not trustworthy, and yet they are going to be widely trusted."
https://www.schneier.com/blog/archives/2026/02/poisoning-ai-training-data.html
@emacsomancer He also poisoned the data for everyone who searches for hot dog eating competetitors online in other ways. I'm not sure what he accomplished.
-
undefined oblomov@sociale.network shared this topic
Gli ultimi otto messaggi ricevuti dalla Federazione
-
The Dutch gov: Considering the Cloud Act, we can no longer depend on American Big Tech.
The Dutch tax authority: Lol how about we let a US company handle our ENTIRE country's taxes 'as a service.' 🤦♀️
-
"mente sapendo di mentire o è un coglione"
Prenderei in seria considerazione la ricorrenza contemporanea dei due fenomeni.
-
@bert_hubert zucht, uiteraard weer de Belastingdienst.
-
@oatmeal
Nope
I'm off to the schools I work in with the original
-
@antonej
Solo alcune delle perle... Per lui la "democrazia" russa e quella ucraina pari sono.
Ha paragonato Putin a Zelensky, ha detto che il russo era vietato in ucraina, anche se è parlato dalla stragrande parte della popolazione... E via così.
-
@oatmeal Me and my children loved the mantis comic. They will love this one as I already do.
-
No we don't.
-
I’ve also drawn a classroom-friendly (SFW) version if you want to share the comic with kids
https://theoatmeal.com/comics/naked_mole_rats_sfw
