@cwebber@social.coop exactly. AI can only exist because of human-made content from the Internet, and because humans have been talking with each other over the internet for roughly 30 years at this point, and most of these conversations have been archived somewhere.
So what would happen if this archive disappears? Maybe not the physical medium of the Internet, but the fact of humans speaking with each other over the Internet and recording their conversations, that can disappear. Less and less often people are asking questions of other humans in forums like Reddit or Stack Overflow, and more and more often humans ask questions to AI apps where the conversation cannot be used as more training data as this would create a destructive feedback loop in the signal that the AI is built to predict, which can lead to the AI behaving erratically.
(I originally wrote the above on my blog post from last August, I'm saying this to not plagiarize myself.)
It is kind of like the Internet is a forest of natural content. AI companies have now cut down the forest without planting new trees. They have all the raw material now and are trying (and failing) to profit from it, but in 5 years that information may become out-of-date, will any humans be asking new questions of each other on platforms not mined for content by AI companies? If not, these LLMs will rapidly become more and more innacurate over time.