I am convinced we are on the verge of the first "AI agent worm".

[cwebber@social.coop]

I am convinced we are on the verge of the first "AI agent worm". This looks like the closest hint of it, though it isn't it quite itself: an attack on a PR agent that got it to set up to install openclaw with full access on 4k machines https://grith.ai/blog/clinejection-when-your-ai-tool-installs-another

But, the agents installed weren't given instructions to *do* anything yet.

Soon they will be. And when they are, the havoc will be massive. Unlike traditional worms, where you're looking for the typically byte-for-byte identical worm embedded in the system, an agent worm can do different, nondeterministic things on every install, and carry out a global action.

I suspect we're months away from seeing the first agent worm, *if* that. There may already be some happening right now in FOSS projects, undetected.

[cwebber@social.coop]

I wrote a blogpost on this: "The first AI agent worm is months away, if that" https://dustycloud.org/blog/the-first-ai-agent-worm-is-months-away-if-that/

People who are using LLM agents for their coding, review systems, etc will probably be the first ones hit. But once agents start installing agents into other systems, we could be off to the races.

[cwebber@social.coop]

Here's another way to put it: if those using AI agents to codegen / review are the *initialization vectors*, we now also have a significant computing public health reason to discourage the use of these tools.

Not that I think it will. But I'm convinced this is how patient zero will happen.

[cwebber@social.coop]

I know some people are thinking "well pulling off this kind of thing, it would have to be controlled with intent of a human actor"

It doesn't have to be.

1. A human could *kick off* such a process, and then it runs away from them.
2. It wouldn't even require a specific prompt to kick off a worm. There's enough scifi out there for this to be something any one of the barely-monitored openclaw agents could determine it should do.

Whether it's kicked off by a human explicitly or a stray agent, it doesn't require "intentionality". Biological viruses don't have interiority / intentionality, and yet are major threats that reproduce and adapt.

[csepp@merveilles.town]

@cwebber This is making me more worried about Vorta's Claude workflows.
Backup software that handles highly sensitive data would be a prime target for such a supply chain attack.

[orca@nya.one]

@cwebber@social.coop Morris II 😑
https://sites.google.com/view/compromptmized

[cwebber@social.coop]

The interesting thing about the AI worm being imminent thing is this is the first time where I have said something about AI where most of the well-informed sides of anti-ai and pro-ai friends I have both fully agree with me. If you are paying attention enough, you can see that all the pieces are falling in place.

In fact, the biggest debate is whether this has happened already, and we just haven't seen proof of it yet. I don't know. Given how long things like the xz attack have sat undetected, and given how much chaos of computation is happening in datacenter usage right now, I wouldn't doubt it.

[cwebber@social.coop]

The question is not if, it's when. I am dead serious that we will have never seen a cybersecurity incident like this before, because it can self-mutate at a pace much faster than random mutation in physical viruses.

Workshopped a phrase for it a bit with @quintessence last night: "evolution through artificially intelligent design" of malicious behaviors.

The only solution I can think of once this happens is to shut down network access, particularly to AI service providers, and roll back to software distros based on software that came out a year older and patch our way back up against known CVEs while we try to sort everything out.

[dlakelan@mastodon.sdf.org]

@cwebber
#guix time-machine to the rescue.

@quintessence