LLMs are spam generators.

[davidgerard@circumstances.run]

@cstross @VictimOfSimony LLM code's real trick is to induce gambling addiction in its suckers, who will swear it works great just one more prompt bro https://pivot-to-ai.com/2025/06/05/generative-ai-runs-on-gambling-addiction-just-one-more-prompt-bro/

[david_chisnall@infosec.exchange]

@cstross @NefariousCelt

A product where the target demographic is Kevin J Anderson.

[leadegroot@bne.social]

@dragonfrog @ThePowerNap @cstross @ViennaMike i have occasionally been using an AI tool called Phind. It was basically a “help with coding” tool. It recently rewrote itself completely into a generic AI tool… and then a month later closed.

I cant help but wonder… Is this the start of the great collapse?

[mdm@mcnamarii.town]

@oisin @cstross Oh my goodness, that's a good way to phrase that.

[nefariouscelt@mastodon.scot]

@david_chisnall @cstross I think I get this reference. (Confused Captain America face) But I was leaning into how an author needs to be told how they do not understand his own oeuvre. Something AI would excel at.

[nefariouscelt@mastodon.scot]

@ThePowerNap @dragonfrog @cstross @ViennaMike The best purpose of a LLM system is to provide a natural language interface to a complex but not mission critical system such as home automation, media libraries and switchboards.

Machine learning on the other hand should lean into seeing without eyes (is a dramatic jump in intensity cancer?)

Both of these cases should be powered through a single IEC C13 socket and not some gW data centre.

[rdp@notpickard.com]

@mighty_orbot @cstross

[talin@mastodon.social]

@cstross Because so many people boosted my post, I want to explore this topic in more detail.

First, a thought experiment: suppose we had cheaply scalable mind control, even if partially effective (LLMs are not mind control but they touch some of the same wires). Democracy would end, since the controllers could sway elections at will. The so-called free market would instantly become a command economy. Science, and any other activity requiring independent thought, would be a dead letter.

[talin@mastodon.social]

@cstross In the book _Rainbows End_, Vinge proposes a form of mind control called "YGBM" technology, which stands for "You Gotta Believe Me", a way to get affected individuals to accept arbitrary propositions as truth.

What this has in common with LLMs is technologically enhanced plausibility.

[cstross@wandering.shop]

@talin I was going to mention "Rainbows End"!

[talin@mastodon.social]

@cstross Now, we know from the earliest days of machine learning that algorithms are capable of exploiting any loophole in their fitness function with seemingly godlike competence.

Unfortunately, we can't make a fitness function for intelligence, truthfulness, or integrity. What we can do, at great expense, is make a fitness function for plausibility.

When an LLM goes through its reinforcement learning, it's behavior is rewarded based on whether some human reviewer believed the result.

[talin@mastodon.social]

@cstross As LLMs improve, they will get better at chasing their fitness function, that is, at convincing people that what they say is true. This is what LLMs have in common with YGBM.

Those who know history know that the invention of cheap printing sparked of centuries of bloody religious conflict. In this regard, LLMs are like "hold my beer".

[cstross@wandering.shop]

@Npars01 Yep, 💯

[highlandlawyer@mastodon.social]

@Npars01 @cstross
Spam making software also assists the nazis too.
"The ideal subject of totalitarian rule is not the convinced Nazi or the convinced Communist, but people for whom the distinction between fact and fiction (i.e., the reality of experience) and the distinction between true and false (i.e., the standards of thought) no longer exist." ~ Hannah Arendt

[yacc143@mastodon.social]

But yes, if you generate text with an LLM, you generally get text that is basically a "probable" text, based upon the training and the hyperparameters. And let's be honest, people that rely upon the training data of an LLM as a "search engine data corpus", they do have a problem.
🙅

[yacc143@mastodon.social]

@cstross Well, they are statistical models of language. To be precise, the corpus they were trained on, and there are a number of other aspects, like representation, what cleanups were applied.

Yes, one way to use these models, the one that seems to fascinate most people (but IMHO not necessarily the most useful, depending upon what one wants to achieve), is to complete a prefix with a plausible ending, statistically speaking.

Generally, they are a compelling NLP technology.

[victimofsimony@infosec.exchange]

@cstross
@ViennaMike
@ireneista

As a #PublicPolicy wonk I'm already remembering how different our common law legal systems have become in just three centuries. Over here they're still worried about the anti-porn #Censorship case where the #SupremeCourt refused to define what they were censoring. They famously said, "I know it when I see it," grumble grumble prurient use in commerce, harrumph typical opinion of ordinary firmness, followed by thirty years of chipping off corners without admitting they were sort of making shit up.

[viennamike@mastodon.social]

@davidgerard @cstross The article you posted included a very useful study. However, contrary to the article, that study is NOT the only study on the topic. Other, far larger, more comprehensive, studies show measured, meaningful productivity improvements. You asked for measurements, so here they are: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4945566 and https://www.youtube.com/watch?v=tbDDYKRFjhk The Stanford study found that the average productivity boost is significant (~20%) but some teams see productivity decrease.

[avadeaux@mastodonsweden.se]

@cstross I understand what you mean, but I’d say they’re simply databases. Spam generator is one of their many uses.

[davidgerard@circumstances.run]

@ViennaMike @cstross your paper's key metric is pull requests, which is addressed in the link you answered but didn't read (one jira leaves, three jiras enter). but thanks anyway.

(over and over i see this from AI guys, rebuttals that ignore the question asked)