LLMs are spam generators.

[yacc143@mastodon.social]

But yes, if you generate text with an LLM, you generally get text that is basically a "probable" text, based upon the training and the hyperparameters. And let's be honest, people that rely upon the training data of an LLM as a "search engine data corpus", they do have a problem.
🙅

[yacc143@mastodon.social]

@cstross Well, they are statistical models of language. To be precise, the corpus they were trained on, and there are a number of other aspects, like representation, what cleanups were applied.

Yes, one way to use these models, the one that seems to fascinate most people (but IMHO not necessarily the most useful, depending upon what one wants to achieve), is to complete a prefix with a plausible ending, statistically speaking.

Generally, they are a compelling NLP technology.

[victimofsimony@infosec.exchange]

@cstross
@ViennaMike
@ireneista

As a #PublicPolicy wonk I'm already remembering how different our common law legal systems have become in just three centuries. Over here they're still worried about the anti-porn #Censorship case where the #SupremeCourt refused to define what they were censoring. They famously said, "I know it when I see it," grumble grumble prurient use in commerce, harrumph typical opinion of ordinary firmness, followed by thirty years of chipping off corners without admitting they were sort of making shit up.

[viennamike@mastodon.social]

@davidgerard @cstross The article you posted included a very useful study. However, contrary to the article, that study is NOT the only study on the topic. Other, far larger, more comprehensive, studies show measured, meaningful productivity improvements. You asked for measurements, so here they are: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4945566 and https://www.youtube.com/watch?v=tbDDYKRFjhk The Stanford study found that the average productivity boost is significant (~20%) but some teams see productivity decrease.

[avadeaux@mastodonsweden.se]

@cstross I understand what you mean, but I’d say they’re simply databases. Spam generator is one of their many uses.

[davidgerard@circumstances.run]

@ViennaMike @cstross your paper's key metric is pull requests, which is addressed in the link you answered but didn't read (one jira leaves, three jiras enter). but thanks anyway.

(over and over i see this from AI guys, rebuttals that ignore the question asked)

[carpetbomberz@mastodon.online]

@oisin @cstross -via Wikipedia.org

"collisional cascading, or ablation cascade" especially given the training data and the Model Collapse predicted when they run out of "clean data" and start to be trained on synthetic `made up` data.

[thepowernap@mefi.social]

@ludicity @dragonfrog @cstross @ViennaMike

I'm kinda betting on this, but it doesn't mean the next couple of years won't be painful.

[thepowernap@mefi.social]

@NefariousCelt @dragonfrog @cstross @ViennaMike

95% agree. I'm currently working an LLM project that is interfacing with something a touch more than home automation. Aside from fine tuning the model and putting constraints on the output (think https://github.com/dottxt-ai/outlines), you can use existing approved software as validation of llm output.

All of this requires in-depth knowlege, patience and exaustive testing.

[nefariouscelt@mastodon.scot]

@ThePowerNap @dragonfrog @cstross @ViennaMike I suspect I am coming from a very risk averse standpoint. The “summarise this” is what scares me with LLM marketing. I have been burned by poor software requirement specifications. Language simplification can do that in multiple sectors. But yeah mapping natural language to a fixed set of goals, that’s fine. Trust AI with my life, no. Sudden random steering inputs WTAF stelantis.

[talin@mastodon.social]

@kupac @cstross It took a few decades after Gutenberg before our cognitive immune systems adapted to the reality of cheaply mass-produced books.

However, I'm suspicious of historical argument by analogy, so I'll leave it at that.

[viennamike@mastodon.social]

@leadegroot @dragonfrog @ThePowerNap
I can certainly see a bubble bursting, but I'm not sure that Phind is a sign. Hard to compete with those with billions to develop core models, and not going to hit any profit home runs reselling access to other's models.

[speakertomanagers@wandering.shop]

@cstross
The Turing Test will not fail an LLM for the same reason humans are ready to accept them as persons: humans evolved with a hardcoded behavior to perceive any language-using process as a cognitive entity and create a mental model of it. Rejecting that behavior takes a metacognitive ability that has not been taught to most people, so the problem isn’t rampant stupidity (look at how many clearly intelligent AI researchers fail this test) but rather ignorance, sometimes wilful, granted.

[rysiek@mstdn.social]

@cstross we have to find ways to do something useful with spam generators otherwise we might lose the social permission to generate more spam!

[tanepiper@tane.codes]

@cstross 3 years ago I came up with https://github.com/tanepiper/Stochastic-Parrot as a way to test the then new OpenAI APIs - after a month it became very clear to me that all someone needs is $100 and a GitHub pipeline and you could quite easily create a spam farm. No need to hire pesky humans for $1 a day - it was probably the first job taken by AI

[datenwolf@chaos.social]

@SpeakerToManagers @cstross

This. And it doesn't help that a huge portion of postmodern philosophy is built on the ideas of the linguistic turn.

I never felt comfortable with this line of thinking and if nothing else, I appreciate the developments of and around LLMs to finally putting some hard and lasting nails into the coffin of the idea of linguistic process having correspondence with cognition.

[mrjunge@raru.re]

@ViennaMike @cstross Just going empirically, if you have to go back in and do massive clean-up of the output that is supposed to represent the end-state of a product, it’s probably spam or at the very least spam-like.

[rozie@mastodon.online]

@cstross I believe we had "spam" long before LLMs, and it was generated just fine. Blame computers!

[keev@sunny.garden]

@cstross

https://www.theguardian.com/technology/2026/jan/24/latest-chatgpt-model-uses-elon-musks-grokipedia-as-source-tests-reveal

[viennamike@mastodon.social]

@mrjunge If you have to spend almost as much time, or more, cleaning up an LLM-assisted product as you originally saved, I agree. LLM's can certainly generate spam or spam-like output at times. But to claim that that's ALL it does is not correct. One company found that initially, their software code writing gains were eaten up by quality control rework, but that after learning how to better use the tools, when to use them, and when not to use them, production went up 30% with no loss of quality.