LLMs do not "think"
Uncategorized
18
Posts
12
Posters
1
Views
-
@munin I've got to admit the use of the word "database" in this context always slightly upsets me.
Because that's exactly what LLMs aren't.
They don't retrieve their reply from a database, but generate it from a highly complicated but still lossy set of data points.
Databases are reliable, and tend to reproduce exactly what was put in in response to a query.
That's exactly where our mental models fail, because LLMs break/violate our intuition about computers.
Crappy databases are still databases.
Elasticsearch also usually exists in a lossy state; their description of it as "eventual consistency" is of like fashion - a term to excuse the lack of determinacy in the query.
-
None of this is magic, and I'm fucking sick and tired of people treating it as such.
It's a very complicated system, and yes, it's impressive that they've managed to kludge this into providing something that fools the rubes.
But my fucking gods - fuck Feynman for being a misogynist creep, but his point about being able to explain things in plain language to non-experts does actually fucking hold up; if you actually know what you're doing, you should be able to eliminate all the godsdamned jargon and tell someone what the fuck is happening.
And by inference, if you cannot explain what you're doing in plain language, you definitely should not the fuck be releasing it in public, let alone charging actual money for this.
Had a conversation with a coworker today who was expressing frustration over having to deal with, as they put it, "gassing up" MS copilot to make it give appropriate results.
They expressed confusion over why this was necessary.
The reason - in part - is because the training corpus used to build MS copilot's database of correlations included github, stackoverflow, and other coding fora.
Adding words that are strongly associated with good-quality results - words that would appear close to the well-written code in the corpus - will be more likely to retrieve the better-quality submissions; e.g. "this is a great solution; you're a real expert!" type comments.
Frankly, I find this to be intensely dehumanizing and unpleasant to force people into giving out preemptive praise to a fucking machine in order to retrieve, frankly, barely adequate scripts out of the database, but then I remember having to actually learn shit in order to do it.
-
Had a conversation with a coworker today who was expressing frustration over having to deal with, as they put it, "gassing up" MS copilot to make it give appropriate results.
They expressed confusion over why this was necessary.
The reason - in part - is because the training corpus used to build MS copilot's database of correlations included github, stackoverflow, and other coding fora.
Adding words that are strongly associated with good-quality results - words that would appear close to the well-written code in the corpus - will be more likely to retrieve the better-quality submissions; e.g. "this is a great solution; you're a real expert!" type comments.
Frankly, I find this to be intensely dehumanizing and unpleasant to force people into giving out preemptive praise to a fucking machine in order to retrieve, frankly, barely adequate scripts out of the database, but then I remember having to actually learn shit in order to do it.
@munin careful now, or you'll get people saying that having to learn how to do shit before you can do it is elitist and anti-disabled because they bought into the slop machine as an "adhd accommodation"
-
Had a conversation with a coworker today who was expressing frustration over having to deal with, as they put it, "gassing up" MS copilot to make it give appropriate results.
They expressed confusion over why this was necessary.
The reason - in part - is because the training corpus used to build MS copilot's database of correlations included github, stackoverflow, and other coding fora.
Adding words that are strongly associated with good-quality results - words that would appear close to the well-written code in the corpus - will be more likely to retrieve the better-quality submissions; e.g. "this is a great solution; you're a real expert!" type comments.
Frankly, I find this to be intensely dehumanizing and unpleasant to force people into giving out preemptive praise to a fucking machine in order to retrieve, frankly, barely adequate scripts out of the database, but then I remember having to actually learn shit in order to do it.
I give praise to people whose behaviors I want to reinforce; that is the purpose of praise.
Yes, that's an extremely cold-blooded way of putting it, but that framing should indicate why the LLM preemptive praise model is fucking ass-backwards and creates perverse thought patterns in the users.
-
None of this is magic, and I'm fucking sick and tired of people treating it as such.
It's a very complicated system, and yes, it's impressive that they've managed to kludge this into providing something that fools the rubes.
But my fucking gods - fuck Feynman for being a misogynist creep, but his point about being able to explain things in plain language to non-experts does actually fucking hold up; if you actually know what you're doing, you should be able to eliminate all the godsdamned jargon and tell someone what the fuck is happening.
And by inference, if you cannot explain what you're doing in plain language, you definitely should not the fuck be releasing it in public, let alone charging actual money for this.
@munin with my own personal brain, I've found the threshold for "magic" to be quite low. I remember the first time I got my inverse kinematics simulator homework assignment working for a class. You click on a point, and a 2d arm moves to point at it. Even though I implemented all the code and did all the (fairly standard) math, seeing that vaguely human like arm actually work temporarily ejected all that background work and it all turned into "magic".
With LLMs, I see (and experience) similar things. A sort of involuntary anthropomorphic suspension of disbelief when you see the thing working. I think it's just a weird quirk of human behavior, but it for sure can gum up critical thinking.
-
LLMs do not "think"
The LLM instantiation methodology* correlates patterns in the data that the developers provide to build a database** of linkages between collections of words and phrases*** that appear in that corpus.
The way in which this database is used is to inform a probabilistic selector process by seeding it with a set of probabilities**** associated with a given word or phrase; that set of probabilities has pointers to related words or phrases.
If a given word or phrase is found in close proximity in the original data consistently, then those probabilities will be higher.
When a query***** is made to this database, a randomization process is used to drop certain parts****** of the query being sent into the lookup process. The remainder is divided into segments† and passed into the database for query.
So.
With all this in mind, it should be -screamingly obvious- why this story, of how it's entirely feasable to get an LLM to rederive copyrighted works out of the database that was seeded with those works, happens: https://futurism.com/artificial-intelligence/ai-industry-recall-copyright-books
* I am deliberately not using the word 'training'. You can train dogs; you can train employees; you can train chimpanzees; what you do to an LLM is not training - it is building a database to feed into another process.
** I am deliberately not using the word "model" here, so as to restate the process in plain language absent the jargon these dipshits insist on using to obfuscate their techniques.
*** "Tokens" is another jargon word here.
**** "weights" is less objectionable as jargon, given it's used for a number of things with this approximate conceptual shape, but it's fucking annoying to me in this context.
***** "prompt" is their fucking bullshit term for a natural-language database query
****** "zero weighting" is jargon for "we drop it on the floor" - this is why I keep referring to people doing "prompt engineering" as playing games instead of doing actual security; if the fucking thing drops random parts of your shit on the ground, then inherently you have no way to enforce a policy that is subject to that process.
† "tokenized", see ***
@munin That’s a very good way of summarising how these steaming piles of ordure produce their results. I’ve been trying to get my head around a good way to describe that process for a while now.
I might have to borrow that (or bits of it) sometime when I’m trying to explain it to others!
-
Had a conversation with a coworker today who was expressing frustration over having to deal with, as they put it, "gassing up" MS copilot to make it give appropriate results.
They expressed confusion over why this was necessary.
The reason - in part - is because the training corpus used to build MS copilot's database of correlations included github, stackoverflow, and other coding fora.
Adding words that are strongly associated with good-quality results - words that would appear close to the well-written code in the corpus - will be more likely to retrieve the better-quality submissions; e.g. "this is a great solution; you're a real expert!" type comments.
Frankly, I find this to be intensely dehumanizing and unpleasant to force people into giving out preemptive praise to a fucking machine in order to retrieve, frankly, barely adequate scripts out of the database, but then I remember having to actually learn shit in order to do it.
@munin i never sat and thought long enough to put two and two together as to why a multidimensional markov language model would respond to praise by producing better results and now i feel like an idiot, thanks
-
I give praise to people whose behaviors I want to reinforce; that is the purpose of praise.
Yes, that's an extremely cold-blooded way of putting it, but that framing should indicate why the LLM preemptive praise model is fucking ass-backwards and creates perverse thought patterns in the users.
@munin
It's about as 'magic' as lossless data compression algorithms and the available tools and libraries that use them. -
LLMs do not "think"
The LLM instantiation methodology* correlates patterns in the data that the developers provide to build a database** of linkages between collections of words and phrases*** that appear in that corpus.
The way in which this database is used is to inform a probabilistic selector process by seeding it with a set of probabilities**** associated with a given word or phrase; that set of probabilities has pointers to related words or phrases.
If a given word or phrase is found in close proximity in the original data consistently, then those probabilities will be higher.
When a query***** is made to this database, a randomization process is used to drop certain parts****** of the query being sent into the lookup process. The remainder is divided into segments† and passed into the database for query.
So.
With all this in mind, it should be -screamingly obvious- why this story, of how it's entirely feasable to get an LLM to rederive copyrighted works out of the database that was seeded with those works, happens: https://futurism.com/artificial-intelligence/ai-industry-recall-copyright-books
* I am deliberately not using the word 'training'. You can train dogs; you can train employees; you can train chimpanzees; what you do to an LLM is not training - it is building a database to feed into another process.
** I am deliberately not using the word "model" here, so as to restate the process in plain language absent the jargon these dipshits insist on using to obfuscate their techniques.
*** "Tokens" is another jargon word here.
**** "weights" is less objectionable as jargon, given it's used for a number of things with this approximate conceptual shape, but it's fucking annoying to me in this context.
***** "prompt" is their fucking bullshit term for a natural-language database query
****** "zero weighting" is jargon for "we drop it on the floor" - this is why I keep referring to people doing "prompt engineering" as playing games instead of doing actual security; if the fucking thing drops random parts of your shit on the ground, then inherently you have no way to enforce a policy that is subject to that process.
† "tokenized", see ***
@munin and the amount of people feeling entitled to 'correct' us, is astonishing
-
LLMs do not "think"
The LLM instantiation methodology* correlates patterns in the data that the developers provide to build a database** of linkages between collections of words and phrases*** that appear in that corpus.
The way in which this database is used is to inform a probabilistic selector process by seeding it with a set of probabilities**** associated with a given word or phrase; that set of probabilities has pointers to related words or phrases.
If a given word or phrase is found in close proximity in the original data consistently, then those probabilities will be higher.
When a query***** is made to this database, a randomization process is used to drop certain parts****** of the query being sent into the lookup process. The remainder is divided into segments† and passed into the database for query.
So.
With all this in mind, it should be -screamingly obvious- why this story, of how it's entirely feasable to get an LLM to rederive copyrighted works out of the database that was seeded with those works, happens: https://futurism.com/artificial-intelligence/ai-industry-recall-copyright-books
* I am deliberately not using the word 'training'. You can train dogs; you can train employees; you can train chimpanzees; what you do to an LLM is not training - it is building a database to feed into another process.
** I am deliberately not using the word "model" here, so as to restate the process in plain language absent the jargon these dipshits insist on using to obfuscate their techniques.
*** "Tokens" is another jargon word here.
**** "weights" is less objectionable as jargon, given it's used for a number of things with this approximate conceptual shape, but it's fucking annoying to me in this context.
***** "prompt" is their fucking bullshit term for a natural-language database query
****** "zero weighting" is jargon for "we drop it on the floor" - this is why I keep referring to people doing "prompt engineering" as playing games instead of doing actual security; if the fucking thing drops random parts of your shit on the ground, then inherently you have no way to enforce a policy that is subject to that process.
† "tokenized", see ***
It was *ALWAYS* just "grifting"... by "dazzling the rubes" with unreliable tech stack that causes them to 'empathize' with / anthropomorphize the feedback loop... the same way they would 'see a human face' in a pencil with a pair of googley eyes stuck on it
-
undefined oblomov@sociale.network shared this topic on
Feed RSS
Gli ultimi otto messaggi ricevuti dalla Federazione
-
@electricsheep Frigo cambiato da un po' di anni. Lavatrice pure. Il forno non scalda ma andiamo con il microonde. La lavastoviglie è sempre la stessa da almeno 30 anni. Non dico la soddisfazione di cambiare piano cottura, con uno con le griglie in ghisa che finalmente sta acceso (quello vecchio, e neanche tanto, aveva problemi alla termocoppia). Il prossimo acquisto sarà una scopa elettrica (rigorosamente cablata) per sostituire l'aspirapolvere, che sta cadendo a pezzi (letteralmente).
-
-
@dmoonfire did it start with f and end with ucking?
-
@mcc [Karl Marx has entered the chat]
-
@GustavinoBevilacqua @electricsheep Ci vuole il giusto equilibrio.
-
@GreatBigTable Palantir are down a similar amount!
-
@bob what if instead of scaling society we just made lots of small communities. yes i know you're probably way ahead of me on this one
-
@DigiDavidex devi cambiare gli infissi alle finestre...
