This is bad.
Uncategorized
149
Posts
33
Posters
0
Views
-
This is bad. This is very, very bad.
I'm not trying to pick on Python here, I pick it because Python is something I'm actively using, and so I have a vested interest in the project *not* being AI-vulnerable.
But it's not good, chat. It's very far from good, in fact.
[edited to add: see two addendums below, they're important context]
@xgranade
Looks like about a dozen commits reference Claude in the commit message or authorship. Note to others: you have to do a bit more work than just "grep -i claude", as there are humans that are named Claude too. But the Anthropic Claude is definitely in there, which raises some legal questions about copyright I think. -
@xgranade
Looks like about a dozen commits reference Claude in the commit message or authorship. Note to others: you have to do a bit more work than just "grep -i claude", as there are humans that are named Claude too. But the Anthropic Claude is definitely in there, which raises some legal questions about copyright I think.@joelle Yeah, at least one list of commits that I saw also grepped for anthropic.com, but that's a very good point.
Anyway, with respect to it being about a dozen, yes, this is fairly limited in its impact so far, but what worries me is that Python itself is in the blast radius of Anthropic's efforts to enclose OSS at *all*, and with seemingly no processes in place to limit that exposure.
It's part of why I've taken to referring to this kind of problem as "AI-vulnerable."
-
As an addendum, I'm using Python as an example here because it's near and dear to my heart. This is not "Python in particular is exceptionally bad," this is "a very bad thing has been happening in OSS *in general* and Python is now in that blast radius, which makes it harder for me to personally ignore."
As a second addendum, since this has come up in several reply threads, the number of commits is limited so far, and doesn't date back past December 5, 2025 so far as I'm aware of.
The Python-specific part of that broader problem is, at least to my mind, that there's not a mechanism that I see for limiting that exposure to those commits, to preventing further and more expansive commits in the future.
-
@xgranade @ireneista Huh, maybe even https://brython.info/ ?
I thought that project had fizzled out, but no, it supports 3.14
@xgranade @ireneista Well, MicroPython has an advantage in that it's still written in C, and it is therefore possible to port C extension modules to it, though there aren't that many which really support it yet
-
This is bad. This is very, very bad.
I'm not trying to pick on Python here, I pick it because Python is something I'm actively using, and so I have a vested interest in the project *not* being AI-vulnerable.
But it's not good, chat. It's very far from good, in fact.
[edited to add: see two addendums below, they're important context]
@xgranade
Huh, back to perl then I guess? :( -
@SnoopJ @theorangetheme No, absolutely. I see this as the leading indicator rather than the damage itself, if that makes sense?
I keep using the term "AI-vulnerable" to try and point to that there isn't necessarily an actual direct impact, so much as a dramatically increased vulnerability surface area.
@xgranade @SnoopJ @theorangetheme I'm curious--how is Claude directly able to do commits? Why is it not "Claude on behalf of Dave Alvarado"? I understand somebody ran an agent against the code base, but someBODY ran the agent against the code base. Somebody prompted it saying "go find security vulnerabilities in Python".
It sure would be nice to know who, not just "Claude".
-
@ireneista @glyph I hope it doesn't, if only because I want to be focusing on my specfic and screenplays, but if it does come to that, I very very much so appreciate your support. ♥
@xgranade @ireneista @glyph *quickly scribbles out a short story involving a fantastical run for the PSF*
-
@xgranade @SnoopJ @theorangetheme I'm curious--how is Claude directly able to do commits? Why is it not "Claude on behalf of Dave Alvarado"? I understand somebody ran an agent against the code base, but someBODY ran the agent against the code base. Somebody prompted it saying "go find security vulnerabilities in Python".
It sure would be nice to know who, not just "Claude".
@dave @xgranade @theorangetheme I'm not sure I really understand the question. In the commits above, it's a co-author rather than a primary author.
But in the general case, it's able to do it by running the command that adds a commit, in a context where the configured name/email for use with `git` will be the name/email associated with the model (the author metadata includes the specific model as well)
Creating such commits without indication of the human involvement (wherever it originated, since Rube Goldberg contraptions are all the rage right now) is IMO unethical but far from unimaginable.
-
@xgranade
Huh, back to perl then I guess? :(@srtcd424 If that's what's useful to you? But I don't personally recommend moving away from Python, nor do I think that's an effective tactic for dealing with the problem.
As mentioned, this is a broad problem in OSS *in general*, and Python is now in the blast radius of that problem. Trying to create a dependency path that doesn't include any AI-vulnerable code is very difficult right now.
-
@dave @xgranade @theorangetheme I'm not sure I really understand the question. In the commits above, it's a co-author rather than a primary author.
But in the general case, it's able to do it by running the command that adds a commit, in a context where the configured name/email for use with `git` will be the name/email associated with the model (the author metadata includes the specific model as well)
Creating such commits without indication of the human involvement (wherever it originated, since Rube Goldberg contraptions are all the rage right now) is IMO unethical but far from unimaginable.
@SnoopJ @xgranade @theorangetheme gotcha. On second look, I see that you were grepping, I misunderstood what I was reading there.
As I've thought about it some more, I think I'm standing by my take. IMO the fact that you contributed with Claude is barely more interesting than the fact that you contributed with VS Code. I think that "oh I used an LLM/Agent" is not a defense against, well, anything.
-
@SnoopJ @xgranade @theorangetheme gotcha. On second look, I see that you were grepping, I misunderstood what I was reading there.
As I've thought about it some more, I think I'm standing by my take. IMO the fact that you contributed with Claude is barely more interesting than the fact that you contributed with VS Code. I think that "oh I used an LLM/Agent" is not a defense against, well, anything.
@SnoopJ @xgranade @theorangetheme I don't think we should be personifying LLMs by calling them "co-authors". Claude didn't author, it recursively autocompleted.
-
@SnoopJ @xgranade @theorangetheme gotcha. On second look, I see that you were grepping, I misunderstood what I was reading there.
As I've thought about it some more, I think I'm standing by my take. IMO the fact that you contributed with Claude is barely more interesting than the fact that you contributed with VS Code. I think that "oh I used an LLM/Agent" is not a defense against, well, anything.
@dave @SnoopJ @theorangetheme It's not interesting, but it is important as part of understanding the vulnerability surface introduced by that code. There are many things about code that are simultaneously boring as fuck and also critically important.
-
@SnoopJ @xgranade @theorangetheme I don't think we should be personifying LLMs by calling them "co-authors". Claude didn't author, it recursively autocompleted.
@dave @SnoopJ @theorangetheme I don't even disagree, but that's the signal that Claude gives us, and there's no Git metadata for "this code was extruded by $x slop machine."
-
@astraluma @xgranade If you search for 'claude' you can find the commits where Claude is a "co-author" https://github.com/search?q=repo%3Apython%2Fcpython+claude&type=commits
-
@astraluma @xgranade If you search for 'claude' you can find the commits where Claude is a "co-author" https://github.com/search?q=repo%3Apython%2Fcpython+claude&type=commits
@nausicaa @astraluma As @joelle pointed out, Claude is also a name that real people have. @SnoopJ's cantrip is going to be less susceptible to false positives by filtering on "anthropic.com" as well.
-
@srtcd424 If that's what's useful to you? But I don't personally recommend moving away from Python, nor do I think that's an effective tactic for dealing with the problem.
As mentioned, this is a broad problem in OSS *in general*, and Python is now in the blast radius of that problem. Trying to create a dependency path that doesn't include any AI-vulnerable code is very difficult right now.
@xgranade
Yeah, sorry, it was dark humour. I'm honestly terrified about where all this heading :( Not personally a python fan probably due to my vintage but it's used for a frightening proportion of software I rely on. -
@dave @SnoopJ @theorangetheme It's not interesting, but it is important as part of understanding the vulnerability surface introduced by that code. There are many things about code that are simultaneously boring as fuck and also critically important.
@xgranade @SnoopJ @theorangetheme yeah I've been thinking about that, and I'm not sure I agree. The outputted code is the outputted code. "y = x + 1" doesn't gain additional attack surface because Claude autocompleted it.
I think there are all sorts of *human* exploits that can happen and are happening, but those are all based on our laziness checking Claude's work, not Claude's output itself. Things like maintainers going "Jesus take the wheel" when Claude writes commits because it's easier
-
@xgranade
Yeah, sorry, it was dark humour. I'm honestly terrified about where all this heading :( Not personally a python fan probably due to my vintage but it's used for a frightening proportion of software I rely on.@srtcd424 No need to apologize, I just want to be clear about my own views on this rather than inadvertently implying criticism of Python *in particular* that I neither mean nor want to make.
-
@xgranade @SnoopJ @theorangetheme yeah I've been thinking about that, and I'm not sure I agree. The outputted code is the outputted code. "y = x + 1" doesn't gain additional attack surface because Claude autocompleted it.
I think there are all sorts of *human* exploits that can happen and are happening, but those are all based on our laziness checking Claude's work, not Claude's output itself. Things like maintainers going "Jesus take the wheel" when Claude writes commits because it's easier
@xgranade @SnoopJ @theorangetheme please don't read any of this as my endorsement of slop, I can't stand it. I'm just trying to pick apart how code autocompleted by Claude is different from the moral hazard of trusting Claude in the first place.
-
@xgranade @SnoopJ @theorangetheme yeah I've been thinking about that, and I'm not sure I agree. The outputted code is the outputted code. "y = x + 1" doesn't gain additional attack surface because Claude autocompleted it.
I think there are all sorts of *human* exploits that can happen and are happening, but those are all based on our laziness checking Claude's work, not Claude's output itself. Things like maintainers going "Jesus take the wheel" when Claude writes commits because it's easier
@dave @SnoopJ @theorangetheme My views here are complicated, but let me try and give a somewhat accurate condensed version?
First, to your `y = x + 1` example, if the code is simple enough, that vulnerability can be mitigated by human review — the problem is still there, I contend, but was contained by review. The problem is that humans *suck* at scanning for that kind of problem. Take the TSA looking for guns in x-ray scans... they keep failing at that, and incredibly badly.
Feed RSS
Gli ultimi otto messaggi ricevuti dalla Federazione
-
The Dutch gov: Considering the Cloud Act, we can no longer depend on American Big Tech.
The Dutch tax authority: Lol how about we let a US company handle our ENTIRE country's taxes 'as a service.' 🤦♀️
-
"mente sapendo di mentire o è un coglione"
Prenderei in seria considerazione la ricorrenza contemporanea dei due fenomeni.
-
@bert_hubert zucht, uiteraard weer de Belastingdienst.
-
@oatmeal
Nope
I'm off to the schools I work in with the original
-
@antonej
Solo alcune delle perle... Per lui la "democrazia" russa e quella ucraina pari sono.
Ha paragonato Putin a Zelensky, ha detto che il russo era vietato in ucraina, anche se è parlato dalla stragrande parte della popolazione... E via così.
-
@oatmeal Me and my children loved the mantis comic. They will love this one as I already do.
-
No we don't.
-
I’ve also drawn a classroom-friendly (SFW) version if you want to share the comic with kids
https://theoatmeal.com/comics/naked_mole_rats_sfw
