Because not using AI tools for what they excel at will produce less secure code.

[filippo@abyssdomain.expert]

RE: https://wetdry.world/@16af93/115961732893013803

Because not using AI tools for what they excel at will produce less secure code.

For example, they are great at debugging (https://words.filippo.io/claude-debugging/), they can find real issues in code review, they know more math than me or most of my colleagues, and they can write static analyzers I would have never had the time to write myself.

[jfkimmes@social.tinycyber.space]

@filippo I think the "they can build [include any quality-increasing measure] I would never have had the time to write myself" is a very underappreciated effect of LLM-assisted development.

[jfkimmes@social.tinycyber.space]

@filippo It's sort of the opposite of vibe coding.

[canacar@ioc.exchange]

@filippo I am sorry, but a cryptographer saying something like "they know more math than me" only tells me that the cryptographer in question does not know how those things work. Please do not underestimate yourself or overestimate the capabilities of a text generator that happens to be have ingested tons of stolen human generated mathematical text that it stitches together (or quotes verbatim without attribution) to look like an answer.

[djspiewak@fosstodon.org]

@filippo @16af93 It’s almost as if they’re a tool like any other: applying that tool correctly creates better outcomes than if you ignore it, just as applying it incorrectly creates worse outcomes.

[filippo@abyssdomain.expert]

@canacar I know my capabilities (and their limits!) thank you very much, and your description suggests you have not seriously tried a state-of-the-art model for more than five minutes.

Load up Claude with Opus 4.5, ask it to reason about stuff you know the right answer for, and get back to me.

I am good at combinatorics/probabilities (IMO Bronze medal), and it still helped me do the analysis for the recent bruteforce of test vectors I did.

[16af93@wetdry.world]

@djspiewak @filippo a tool of ethically at best dubious nature, but then again the torment nexus is just a tool too

[filippo@abyssdomain.expert]

@16af93 @djspiewak ^ a social media post sent over the internet using a device made in China, each a tool of ethically dubious nature

[16af93@wetdry.world]

@filippo @djspiewak there is no ethical consumption under capitalism, but at least the phone that sends these posts has been chosen to reduce the harm caused, further a phone is nearly universially mandatory to use to participate in society - so not that much of a gotcha as you think

[djspiewak@fosstodon.org]

@16af93 @filippo There's definitely a lot of dubiousness going on, I agree. As someone who has about two and a half decades of OSS corpus which has been stolen and swirled around in the bowels of these things, I'm very uncomfortably familiar with the contradiction here. But my discomfort or lack thereof isn't going to change anything. I can disadvantage myself and those who rely on my production, but I can't really affect the problem at its root.

[16af93@wetdry.world]

@djspiewak @filippo usage gives legitimacy to ai, and legitimacy allows further exploitation and laundering of exploitation

personally these tools produce too much harm on every level to be given legitimacy (see the recent trends with ai psychosis, people being diluted by these overly agreeable models to the point of being emotionally disturbed by models changing their behavior (see the ai gf crowd), and curl needing to stop their bug bounty program because of ai swamping them.with bogus reports)

[canacar@ioc.exchange]

@filippo the "reasoning" is a series of RAG queries, which in turn are web searches or agent outputs that then get added to the context, with no additional component of "understanding" or "knowing" or "reasoning". Just text generation with more context which may or may not be correct. Yes, they are helpful if you can verify the output and they speed things up if you can easily identify and discard incorrect outputs

I am not a developer. I am on the other side, dealing with summaries devoid of content or originality and and increased workload because people think that these things are like a fellow developer that "knows" or "learned" something just because they did it correctly once.

In that, I support your effort pointing these tools to better patterns, but refuse to anthromorphize it.

[felixlinker@infosec.exchange]

@filippo Using AI to generate test cases for my current project, and it's just so valuable. Would have never done that on my own. Especially because the cost of setting up the entire testing pipeline is often quite high for me (remembering or learning a testing framework).

I use QuickCheck for Haskell, i.e., I test invariants on my code with random inputs. It's so nice to describe the expected invariants in natural language and have them almost ready to test.