from https://www.anthropic.com/research/AI-assistance-coding-skills

[jacqueline@chaos.social]

from https://www.anthropic.com/research/AI-assistance-coding-skills

> We found that using AI assistance led to a statistically significant decrease in mastery. On a quiz that covered concepts they’d used just a few minutes before, participants in the AI group scored 17% lower than those who coded by hand, or the equivalent of nearly two letter grades. Using AI sped up the task slightly, but this didn’t reach the threshold of statistical significance.

[emma@orbital.horse]

@jacqueline and yet, they continue to push garbage like Claude.

[uint8_t@chaos.social]

@jacqueline I’m so surprised they published this

[gkrnours@mastodon.gamedev.place]

@emma @jacqueline I'm honestly surprised they did publish it, that's the equivalent of the oil industry finding about climate change in the 70s but they choose to hide it

[urlyman@mastodon.social]

@gkrnours @emma @jacqueline

I suppose it’s possible that some insiders see that deflating the bubble relatively gently is preferable to it popping?

[f4grx@chaos.social]

@jacqueline the only logical conclusion to this research is that they should stop existing.

Otherwise it's just "oh hey we have a mega tool that fucks the human mind and we're going to continue pushing it to everyone because money"

Despicable.

[aurelia@social.treehouse.systems]

@emma @jacqueline the thinking that created openai (and anthropic being an openai offshoot that thinks it's more closely aligned with their "original" mission) is that AGI is inevitable, and if it is it should be them creating it with an enlightened guiding hand (or something like this)

the more i read about anthropic the less i believe their "constitutional AI" thing is a bit, they have a large number of true believers that think they need to save humanity from rogue AI

[tully@cathode.church]

@uint8_t @jacqueline it's been snapshotted: https://web.archive.org/web/20260000000000*/https://www.anthropic.com/research/AI-assistance-coding-skills

[x_hace@mastodon.green]

@jacqueline

There is a nuance in that article. Good to see that some software engineers use the AI as a tool to gauge ideas instead of fully relying on it.

And that they are thinking how to improve their models for improved quality.

[dnavinci@genomic.social]

@urlyman
I'm a stellar engineer by PhD biologist standards, and I'm a midling coder by senior software engineers.
Which is to say that I can't get jack done in 35 minutes, especially not with a new library.
As far as Claude goes, opus is by far the best in the entire market, bar none.
But yeah, I can't compose a new problem in a way it understands in 35 minutes.
If something takes 30 minutes and your first instinct is "ask a bot" you're addicted and not thinking.
@gkrnours @emma @jacqueline

[emma@orbital.horse]

@dnavinci@genomic.social coming into my mentions to defend CSAM slop bots is not welcome.

[dnavinci@genomic.social]

@emma
I'm commenting specifically about the limited utility of code helper tools, slop-bots or deterministic.

You think that sounds like defense?

[gkrnours@mastodon.gamedev.place]

@dnavinci @urlyman @emma @jacqueline as a web developer with 9 years of professional experience, I need multiple hours to grasp a new library (unless it has great documentation) and I can't get anything done in 35 minutes. Most of the time, I can't get anything decent done unless I sleep on it.