from https://www.anthropic.com/research/AI-assistance-coding-skills
Uncategorized
13
Posts
10
Posters
0
Views
-
from https://www.anthropic.com/research/AI-assistance-coding-skills
> We found that using AI assistance led to a statistically significant decrease in mastery. On a quiz that covered concepts they’d used just a few minutes before, participants in the AI group scored 17% lower than those who coded by hand, or the equivalent of nearly two letter grades. Using AI sped up the task slightly, but this didn’t reach the threshold of statistical significance.
-
from https://www.anthropic.com/research/AI-assistance-coding-skills
> We found that using AI assistance led to a statistically significant decrease in mastery. On a quiz that covered concepts they’d used just a few minutes before, participants in the AI group scored 17% lower than those who coded by hand, or the equivalent of nearly two letter grades. Using AI sped up the task slightly, but this didn’t reach the threshold of statistical significance.
@jacqueline and yet, they continue to push garbage like Claude.
-
from https://www.anthropic.com/research/AI-assistance-coding-skills
> We found that using AI assistance led to a statistically significant decrease in mastery. On a quiz that covered concepts they’d used just a few minutes before, participants in the AI group scored 17% lower than those who coded by hand, or the equivalent of nearly two letter grades. Using AI sped up the task slightly, but this didn’t reach the threshold of statistical significance.
@jacqueline I’m so surprised they published this
-
@jacqueline and yet, they continue to push garbage like Claude.
@emma @jacqueline I'm honestly surprised they did publish it, that's the equivalent of the oil industry finding about climate change in the 70s but they choose to hide it
-
@emma @jacqueline I'm honestly surprised they did publish it, that's the equivalent of the oil industry finding about climate change in the 70s but they choose to hide it
I suppose it’s possible that some insiders see that deflating the bubble relatively gently is preferable to it popping?
-
from https://www.anthropic.com/research/AI-assistance-coding-skills
> We found that using AI assistance led to a statistically significant decrease in mastery. On a quiz that covered concepts they’d used just a few minutes before, participants in the AI group scored 17% lower than those who coded by hand, or the equivalent of nearly two letter grades. Using AI sped up the task slightly, but this didn’t reach the threshold of statistical significance.
@jacqueline the only logical conclusion to this research is that they should stop existing.
Otherwise it's just "oh hey we have a mega tool that fucks the human mind and we're going to continue pushing it to everyone because money"
Despicable.
-
@jacqueline and yet, they continue to push garbage like Claude.
@emma @jacqueline the thinking that created openai (and anthropic being an openai offshoot that thinks it's more closely aligned with their "original" mission) is that AGI is inevitable, and if it is it should be them creating it with an enlightened guiding hand (or something like this)
the more i read about anthropic the less i believe their "constitutional AI" thing is a bit, they have a large number of true believers that think they need to save humanity from rogue AI
-
@jacqueline I’m so surprised they published this
-
from https://www.anthropic.com/research/AI-assistance-coding-skills
> We found that using AI assistance led to a statistically significant decrease in mastery. On a quiz that covered concepts they’d used just a few minutes before, participants in the AI group scored 17% lower than those who coded by hand, or the equivalent of nearly two letter grades. Using AI sped up the task slightly, but this didn’t reach the threshold of statistical significance.
There is a nuance in that article. Good to see that some software engineers use the AI as a tool to gauge ideas instead of fully relying on it.
And that they are thinking how to improve their models for improved quality.
-
I suppose it’s possible that some insiders see that deflating the bubble relatively gently is preferable to it popping?
@urlyman
I'm a stellar engineer by PhD biologist standards, and I'm a midling coder by senior software engineers.
Which is to say that I can't get jack done in 35 minutes, especially not with a new library.
As far as Claude goes, opus is by far the best in the entire market, bar none.
But yeah, I can't compose a new problem in a way it understands in 35 minutes.
If something takes 30 minutes and your first instinct is "ask a bot" you're addicted and not thinking.
@gkrnours @emma @jacqueline -
@dnavinci@genomic.social coming into my mentions to defend CSAM slop bots is not welcome.
-
@dnavinci@genomic.social coming into my mentions to defend CSAM slop bots is not welcome.
@emma
I'm commenting specifically about the limited utility of code helper tools, slop-bots or deterministic.You think that sounds like defense?
-
@urlyman
I'm a stellar engineer by PhD biologist standards, and I'm a midling coder by senior software engineers.
Which is to say that I can't get jack done in 35 minutes, especially not with a new library.
As far as Claude goes, opus is by far the best in the entire market, bar none.
But yeah, I can't compose a new problem in a way it understands in 35 minutes.
If something takes 30 minutes and your first instinct is "ask a bot" you're addicted and not thinking.
@gkrnours @emma @jacqueline@dnavinci @urlyman @emma @jacqueline as a web developer with 9 years of professional experience, I need multiple hours to grasp a new library (unless it has great documentation) and I can't get anything done in 35 minutes. Most of the time, I can't get anything decent done unless I sleep on it.
-
undefined mora@mastodon.uno shared this topic on
Gli ultimi otto messaggi ricevuti dalla Federazione
-
@aeva I had a pharmacist (in California) that filled on day 27/30, though at some point I had to switch and my current one only does 29/30. Still rough but not "literally zero margin" rough.
More than one pharmacist has claimed to me that their refill policy was law, when in reality it was some combination of insurance policy, corporate policy, and their own ego (they have some discretion and this is one of few places where they actually get to use it). Walgreens was the worst about that.
-
@aeva thanks for the med reminder
-
@jonbro this little guy is great :O do you have any advice for working on smaller figures like this?
-
-
-
@fayden Beautiful!
-
@MonkeyPanic@nerdculture.de That makes sense, and I think it points to a real limitation of the study. The participants were all late learners, people who didn't grow up with the foreign language at home. If you acquired both languages early and tied them to lived experience and emotion, the “foreign language effect” probably looks very different. Your Korean triggering emotions despite the vocabulary gap seems like evidence of exactly that.
-
@aeva ah yes...
sending love from the equally cruel UK...
