AI models don’t really 'get' the BSDs.

[dexter@bsd.network]

@stefano One takeaway from the OpenZFS Summit was that dated, canonical documentation is critical, so Reddit posts don’t become the source of truth.

[skylark13@mastodon.gamedev.place]

@stefano I know you know this, just reiterating to further depress myself and others I guess: This is sadly not specific to BSD or any one piece of knowledge. AI models don't 'get' anything. They have no concept of knowing anything, and SEO pages generated by AI containing incorrect information are polluting the training sets of AI models, so nothing good can come of it.

The video linked from this thread eloquently makes that point, among many others.

https://akkoma.xn--pikabl-0xa.se/objects/34faab37-a455-44f7-abeb-d0b937cd479a

[teajaygrey@snac.bsd.cafe]

IMHO, it's kind of a ray of hope that BSDs are (for now at least) relatively immune to A"I" BS?

CC: @stefano@bsd.cafe

[skylark13@mastodon.gamedev.place]

@teajaygrey @stefano

You mean BSD code itself? I hope there are strong policies to prevent it (for contributors to follow), otherwise sooner or later it will happen. Related: https://mastodon.gamedev.place/@skylark13/115842600214466205

But even if it doesn't affect (infect) the code, it is a huge net-negative for users trying to find information to troubleshoot issues by themselves.

[teajaygrey@snac.bsd.cafe]

The policy postulation is ponderous! I dunno if or what BSD projects might adopt with regard to such things, but it's interesting food for thought!

The net-negative perspective kind of goes with the territory of being involved with less popular projects, but I've already been in these sorts of XKCD situations: https://xkcd.com/979/

Though it can be vaguely hilarious to google (back before that became its own A"I" minefield) something, and find a post, where I was the only one banging my head against an issue (or providing some vague solution, that others apparently hadn't improved upon in the ensuing years).

CC: @stefano@bsd.cafe

[mirabilos@toot.mirbsd.org]

@stefano I think it’s a good thing. Weeds out the slop.

[stefano@mastodon.bsd.cafe]

Questo post è eliminato!

[morgant@mastodon.social]

@stefano @teajaygrey @skylark13 Been there! "Wow, this person did all the _exact_ same investigation and troubleshooting that I… oh. It's me."

Also, remember how frustrating #FilterBubbling in search engines became (long before this AI shit)?!

"Why can't I find anything _not_ on a StackOverflow-powered site?!"

— and —

"I googled _exactly_ the same phrase you said you did and I'm _not_ finding the sources you found!"

[light@noc.social]

@NoFlexZone
Depends on what it is for the person using BSD. "The point" of doing something is never the same for everyone.
@stefano

[peteorrall@mastodon.bsd.cafe]

@stefano Revisiting this post. While I've largely used #ChatGPT and some #Gemini when researching #Linux and #FreeBSD stuff, I'd say #AI has come a long way. Are they perfect? No. Is it a replacement for man pages and the Handbook? No. Can they provide more info, context, and real world examples when the official #documentation falls short? That's a resounding yes.

It's been a while since I've seen AI give utterly incorrect info relating to FreeBSD. I am far from being the FreeBSD god I want to be and even I recognized the incorrect info at those particular times. But this was also a good 1.5 years ago.

I can't speak for #Illumos as I don't use it.

[jackemled@furry.engineer]

@glent @david @stefano It's all marketing. A guy you've never met but who has some charisma opens up his trench coat, shows you this thing he says will make doing things easier, & says the first hit is free. Documentation doesn't promise that, it just tells you how to do the thing the correct way. People don't want the right way, they want the easy way. They don't want to learn, they just want it over with.

They trust the charismatic guy in the trench coat because he tells them myths about the work almost doing itself if they would just try the thing he's offering to them. After the first hit, it's all over. The victim is locked in because "This is the easy way, isn't it? All that stuff sounds so complicated, I'm afraid to even try it without using this. It feels so fast & easy to do it this way even though it makes me feel bad too.", then they get used to it, their standards have been lowered, & they try it with other things, then it's too late.

[lispi314@udongein.xyz]

@stefano@mastodon.bsd.cafe The funny thing is a lot of the BSDs have fairly comprehensive manuals available from the official site.

[ianthetechie@fosstodon.org]

@peteorrall @stefano same. It’s improved a lot and will continue to do so. I remember getting total rubbish from ChatGPT 4. But these days major models do a pretty good job.

My main use is also gap filling for more esoteric stuff. The great thing is you can verify stuff ;) just like everything you read on the internet.

Of course people will believe whatever Claude says. Just like they believe whatever they see on TV. Or some shady website that has great SEO.

[ianthetechie@fosstodon.org]

@peteorrall @stefano it important though that we acknowledge it and help others to see it too. That said I do find it to be a very useful tool.

[tusharhero@mathstodon.xyz]

@stefano this also happens with GNU Guix, if you ask it about guix, it keeps trying to run nix commands.

#guix

[zstg@fedia.social]

@stefano oh my 😞. Could you share some examples?

[stefano@mastodon.bsd.cafe]

@zstg in general, linuxisms everywhere. But this is slowly changing.

[jperkin@federate.me.uk]

@peteorrall @stefano As you tagged #illumos: in my experience Claude does a fantastic job of fixing hard problems using native illumos tooling. It knows how to talk DTrace, mdb, ptools, etc. I can give it a /goal and it will generally go away and do it with little prompting. Usually the main area I have to help it along with is analyzing flamegraphs.

That said it is disheartening to see them drop illumos/*BSD/etc support, so hopefully others will step up here so we can continue agentic probes.

[speaktrap@mastodon.com.pl]

@nk @stefano Have you ever used a BSD?

[stefano@mastodon.bsd.cafe]

@nk outdated fragmented and complex? BSD documentation? Quite the opposite.