AI models don’t really 'get' the BSDs.

[stefano@mastodon.bsd.cafe]

AI models don’t really 'get' the BSDs. As a result, they often provide incomplete, imprecise, or flat-out wrong answers by defaulting to Linux paradigms. When it comes to illumos-based systems, they just completely lose the plot.

This is becoming a serious issue for the BSDs and illumos ecosystems. We are seeing entire websites flooded with AI-generated tutorials and guides that are totally incorrect. Most people don't realize this; they follow the instructions, fail, and then assume that the BSDs doesn't work well or are 'unstable' because they have supposedly changed since the guide was written.

Luckily, some people eventually find my blog, reach out, and finally understand what's actually going on. Others, unfortunately, end up on major social sites or comments, claiming that these systems are broken.

In 2026, one of our greatest challenges will be teaching people how to vet their sources and filter information.
And I see this as a very, very uphill battle.

#IT #SysAdmin #FreeBSD #NetBSD #OpenBSD #illumos #News #UnderstandingText #Disinformation

[clf@mastodon.bsd.cafe]

@stefano when it's not full of linuxisms it hallucinates features from other BSDs because it doesn't see every BSD OS as different. We do things the old way on BSD. This is a good thing.

Similarly, even when using PyTorch, they often suggest deprecated features or options that may exist in another library but not in this one. I don't see AI being very useful for development in general other than having something to brainstorm with.

[michel@mastodon.bsd.cafe]

@stefano that's why I will go back to FreeBSD to host my site and maybe the wife's and I will try with all my forces to document it all and make it public

[david@mastodon.crossfamilyweb.com]

@stefano I think we can combat this with more and better blog posts and targeted documentation. The AI walks our docs, if we're better, it is better. I plan on starting doing this myself. I am working on getting my homelab in better shape, then I plan on posting about all of the stuff I do more.

[stefano@mastodon.bsd.cafe]

@david that's a good plan

[paradegrotesque@mastodon.sdf.org]

@stefano

Is that a problem... Or an opportunity?

Hear me out: what if we all put everywhere "do not trust AI with [BSD | Illumos], use these sites instead"?

This would actually redirect people to valid sources of information and puncture the AI nonsense.

[stefano@mastodon.bsd.cafe]

@ParadeGrotesque good point of view.

[nk@sns.neonka.info]

@stefano Or maybe the info about BSDs is so complex, fragmented, and outdated that it cannot be automatically processed?

[stefano@mastodon.bsd.cafe]

@nk complex: no, I don't think so
Fragmented? Maybe - but their man pages and the handbooks like the FreeBSD one are extremely useful and easy to find.
Outdated? It's quite uncommon that something related to a BSD becomes (totally) outdated

There are simply less resources about them, so less documentation to train their models

[nuintari@mastodon.bsd.cafe]

@stefano Come on Sisyphus, we got this!

[pitrh@mastodon.social]

@stefano It's possible newer versions fare better, but this https://nxdomain.no/~peter/chatgpt_writes_pf.conf.html piece from mid-2023 had ChatGPT failing to produce anything even close to valid for pf.conf.

The results are classic GIGO but with lots of extra hardware thrown at it for no good reason.

[skylark13@mastodon.gamedev.place]

@stefano I know you know this, just reiterating to further depress myself and others I guess: This is sadly not specific to BSD or any one piece of knowledge. AI models don't 'get' anything. They have no concept of knowing anything, and SEO pages generated by AI containing incorrect information are polluting the training sets of AI models, so nothing good can come of it.

The video linked from this thread eloquently makes that point, among many others.

https://akkoma.xn--pikabl-0xa.se/objects/34faab37-a455-44f7-abeb-d0b937cd479a

[teajaygrey@snac.bsd.cafe]

IMHO, it's kind of a ray of hope that BSDs are (for now at least) relatively immune to A"I" BS?

CC: @stefano@bsd.cafe

[skylark13@mastodon.gamedev.place]

@teajaygrey @stefano

You mean BSD code itself? I hope there are strong policies to prevent it (for contributors to follow), otherwise sooner or later it will happen. Related: https://mastodon.gamedev.place/@skylark13/115842600214466205

But even if it doesn't affect (infect) the code, it is a huge net-negative for users trying to find information to troubleshoot issues by themselves.

[teajaygrey@snac.bsd.cafe]

The policy postulation is ponderous! I dunno if or what BSD projects might adopt with regard to such things, but it's interesting food for thought!

The net-negative perspective kind of goes with the territory of being involved with less popular projects, but I've already been in these sorts of XKCD situations: https://xkcd.com/979/

Though it can be vaguely hilarious to google (back before that became its own A"I" minefield) something, and find a post, where I was the only one banging my head against an issue (or providing some vague solution, that others apparently hadn't improved upon in the ensuing years).

CC: @stefano@bsd.cafe

[stefano@mastodon.bsd.cafe]

@teajaygrey @skylark13 sometimes you find out that person in 2003 were...you, but you forgot how you dealt with it