AI models don’t really 'get' the BSDs.

[stefano@mastodon.bsd.cafe]

AI models don’t really 'get' the BSDs. As a result, they often provide incomplete, imprecise, or flat-out wrong answers by defaulting to Linux paradigms. When it comes to illumos-based systems, they just completely lose the plot.

This is becoming a serious issue for the BSDs and illumos ecosystems. We are seeing entire websites flooded with AI-generated tutorials and guides that are totally incorrect. Most people don't realize this; they follow the instructions, fail, and then assume that the BSDs doesn't work well or are 'unstable' because they have supposedly changed since the guide was written.

Luckily, some people eventually find my blog, reach out, and finally understand what's actually going on. Others, unfortunately, end up on major social sites or comments, claiming that these systems are broken.

In 2026, one of our greatest challenges will be teaching people how to vet their sources and filter information.
And I see this as a very, very uphill battle.

#IT #SysAdmin #FreeBSD #NetBSD #OpenBSD #illumos #News #UnderstandingText #Disinformation

[clf@mastodon.bsd.cafe]

@stefano when it's not full of linuxisms it hallucinates features from other BSDs because it doesn't see every BSD OS as different. We do things the old way on BSD. This is a good thing.

Similarly, even when using PyTorch, they often suggest deprecated features or options that may exist in another library but not in this one. I don't see AI being very useful for development in general other than having something to brainstorm with.

[michel@mastodon.bsd.cafe]

@stefano that's why I will go back to FreeBSD to host my site and maybe the wife's and I will try with all my forces to document it all and make it public

[david@mastodon.crossfamilyweb.com]

@stefano I think we can combat this with more and better blog posts and targeted documentation. The AI walks our docs, if we're better, it is better. I plan on starting doing this myself. I am working on getting my homelab in better shape, then I plan on posting about all of the stuff I do more.

[stefano@mastodon.bsd.cafe]

@david that's a good plan

[paradegrotesque@mastodon.sdf.org]

@stefano

Is that a problem... Or an opportunity?

Hear me out: what if we all put everywhere "do not trust AI with [BSD | Illumos], use these sites instead"?

This would actually redirect people to valid sources of information and puncture the AI nonsense.

[stefano@mastodon.bsd.cafe]

@ParadeGrotesque good point of view.

[nk@sns.neonka.info]

@stefano Or maybe the info about BSDs is so complex, fragmented, and outdated that it cannot be automatically processed?

[stefano@mastodon.bsd.cafe]

@nk complex: no, I don't think so
Fragmented? Maybe - but their man pages and the handbooks like the FreeBSD one are extremely useful and easy to find.
Outdated? It's quite uncommon that something related to a BSD becomes (totally) outdated

There are simply less resources about them, so less documentation to train their models

[nuintari@mastodon.bsd.cafe]

@stefano Come on Sisyphus, we got this!

[pitrh@mastodon.social]

@stefano It's possible newer versions fare better, but this https://nxdomain.no/~peter/chatgpt_writes_pf.conf.html piece from mid-2023 had ChatGPT failing to produce anything even close to valid for pf.conf.

The results are classic GIGO but with lots of extra hardware thrown at it for no good reason.