AI models don’t really 'get' the BSDs.

[stefano@mastodon.bsd.cafe]

AI models don’t really 'get' the BSDs. As a result, they often provide incomplete, imprecise, or flat-out wrong answers by defaulting to Linux paradigms. When it comes to illumos-based systems, they just completely lose the plot.

This is becoming a serious issue for the BSDs and illumos ecosystems. We are seeing entire websites flooded with AI-generated tutorials and guides that are totally incorrect. Most people don't realize this; they follow the instructions, fail, and then assume that the BSDs doesn't work well or are 'unstable' because they have supposedly changed since the guide was written.

Luckily, some people eventually find my blog, reach out, and finally understand what's actually going on. Others, unfortunately, end up on major social sites or comments, claiming that these systems are broken.

In 2026, one of our greatest challenges will be teaching people how to vet their sources and filter information.
And I see this as a very, very uphill battle.

#IT #SysAdmin #FreeBSD #NetBSD #OpenBSD #illumos #News #UnderstandingText #Disinformation

[clf@mastodon.bsd.cafe]

@stefano when it's not full of linuxisms it hallucinates features from other BSDs because it doesn't see every BSD OS as different. We do things the old way on BSD. This is a good thing.

Similarly, even when using PyTorch, they often suggest deprecated features or options that may exist in another library but not in this one. I don't see AI being very useful for development in general other than having something to brainstorm with.

[alexhaydock@infosec.exchange]

@stefano I’ve noticed this. They show similar problems with macOS because they hallucinate command line options and flags that exist in the GNU/Linux versions of tools but not in the macOS versions which I assume have more BSD heritage.

[fedops@fosstodon.org]

@stefano similar issue, though more subtle, for Linux as well. Most "ai"-generated answers are wrong in ways not obvious to people who don't know much about the subject (which is clearly the intended audience for this junk).

I've gone so far as telling my guys to not use chatgpt & ilk to find answers. That was after a couple of near misses due to BS command line arguments.

[michel@mastodon.bsd.cafe]

Questo post è eliminato!

[david@mastodon.crossfamilyweb.com]

@stefano I think we can combat this with more and better blog posts and targeted documentation. The AI walks our docs, if we're better, it is better. I plan on starting doing this myself. I am working on getting my homelab in better shape, then I plan on posting about all of the stuff I do more.

[stefano@mastodon.bsd.cafe]

Questo post è eliminato!

[paradegrotesque@mastodon.sdf.org]

@stefano

Is that a problem... Or an opportunity?

Hear me out: what if we all put everywhere "do not trust AI with [BSD | Illumos], use these sites instead"?

This would actually redirect people to valid sources of information and puncture the AI nonsense.

[stefano@mastodon.bsd.cafe]

Questo post è eliminato!

[jezcaudle@infosec.exchange]

@stefano This is a good thing. While the mindset thinks using AI - as in LLM’s - is okay, they won’t understand the BSD’s. When they are ready - mature enough to understand the BSD mindset - they will reject the LMM’s first and from that everything else will follow.

[nk@sns.neonka.info]

@stefano Or maybe the info about BSDs is so complex, fragmented, and outdated that it cannot be automatically processed?

[stefano@mastodon.bsd.cafe]

Questo post è eliminato!

[paradegrotesque@mastodon.sdf.org]

@TheDragon

Not really, because what I had in mind is linking to actual, real documentation from the projects.

For instance, for OpenBSD (which has the best documentations), link to man.openbsd.org and the FAQ, with a few explanations.

Are you trying to do this under NetBSD? Take a look at this section of the NetBSD Handbook, plus a few man pages on man.netbsd.org and some command examples, etc... etc...

No 'rings' needed. Include lots of warnings about AI first, of course.

@stefano

[tml@mementomori.social]

@stefano As somebody who was around back in the days when some BSD folks were arrogant "all Unixes are BSD, who cares about silly HP-SUX etc", there is a certain opportunity for Schadenfreude here. (Joke.)

[nuintari@mastodon.bsd.cafe]

Questo post è eliminato!

[pitrh@mastodon.social]

@stefano It's possible newer versions fare better, but this https://nxdomain.no/~peter/chatgpt_writes_pf.conf.html piece from mid-2023 had ChatGPT failing to produce anything even close to valid for pf.conf.

The results are classic GIGO but with lots of extra hardware thrown at it for no good reason.

[tivasyk@awawa.club]

@stefano@bsd.cafe «And I see this as a very, very uphill battle».

it's not even a battle; it's a bog that you're being pushed into while the battle happens elsewhere :-(

[denisbloodnok@mendeddrum.org]

@fedops @stefano BSD vs SYSV "killall" still a regular source of comedy? Plus ca change; I remember getting caught by that in the 90s.

[fedops@fosstodon.org]

@denisbloodnok they've moved beyond that in more sinister ways. If you're looking at a hash prompt 80% is just not good enough and reading man pages should be second nature if something exceeds your muscle memory. PFYs need that made clear.
@stefano

[tml@mementomori.social]

@stefano (Actually, now when I try to remember, I guess the HP-UX kernel *was* based on BSD, but then they wanted to hide that in userland and APIs and follow the "System V, consider it standard" mantra for corporate credibility. Or something.)