Hey #freebsd #bhyve users!

[mwl@io.mwl.io]

Hey #freebsd #bhyve users!

I'm trying to test draid virtually for #openzfsmastery, because nobody's gonna give me a shelf of 60 disks to play with. I set everything up like:

disk35_name="disk35"
disk35_type="ahci-hd"
disk35_dev="sparse-zvol"

Seems that if I have 35 ahci-hd entries, disk0 through disk34, bhyve and freebsd works. At disk35, the host panics on boot.

Is this expected with bhyve? Or does FreeBSD need a special tweak with 36 disks?

[scalonnec@pouet.chapril.org]

@mwl @grahamperrin ?

[paradegrotesque@mastodon.sdf.org]

@mwl

Come on, man, just ask for a big honking disk shelf or NAS from your sponsors already!

I suspect you would get a brand new one 3 weeks later. I know I would throw a little bit of money your way.

(Just a little bit, as a treat)

[mwl@io.mwl.io]

@ParadeGrotesque

I already freeload sufficiently, thank you!

and I'm not gonna ask for expensive kit just to write one chapter.

[mwl@io.mwl.io]

@Tubsta ty!

I freely admit that nobody sane would do this, but here I am...

[polishdub@bsd.network]

@Tubsta Oh come clean, this limitation was put here just to throw @mwl for a loop! :D

[dch@bsd.network]

@mwl have you tried this with NVMe type as disk? s/virtio-blk/nvme/ for the type.

also maybe it runs out of “pci slots” try adding this:

ahci_device_limit="8"

I’ll try this tomorrow and see what comes up

[grahamperrin@mastodon.bsd.cafe]

@scalonnec I don't (can not) use bhyve on Kubuntu.

@mwl

[mwl@io.mwl.io]

@dch I have not, good idea! maybe virtio as well? Gonna play around some this weekend...

[ctuffli@mastodon.online]

@Tubsta @dch @mwl hopefully goes without saying, but yell at me if a virtual shelf of N NVMe disks in bhyve doesn’t work

[ctuffli@mastodon.online]

@Tubsta @dch @mwl bhyve "slot" specification in the max is bus:pcislot:function where bus is 0-255, pcislot is 0-31, and function is 0-7. If the maths don't fail me, that is around 64k-ish devices

[dexter@bsd.network]

@ctuffli @Tubsta @dch @mwl So, we talking bhyve or vm-bhyve?

I would not be surprised if vm-bhyve doesn’t assume that, “no one would ever want more than 8 devices”, or at least devices of one type.

Have you tried raw bhyve?

[drscriptt@oldbytes.space]

@mwl @ParadeGrotesque if we were close I have EMC Fibre Channel disks in shelf that I could loan to you.

[mwl@io.mwl.io]

@Tubsta @dch @dexter @ctuffli thank you!!!

time zones rock, I go to sleep and someone else fixes my problems. ;-)

[mwl@io.mwl.io]

@dexter @ctuffli @Tubsta @dch vm-bhyve, because I'm lazy

[mwl@io.mwl.io]

@Tubsta @dch @dexter @ctuffli

bhyve seems to work well, but yeah, those command lines are

[dexter@bsd.network]

@mwl @Tubsta @dch @ctuffli I am speaking purely from a diagnostics perspective. I believe, to their credit, their log shows the command that was executed. Which is kinda important, given the many “bhyve” bugs along the lines of “it’s treating my boot drive as if it is a CD-ROM drive …” #TrueStory!

So yeah. Share the underlying actual stuff please because very few vm-bhyve bugs are bhyve bugs.

[dexter@bsd.network]

@mwl @Tubsta @dch @ctuffli I mean like, I didn’t write an OpenBSD bug report when I found that typo in Absolute OpenBSD. In of course the one chapter you didn’t trust me to have in advance. Nosiree. THEN you tasked me with raising over $150K to date (USD no less, not that Looney stuff) for a conference.

But yeah sure, blame bhyve.

Until of course it comes time to write a book on the subject… ahem!

[mwl@io.mwl.io]

@dexter @Tubsta @dch @ctuffli

Sure, I ask if something's weird and file a bug.

But sometimes, I'm doing something actively daft. If bhyve-vm hangs trying to launch a host with 75 drives, my first instinct is to say "well, this would be because I'm a damn idiot trying to use 75 drives on one VM..."

[dexter@bsd.network]

@mwl @Tubsta @dch @ctuffli Bruh. bhyve now supports NUMA domains, because test ALL THE THINGS. 75 drives is like… a good start. All scaling issues should be explored to their unholy extremes.

U got this!