@lymenzies hmmm, Mellanox ConnectX that is... Marvel also makes RDMA NICs but they are not InfiniBand. All Mellanox CX series (don't bother with CX-3, driver is ancient and power efficiency is bad, CX-4 and newer are good)... those are EN compat (either native or via slowness with encapsulated IPoIB). I still have eight CX3 QDR-IB Nics — sitting new in a box, same with CX4 100G... they gotta go. I've moved my homelab mostly over to CX5, Qlogic with RDMA, and some Intel (meh) to keep up with specific features (really, CX-6/7 is ideal, but that's a tangent).40G switch... various options under $100- Dell PowerSwitch S6000 32-Port 40GbE QSFP+- Cisco Nexus N9K-C9396PX 48x 1/10G SFP+ 6x40GB QSFP- MELLANOX 851-0167-01 8-PORT 40GBPS