Salta al contenuto

Piero Bosio Social Web Site Personale Logo Fediverso

Social Forum federato con il resto del mondo. Non contano le istanze, contano le persone

I'm giving the Marvell ConnectX cards a second go, as in, a second round of purchases.

Senza categoria
4 2 0
  • I'm giving the Marvell ConnectX cards a second go, as in, a second round of purchases. I triple checked these are Ethernet capable. All I want for now is to feel the speed of 40gbps.

    Maybe later I'll fool around with InfiniBand. I'm looking for a cheap managed switch < $100.

  • I'm giving the Marvell ConnectX cards a second go, as in, a second round of purchases. I triple checked these are Ethernet capable. All I want for now is to feel the speed of 40gbps.

    Maybe later I'll fool around with InfiniBand. I'm looking for a cheap managed switch < $100.

    @lymenzies hmmm, Mellanox ConnectX that is... Marvel also makes RDMA NICs but they are not InfiniBand. All Mellanox CX series (don't bother with CX-3, driver is ancient and power efficiency is bad, CX-4 and newer are good)... those are EN compat (either native or via slowness with encapsulated IPoIB).

    I still have eight CX3 QDR-IB Nics — sitting new in a box, same with CX4 100G... they gotta go. I've moved my homelab mostly over to CX5, Qlogic with RDMA, and some Intel (meh) to keep up with specific features (really, CX-6/7 is ideal, but that's a tangent).

    40G switch... various options under $100

    - Dell PowerSwitch S6000 32-Port 40GbE QSFP+
    - Cisco Nexus N9K-C9396PX 48x 1/10G SFP+ 6x40GB QSFP
    - MELLANOX 851-0167-01 8-PORT 40GBPS

  • @lymenzies hmmm, Mellanox ConnectX that is... Marvel also makes RDMA NICs but they are not InfiniBand. All Mellanox CX series (don't bother with CX-3, driver is ancient and power efficiency is bad, CX-4 and newer are good)... those are EN compat (either native or via slowness with encapsulated IPoIB).

    I still have eight CX3 QDR-IB Nics — sitting new in a box, same with CX4 100G... they gotta go. I've moved my homelab mostly over to CX5, Qlogic with RDMA, and some Intel (meh) to keep up with specific features (really, CX-6/7 is ideal, but that's a tangent).

    40G switch... various options under $100

    - Dell PowerSwitch S6000 32-Port 40GbE QSFP+
    - Cisco Nexus N9K-C9396PX 48x 1/10G SFP+ 6x40GB QSFP
    - MELLANOX 851-0167-01 8-PORT 40GBPS

    @winterschon

    I don't know why I wrote Marvell, must have had them on my mind.

    The price point on the ConnectX-3 cards is too tempting, $25 for the pair. I'd like to see if it's worth the trouble, then upgrade from there.

    While I probably do need to raise the minimum I'm willing to pay to get something useful, I have learned a lot from cast-off hardware over the years

    I appreciate your recommendations. I'll look into the Mellanox since it's not large and not power hungry.

  • @winterschon

    I don't know why I wrote Marvell, must have had them on my mind.

    The price point on the ConnectX-3 cards is too tempting, $25 for the pair. I'd like to see if it's worth the trouble, then upgrade from there.

    While I probably do need to raise the minimum I'm willing to pay to get something useful, I have learned a lot from cast-off hardware over the years

    I appreciate your recommendations. I'll look into the Mellanox since it's not large and not power hungry.

    @lymenzies ack, the cx3 prices are very good. however, if you want somewhere between 6-8 ConnectX-3 VPI (ie: they do 40G IB and 10G IPoIB) which are like-new in boxes with their cable sets, for free — just cover USPS priority shipping cost (~$15 approx, continental USA).

    I learned a lot with used hardware, and really wouldn't have had the opportunity to get to where I am now without that approach. so to spread the sharing-is-caring ethical approach, you are welcome to own them! right now they're hanging out in my hardware stockpile inventory with no intended use. originally I had two of those Mellanox switches for a mini cluster, but I left them at my desk at Fastly just before covid — then they disappeared. cute little blue boxes, very loud fans.

    in similar news, I am also giving away other hardware, so if you need other stuff just let me know. if you want low latency more so than RDMA protocols, I have a triple box of Solarflare 10G (see attached), which are formerly very popular with the HFT and distributed computing architectures where PTP time-code syncing is critical for transaction processing without collisions.


Gli ultimi otto messaggi ricevuti dalla Federazione
Post suggeriti