Skip to content

Piero Bosio Social Web Site Personale Logo Fediverso

Social Forum federato con il resto del mondo. Non contano le istanze, contano le persone

Lumen has sinkholed over 550 command and control servers for the Kimwolf botnet

Uncategorized
1 1 0

Gli ultimi otto messaggi ricevuti dalla Federazione
  • @mos_8502 nvidia has a $3999 computer, the DGX Spark, that is roughly the same speed and specs as the AMD option, in benchmarks, but the nvidia ecosystem is more mature. And, ASUS makes a mini PC with the same nvidia chipset and specs for $2999. So, if you need the nvidia level of compatibility, you have to spend more for 128GB. But, that's become less important in the past year or so as AMD has invested in AI tooling. So, for now, AMD is a bargain, relative to Apple or nvidia.

    read more

  • @vwbusguy @ids1024 My thinking is, longer term, for whatever use can be made of it, I would prefer a little box I can put on a shelf that sits on my local network and provides roughly the same interface as Claude or ChatGPT. Something that doesn’t suck down too much power.

    read more

  • @ids1024 @mos_8502 The worst thing is offloading GPU compute to system memory for a large model. It can be swapping on a spinner bad. The good news is it's unlikely an individual would really need to use the larger model versions.

    read more

  • @mos_8502 @vwbusguy As I understand, for large AI models you want a lot of VRAM (ideally more than even high end modern gaming GPUs). So that old GPU and workstation hardware wouldn't be especially helpful. (If you want it to be reasonably fast.)

    read more

  • @mos_8502 so, it continues to be cheapest to buy the investor-subsidized compute and GPU that OpenAI, Anthropic, Google, Microsoft, etc. want to provide. But, it does feel bad to me to trust in that or become dependent on that, especially since the models are all proprietary and I don't know what they're doing exactly or what they're doing with my data or that they'll do in the future.

    read more

  • @mos_8502 pytorch can use cpu. You'll just have to be a little more patient.

    read more

  • @mos_8502 GLM 4.7 can run in 205GB at the 2-bit quantization. Some of that can be system memory, so if you had a system with a couple of large GPUs and a ton of system RAM, you could run a very good open model...among the best, comparable to Sonnet 4.5. Still not Opus/Codex 5.2 level, but it'll write working code. https://unsloth.ai/docs/models/glm-4.7

    read more

  • @mos_8502 cheapest option a year or two ago was probably 4 3090s on a server motherboard.

    read more
Post suggeriti