@glyph Did you quote post something?
-
@glyph Worth noting - these machines also depend on some rather recent physics to even be plausible. So, there's also a layer of physics, material science, etc. academics at mostly universities, scattered across the world, mostly funded by public spending on sciences, to even get to the point the engineers could try to build something that works at all on this scale.
@miss_rodent yeah the video is partially a story about that part of the process, or at least a window into it. apropos of the point in my linked thread, even if LLMs could do this kind of reasoning (they can't) we are a thousand years and dozens of general breakthroughs away from a context window large enough to even hold the required prerequisites to start reasoning out these problems even in just the AI tech itself
-
@miss_rodent yeah the video is partially a story about that part of the process, or at least a window into it. apropos of the point in my linked thread, even if LLMs could do this kind of reasoning (they can't) we are a thousand years and dozens of general breakthroughs away from a context window large enough to even hold the required prerequisites to start reasoning out these problems even in just the AI tech itself
@glyph Oh, yeah, LLMs are ... idk, this hype wave jsut makes me think of ELIZA (1966 simulated therapist), really.
With about the same chance of achieving sapience. -
@glyph Oh, yeah, LLMs are ... idk, this hype wave jsut makes me think of ELIZA (1966 simulated therapist), really.
With about the same chance of achieving sapience.@glyph Complete with ELIZA effect....
https://en.wikipedia.org/wiki/ELIZA_effectThis just feels like one of the dumbest 'those who don't learn from history are doomed to repeat it' situations I've been alive for.
-
@glyph yeaaaah
-
-
-
@glyph @mcc then it becomes potentially easier. You would have to source some of the chemicals, which may not be produced in NA anymore but eh. That is probably manageable.
Note that if you Manhattan project it, you may be able to produce a few batch but you will lack the repeatable good yield. That needs experience that need time.
-
@glyph DUDE Thank you for linking this: https://theoutpost.ai/news-story/major-study-reveals-ai-benchmarks-may-be-misleading-casting-doubt-on-reported-capabilities-21513/
I've been angry about yet another "look, the benchmark says this LLM can do {x} amazing thing!" (with spurrious claims and lots of caveats) for a full day.
-
@glyph I really like your point about scaling down. Even when I was in undergrad, there were already pieces about how the long free lunch was finally over, but damned if the strategy of relying on hardware progress for all software improvements isn't entirely unsustainable.
-
@glyph great read, always a delight when people make well-argued claims for not trying to go for unreasonable scales!
-
-
@glyph I really like your point about scaling down. Even when I was in undergrad, there were already pieces about how the long free lunch was finally over, but damned if the strategy of relying on hardware progress for all software improvements isn't entirely unsustainable.
-
-
@ireneista @cthos @xgranade this isn't exactly a "cheerful" thought, but it's also not horribly grim: I think we already saw it break down in 2020, and we saw both how brittle it is (nobody had enough slack in their supply chain to actually weather the disruption without exposing catastrophic delays to customers) but also its resilience (customers were super mad, alternate pathways DID come online in less than a year)
-
@glyph I see a pretty clear parallel to the middle period of, and tail end of the 19th century, where a lot of the "easy" discoveries had been, well... discovered... and the rate of progress in the sciences and inventions slowed until there was some significant breakthrough (the steam engine, in the first part of the century, and then electricity, oil, and oceanic telegraph lines, and later the linotype, breaking the mid-century lul and speeding advances up to WWI again)
-
@glyph I see a pretty clear parallel to the middle period of, and tail end of the 19th century, where a lot of the "easy" discoveries had been, well... discovered... and the rate of progress in the sciences and inventions slowed until there was some significant breakthrough (the steam engine, in the first part of the century, and then electricity, oil, and oceanic telegraph lines, and later the linotype, breaking the mid-century lul and speeding advances up to WWI again)
@glyph not to say we'll have another linotype or telegraph; more that I think we're running out of "easy" parts for progress with computers, so unless we get something new through material science, physics, chemistry, something like that, it's likely that all we have left available with current understanding is the hard stuff.
-
@glyph maaaaaan I just started reading and I am ALREADY MAD
about your rudely accurate observation we are now ONE QUARTER INTO THE CENTURY
🕸️🧓🏻🪦
-
@glyph not to say we'll have another linotype or telegraph; more that I think we're running out of "easy" parts for progress with computers, so unless we get something new through material science, physics, chemistry, something like that, it's likely that all we have left available with current understanding is the hard stuff.
@glyph Maybe quantum computing will unlock a whole new set of easy things (I'm... not confident in that, but, it could do. It's at least a lot faster for certain sorts of things, in the rare cases we can figure out how to actually implement something useful.)
But, there could also just... not be "easy" parts left. -
@glyph maaaaaan I just started reading and I am ALREADY MAD
about your rudely accurate observation we are now ONE QUARTER INTO THE CENTURY
🕸️🧓🏻🪦
@bitprophet you're the first one to catch that (I guarantee it hurt more to write than it does to read :))
-
@glyph Maybe quantum computing will unlock a whole new set of easy things (I'm... not confident in that, but, it could do. It's at least a lot faster for certain sorts of things, in the rare cases we can figure out how to actually implement something useful.)
But, there could also just... not be "easy" parts left.@miss_rodent the expert I would go to ask about that is @xgranade and I am pretty confident that she would not be bullish on this particular likelihood any time soon