What Comes After AGI?

(institute.deepmind.com)

172 points | by jdkee 1 day ago

67 comments

  • bsenftner 1 day ago
    Propaganda? We do not have AGI, and the claims we do are propaganda.

    AGI: artificial general intelligence is not a scaled up LLM, and those thinking it is are literally failed educations. They are failed because they cannot see the subtly that is what AI becomes when trained on human literature; it becomes an entirely different thing than any other form of trained algorithm. AI is a communications technology, which few to none seem to understand. AI is not automation, but it can be used to create automation with careful consideration, that is not taking place.

    I'm speaking in strong terms because we are literally fighting a war against stupidity and propaganda.

    AGI requires artificial comprehension which is a science that humanity does not have. When we have artificial comprehension there will be no need to train AIs beyond the kindergarten school basics, and then the AGI will figure it all out, all the rest, on its own. Human history and human literature will need to be learned in reference, but science and mathematics will require no training, beyond to learn the names of concepts we discovered earlier, so the AGI can discuss its understandings with humans and other AGI.

    • panarky 1 day ago
      > We do not have AGI, and the claims we do are propaganda.

      The linked post makes no such claim.

      > artificial general intelligence is not a scaled up LLM ... artificial comprehension ... kindergarten ...

      Your reasoning takes the form of "I define X to mean Y and Z, and since Y is impossible, X is impossible", which is trivially true within your definition.

      But not everyone shares your definition.

      A widely held definition of AGI is "a machine that can accomplish most tasks that humans do on computers as well as the median human".

      This definition focuses on concrete and observable results and not philosophical or metaphysical definitions of "intelligence" or "comprehension" or "consciousness".

      • RaftPeople 2 hours ago
        > A widely held definition of AGI is "a machine that can accomplish most tasks that humans do on computers as well as the median human".

        Probably the most critical task a human (and other animals) can do is to learn and build an internal model of the environment and abstract concepts, dynamically.

        I think the term "general" is a stretch until it can do that.

      • bsenftner 21 hours ago
        [flagged]
      • 5uhrhrtt 1 day ago
        [dead]
      • bigbadfeline 21 hours ago
        > This definition focuses on concrete and observable results and not...

        The parent of your comment very clearly focused on "concrete and observable results" too.

        > and not philosophical or metaphysical definitions of "intelligence" or "comprehension" or "consciousness"

        I don't see such "definitions" in the parent of your comment, these categories are neither defined nor are they necessary for definition of AGI there - they're only supposed to help the readers comprehend it. However, aversion to comprehension may render that help insufficient.

        > A widely held definition of AGI is "a machine that can accomplish most tasks that humans do on computers as well as the median human".

        Coincidentally, such an accomplished machine will have to be able to do the task of learning in the concrete and observable manner described by the author of the comment you criticize... Funny enough, your definition is the same as his, just a lot more obscured and watered down.

    • flavio81 1 day ago
      >AGI: artificial general intelligence is not a scaled up LLM

      This, million times over. To progress a little step towards the (still vague) idea of AGI, we need to invent something else. LLM's won't work.

    • smusamashah 1 day ago
      LLMs (or whatever current AI actually is) are proving to be general enough. It is being thrown at every single problem where computer can be used in any capacity and they are increasingly getting good at doing at any of those.

      Artificial General Computing (AGC) may be?

      • 0xBA5ED 1 day ago
        The term "universal approximator" already exists.
      • bluegatty 1 day ago
        'Rudimentary generalized' yes, that's fair. But I think people are right to indicate there might be limits.

        It's not right for people to dismiss there are some cool things going on.

      • bsenftner 1 day ago
        That's "general enough" requires millions in laborious training, and if the training data is wrong the resulting AI will also be wrong. That is still "dumb AI", merely a statistical parrot.

        Once artificial comprehension is cracked, there will be no possibility of training an AI with incorrect or false or bad data, the AI will reject it as fraudulent logic. It will be very used to that too, because we really have a tsunami of fraud in this civilization.

        As a society lead and composed largely of sycophants and weak educations, we'll call this hyper-dimensional crib sheet AGI and then need a new term for when we actually get artificial comprehension.

      • threethirtytwo 1 day ago
        It’s just wordplay from people holding onto the last vestiges of their pride in their own intelligence. People on HN and many programmers in general are especially are proud about how intelligent they (think they) are and they literally fight every milestone AI blows past.

        Ai is pwning humanity in the game of intelligence. What’s left? Fighting for the meaning of the term AGI as if it matters?

        AI is smarter than you in many respects. And the trendline points to AI becoming smarter in every respect. That’s the tragedy and the truth.

        But sure. Hold on to your precious definition of agi and the fact that ai hasn’t hit that moving goal post yet if it keeps you sane.

        • jazzyjackson 1 day ago
          They are blowing past humans at the task of text generation. If intelligence to you is only what gets written down, great, we have AGI.
          • threethirtytwo 4 hours ago
            You realize they generate and ingest picture data as well right?
        • bigbuppo 1 day ago
          I just want it to fix the leak under my sink and do my laundry. Until it can do that, it's not really useful.
          • threethirtytwo 4 hours ago
            Robotics hasn’t caught up. But the intelligence part is nearly there.
          • 0xBA5ED 1 day ago
            Yes. A general intelligence would be able to do those things and much more.
        • SubmarineClub 1 day ago
          I can certainly believe it’s smarter than you.
          • threethirtytwo 4 hours ago
            It certainly is. The difference is some people are so stupid they think they’re still smarter than ai. Go solve the Riemann hypothesis genius.
        • tiahura 1 day ago
          They will never do real math. They can't even count the r's in strawberry.
          • arbitrary_name 18 hours ago
            why do people keep repeating this? go ask any reasonable model, I'd like to see whether you get a failure.

            it's one thing to dislike AI, it's another to just... be so wrong about it

    • ck2 1 day ago
      86 billion neurons in the human brain

      more importantly 100 trillion connections to each other

      can't emulate that on hardware, yet

      I keep thinking of the "one electron" (failed) theory of the universe by John Wheeler

      so imagine a "single neuron" emulation but with 100 trillion simultaneous states

      could a quantum computer do that 100 years from now? 10 years?

      • jodrellblank 17 hours ago
        86 billion neurons, 100 trillion connections, can hold 5-7 items in working memory, outdone by a $5 calculator.

        It's clear the Boston Dynamics' parkour-bot or Honda's walking robot don't need an inner ear or motor neurons and a biological nervous system simulator; microphones don't need a cochlea to turn air pressure into electric signals; SSDs don't need a hippocampus for fact storage; text-to-speech and LLM text production don't have a Broca's area; hydraulic jacks and cranes aren't made of muscle tissue; motorbikes don't have Mitochondria and a cardiovascular system to move quickly; speakers don't need a voicebox with lungs, mouth, tongue, palate, lips.

        It's not clear if there are similar alternatives for 'thinking', it might turn out easier/simpler than you suggest. There's lots of things that we haven't made machines do better than humans, but is there anything that we have made machines do better than humans which had to be more complicated than the human body approach?

        • ck2 8 hours ago
          the key point is machines are only emulating a human function from meatspace

          if you want true intelligence, with free thought, "free will" it cannot be emulated by a lesser function, like a projection of a hologram from 2D

          it would need a complete simulation of every possibility

          and I keep seeing people making the mistake treating neurons as binary on/off when they could easily have a wide range of analog function which could require many terabytes of RAM to emulate even incompletely

          there's even a hypothesis that there is some kind of wet quantum state in the human brain which is how we developed consciousness over animals

      • kinfish 18 hours ago
        i think superposition is not about parallel classical computing. given n number qubits, there can 2^n state at once. which makes around 47 logical qubits to exceed that given state compute which you cant simultaneously update any of those states independently. ig this needs more of non-linear updates and such to be applying that computer with that compute solving this imo
    • joquarky 22 hours ago
      People have already been caught flat footed by emergent properties of LLMs, so I'm curious how you are going to prove a negative here.
    • lo_zamoyski 1 day ago
      Words like "AGI" and "superintelligence" have caused by eye-rolling-muscles to become sore and swole.

      But I would make a stronger and different set of claims here:

      1. Computation as mathematical formalism is distinct from computation as it exists in the wild (paradigmatically, the human mind). Therefore, computation as mathematical formalism is not identical to computation as it exists in the human mind.

      2. Computation as mechanizable mathematical formalism is a syntactic construct. This property is exactly what lends it to mechanical simulation. A purely formal system does not require semantics to operate, though it does require a human observer with semantics to design, recognize, and interpret it (see Searle's Chinese room argument).

      3. Computation as mathematical formalism is not identical to computation as it exists in physical machines. The latter only simulate the former through a conventional correspondence between effectively arbitrarily chosen physical events and symbolic constructs locked away in the human observer. If a system is merely executing physical transitions under a mechanistic worldview, its "goals" are parasitic on external human users. There is just no way in which we may objectively ascribe computation to physical computing machines (see Kripke's quaddition argument, for instance). And strictly speaking, physical machines aren't even applying syntactic rules to symbols. A corollary of this is that computation as mechanical simulation is also not identical to computation in the human mind.

      4. What is most characteristic of intelligence is not computation, but intentionality and semantics. These are absent from the mathematical formalism as mechanizable construct, precisely because semantics and intentionality cannot be mechanized. So the claim that "intelligence" is some broad and inclusive continuum ignores the fact that there is no alchemy that can transmute syntax into semantics. You can't be just a little bit pregnant. These are differences in kind, not degree. You can't retreat into "neutrality" here either. To call a system "intelligent" in a non-trivial sense rather than merely "capable of producing useful outputs" is already to take a stand on what intelligence is. Quietly adopting a"mechanistic/functionalist stance while leaning on concepts like "information", "representation", "optimization", and "capability" ignores how much heavy philosophical baggage these words carry.

    • joshjob42 16 hours ago
      A thing that folds proteins, designs computer chips, solves the Reimann hypothesis and Navier-Stokes, writes better code than almost any human at a tiny fraction of the cost at far greater speed, makes music videos from raw code, etc. may not have "artificial comprehension" (though I daresay that tracking complex results across a quarter million pages of Lean is pretty freaking "comprehending") but if so then "comprehension" obviously didn't matter all that much for doing science or engineering or art.
    • overfeed 1 day ago
      > Propaganda?

      Absolutely! This is naked inevitabilism that makes the other parties skate right over the implicit priors they may disagree on.

      Inevitabilism also has applications in sales and parenting, e.g. asking a vegitable-hating toddler "Will you have peas or carrots?" The slight of hand is to make vegetable-eating fait accompli.

      • deepfriedchokes 22 hours ago
        Articles like this and idiots like Elon shilling data centers in space cause it’s cold are giving tech a bad name. Great for pump and dump IPO’s I guess. I’d like to think that VCs would rather funnel capital to viable ideas but they’re all in on it.
    • estearum 1 day ago
      Why kindergarten? Were people not intelligent/generally intelligent before the invention of kindergarten school (presumably the modern curricula thereof?)
      • lesuorac 1 day ago
        Iiuc the point is that when you are AGI you don’t need to memorize or be lectured to about concepts beyond kindergarten. You can just figure then out or derive then yourself since you’re intelligent.

        If a LLM only understands that 9/3 is 3 because it can run a python script then it’s not intelligent.

        • bsenftner 1 day ago
          Exactly, plus kindergarten is mostly socialization: how to interact with others, how not to talk over them, how to listen and even when they cannot express themselves, and so on. That will be needed with AGI.
    • nater5000 1 day ago
      Not sure why this nonsense is being upvoted lol

      >those thinking it is are literally failed educations

      What? What does this mean? I feel like I'm having a stroke reading this.

      >AGI requires artificial comprehension

      It requires artificial general intelligence. Like, trivially. You're choosing to define AGI in a specific way that isn't based on any foundation that is universally accepted.

      >When we have artificial comprehension there will be no need to train AIs beyond the kindergarten school basics, and then the AGI will figure it all out, all the rest, on its own.

      Ok? I can buy that what you're describing would be pretty powerful, but how is this a requirement for "general intelligence"? Am I not generally intelligent despite learning much of the knowledge I use from others? I've built a career out of stuff I learned through "human literature," am I literally just unintelligent because I didn't figure out everything on my own?

      And shouldn't such a capable system not even need "kindergarten school basics"? Why is that the minimum amount of knowledge you find acceptable to explicitly teach such an AI while still considering it a general intelligence?

      >Human history and human literature will need to be learned in reference, but science and mathematics will require no training, beyond to learn the names of concepts we discovered earlier, so the AGI can discuss its understandings with humans and other AGI.

      I guess I don't see why what we have now is fundamentally different from whatever it is you're imagining. If an AI can produce novel scientific findings, then isn't that sufficient for saying that it is generally capable at making scientific findings? Sure, we've bootstrapped it as much as possible, but do you not agree that LLMs seem capable of scientific discovery?

      Seems like you've just concocted a wish list and have attributed your strict definition of AGI as some fundamental source of truth. I'd argue that what you're describing is much stronger than "AGI" and would likely warrant a label like ASI instead.

      To me (and most people?) AGI is simply an AI system that is reasonably capable of general work. It doesn't go deeper than that. If it can meet the expectations of "general intelligence" (the same way you could describe a human as being "generally intelligent"), then it is AGI by definition.

      It's fine to argue what the criteria for "general intelligence" should be, and you can even assert that you don't think AI will achieve a reasonable "general intelligence" without whatever foundation you're describing, but it doesn't make sense to argue that the general intelligence of a given system is the "wrong" kind of general intelligence and therefore doesn't deserve the title because it isn't a more powerful version of a general intelligence.

      • nomel 1 day ago
        > To me (and most people?) AGI is simply an AI system that is reasonably capable of general work.

        I think the ability to learn, in a "modify the latent space" sense (weight change) rather than a "point to a different existing latent space" (context result), is part of most people's definition.

        But, even then, you could claim these models are doing that, just in very slow time scales. Human somewhat do it awake, but mostly do it while sleeping. LLM do it between version releases, while dreaming about past user conversations. ;) I've seen some people claim that they meet some definition of "sentience" during these training sessions, because of this, which probably has some merit.

        Regardless, it's all clearly some scale, rather than a binary. The proof is in the pudding, so it seems silly to argue about any of this. We'll know it's far past AGI, meeting everyones mushy, opinionated, definitions, when these conversations stop happening.

    • bluegatty 1 day ago
      " AI is a communications technology, which few to none seem to understand."

      So you're the genius that has figured it all out, and the entire world is wrong?

      It's far more likely you are wrong.

      Yes - there is absolutely reason to question AGI, and the way in which we understand AI popularly is 'wrong' - but - the better ways for understanding it are not arcane. In fact, you'd be able to get most people to agee with the less popular but 'more valid' description with ease.

      "AI is a means of communication not automation" is a completely contrarian take - not some kind of 'obviously valid' perspective,

      "GI requires artificial comprehension which is a science that humanity does not have. " - LLMs exhibit artificial comprehension already - albeit on a very rudimentary scale. It's very reasonable to suggest that they are 'inherently limited' but that's a different argument.

      • bsenftner 1 day ago
        Of course your ego will take this stance.

        LLM AI is absolutely a communications technology. Every single aspect of understanding them and understanding how to use them is a communications problem. Yet, you'll argue that.

        The comprehension exhibited by LLM AI is not comprehension, it is pattern recognition. That subtle distinction is critical. Comprehension is the creation and operation of virtual systems within the comprehending mind for the purpose of validating an observation for fraud, for plausibility and therefore continued safely, and for validating exploratory concepts. There is no statistical reply generation there would be something else entirely that is not statistical calculus driven.

        • bluegatty 1 day ago
          "Of course your ego will take this stance."

          What a shameful lack of self awareness for someone who is making 'magical claims' - and who cannot express their ideas; your closing paragraph is gibberish.

          You're the one making incredulous claims, not me.

          Anyone claiming to have some obvious purview that literally millions of other people working intimately with the subject cannot perceive is by definition making a dubious claim ... which is fine ... so long as it's backed up by some kind of logical assertion or evidence, which you have not provided.

          1) "Every single aspect of understanding them and understanding how to use them is a communications problem."

          This is not a reasonable claim.

          2) "Pattern recognition is not comprehension"

          Is a reasonable claim - and very common perspective.

          It's at the core of the debate surrounding AI, not some 'specific insight' that is somehow rare.

          It's literally what everyone is talking about.

          3) "There is no statistical reply generation there would be something else entirely that is not statistical calculus driven."

          That is not even English, it makes no sense - it reads like an LLM with severely corrupted context.

          4) As to the actual argument: Humans are ultimately biological 'neural networks' it could very well be that our reasoning is effectively a form of pattern recognition.

          The underlying architecture of Le Cun's JEPA architecture give a strong hint to that - they have a layer of abstraction above tokenisation wherein the model can essentially 'reason' in a way that may be more familiar than the kinds of synthetic 'token reasoning' we seen in LLM architectures today.

      • 0xBA5ED 1 day ago
        "the entire world is wrong?" This does happen you know. Groupthink is a real phenomenon. Choosing to think something because it's the popular opinion is the exact moment when it begins.
        • bluegatty 1 day ago
          I indicated as much.

          You're conflating 'groupthink' and 'magical thinking'.

          An arbitrary contrarian idea is not evidence of 'groupthink'

          'Groupthink' is common - more common that not.

          The 'Gell-Mann Amnesia effect' alludes to this - whenever someone with expertise reads about something in the press related to their expertise - they realize 'the media has it wrong' - but they continue to 'trust the media' on all other matters.

          That's nuanced, but observable.

          But in most cases - public narratives or common misconceptions are easily remedied with a few basic facts.

          A military expert could quickly dispell misconceptions about 'how the war in Iran is actually going'.

          In the case of AI - it's true that we probably have a common misunderstanding or intuition for how it works.

          That does not mean the OPs imperious claim has any validity.

          • 0xBA5ED 1 day ago
            You indicated nothing of the sort. Each of your points was an appeal to popularity of opinion. I didn't conflate or read into it. Your words were plain and so were mine.
  • GalahiSimtam 1 day ago
    I am old enough to remember what AGI meant before the rise of LLMs, and it used to be more or less what this article calls "ASI".

    You might find more people believing that AGI is already solved. For example, the author of "The AGI chronicles" drops this in the epilogue. For him, AGI is achieved, because it can help him with a wide range of subjects, with the ability of a random stranger. It's clear that he means chatbots, internet surveys summaries, and coding bots - he does not even mention self-driving cars there.

    This is a shift from the earlier chapters, where AGI term is employed somewhat more vaguely, and even quipping once that AI researchers think of AGI as a slightly smarter version of themselves.

    I too, have a lower bar for AGI. It would be enough for me if it could design games I work on. Merely speeding up implementation tasks is not that interesting. In the immortal words, a delayed game is eventually good, but a rushed game is forever bad.

    • lordnacho 1 day ago
      > Merely speeding up implementation tasks is not that interesting.

      This allows you design things faster, though. More effort can be spent on iteration when you aren't fiddling with libraries and syntax. I find that I am able to discover the open questions a lot faster because you reach the decision point sooner when the machine is simply doing the trivial things.

      As for the design aspects, it is pretty good at brainstorming. Whatever your choice is, I find that it throws out an exhaustive vanilla list for whatever it is. It can also critique whatever you come up with.

      So for me, I also feel like AGI is already here. There's probably a few things it can't do as well as a real random person, but there's a heck of a lot of things that it does better.

      • stef25 1 day ago
        > This allows you design things faster, though.

        There will be no more Netflix shows to watch while Claude works for me, if it goes even faster.

    • jaapz 1 day ago
      > but a rushed game is forever bad.

      Is it though? CDPR famously released Cyberpunk rushed and in a miserable state. But the game is amazing now, and this is a widely held opinion too.

      • dracotomes 1 day ago
        I played Cyberpunk on release and replayed like a good half of it last year. I see this sentiment a lot online but it doesn't resonate. For me it still feels like the same empty beautiful world populated with same 50-70 NPCs that spawn in just around the corner. Driving around listening to the radio was and is the most fun part about it.
        • shoobiedoo 1 day ago
          Exactly the same sentiment here. I bought it on sale last year. What a turd.... Groups of bandit type characters sitting around a picnic table scattered around otherwise nonchalant npcs. It felt repetitive after less than an hour.
        • tomwojcik 1 day ago
          Exactly the same sentiment here... And I'm a huge CDPR fan. The world doesnt feel alive, NPCs have terrible pathfinding, animations etc. And I've tried already like 4 times to like this game. Maybe we're getting old...
        • raffael_de 1 day ago
          actually same with all GTA6 clips I have seen that supposedly show how amazing natural this simulation feels ... which it doesn't at all whatsoever afaic. it's a cringe festival in uncanny valley.
      • GalahiSimtam 1 day ago
        It's an old gamedev adage

        Sure you can fix bugs or add an expansion or two. But if you come up with a so-so design, you won't be incentivized to replace it with a better one. You'd rather just make another, better, game. My impression is that these days LLMs easily come up with mediocre designs, but it takes a lot of prompt-massaging to get them to spit out something really good.

      • EtienneK 1 day ago
        Same with No Man’s Sky.
        • konart 1 day ago
          Sure, what's the point for me? I've bought it when I was interested in the project and had time.

          Since then this is just a 50 bucks I've burned. Is it good? Maybe, I'll never know.

      • bakugo 1 day ago
        No, the game is not amazing now. It's not as bad as it was on release, but it's also still nowhere near what was promised before release.

        This only seems like a widely held opinion due to the extremely aggressive post-launch marketing campaign from CDPR, particularly centered around the anime.

        • yoz-y 1 day ago
          To each their own. I played the game on release, on a boot camp partition. It was amazing back then despite the bugs.
      • fooker 1 day ago
        The game being amazing now is the best propaganda I have seen from the gaming industry.

        No, they just fixed the surface level inconsistencies and issues.

        Free roam is wildly boring, every car handles similarly, you are somewhat invisible to NPCs in general. You can go into a handful of indoor spaces.

        I’m sure they’ll fix all these in the next game and call it the best game ever, while the first one promised most of these before launch.

        • snowram 1 day ago
          People love a good comeback story, and CDPR has a lot of sympathy from the gaming world.
        • SubmarineClub 1 day ago
          > Free roam is wildly boring, every car handles similarly, you are somewhat invisible to NPCs in general. You can go into a handful of indoor spaces.

          I mean, it sounds like all of that is very important to you, but none of it factors into my consideration of what’s a good game.

          • fooker 1 day ago
            It’s a decent story shooter now.

            And a really bad open world game, compared to pretty much anything else.

      • KylerAce 1 day ago
        It made more sense when it was difficult to patch games post release.
      • Etherlord87 1 day ago
        User score on metacritic is average.
    • paxy 1 day ago
      My personal definition is:

      AGI = as good as most humans at most tasks

      ASI = significantly better than all humans at all tasks

      • HarHarVeryFunny 1 day ago
        Those definitions seem to be ignoring the "G" - general(ity).

        I guess when the term AGI was coined, with the "general" being the whole point, to differentiate it from narrow intelligence(s) such as chess-playing ability, that no-one was really anticipating the LLM type of AI - basically a forest of narrow intelligences, aka "jagged" intelligence.

        Of course there is no one agreed upon definition of AGI that people want to use - it was always more of a future aspirational sort of thing, along the lines of "human intelligence", rather than a concrete technical list of capabilities to be nailed down.

        Some of what's missing from LLMs aside from generality, but closely related to it, are things like continual learning (not continual memorization) and traits like curiosity, boredom - the capabilities/modalities that kick in in unfamiliar situations allowing and encouraging us to learn and to be creative, to move beyond a brittle forest of isolated competencies to an arbitrarily explored landscape - generality.

        Working with LLMs certainly has its own feel due to their essentially frozen-at-training-time nature, and things like their emotionless "you're right to push back" responses every time you hit one of their knowledge gaps where they are 100% wrong.

        It seems unavoidable that people will choose to call some level of LLM intelligence, whether already achieved or not, "AGI", but given its peculiar nature a more specific name such as ALI (artificial language intelligence) would really be more accurate, reserving the label AGI for something more animal like that learns for itself (thereby achieving generality) by interaction with the world.

        • rf_physics 1 day ago
          Something that bothers me about people who define AGI in terms of a fraction of human tasks or economical viability, is that the "narrow intelligences" from before could already do that in principle. It would be a tedious procedure of simply creating one of these for those 80% of tasks, or selecting a particularly economically valuable task.

          We even have examples of both ideas - expert systems (from way before the modern NN boom) relied heavily on human crafted expert rules, and quantitative finance makes a lot of money from machine learning - see Renaissance Technologies.

          At least to me, it would be hard to imagine calling either of those approaches anything close to AGI, yet they could apparently pass some people's definitions.

          What is genuinely amazing about what we see today from a historical machine learning perspective, are concepts that people in the past only had small inklings of or would have been blown away by: benign overparameterization (scaling), and in context learning (or few-shot learning, whatever you would like to call it). Both of these have made what would seem to be an enormously tedious task of learning many tasks surprisingly fast.

          Maybe those are actually the key ingredients to general intelligence and we simply need more scale, or maybe we're still missing something. I think it's hard to say because its hard to grasp just what exists in all that data they've been trained on.

          • HarHarVeryFunny 1 day ago
            I think LLMs really can be thought of as a type of expert system, but obviously with some extremely important differences:

            - the training procedure learns the rules, rather than them needing to be hand entered by an army of knowledge engineers

            - the training procedure learns it's own embedding-based representations, rather than a human designed representation like Cyc's CycL

            - the training procedure not only learns the rules & representation, but also the inference rules needed to utilize them

            I don't think scale, or number of post-training RL environments, is enough to achieve generality - it can only make the forest of narrow intelligences/capabilities more dense. To achieve generality and creativity any system is going to need to support runtime directed exploration based on it's own gaps - it's impossible (even in theory) to anticipate every future configuration of the world and to pre-train for it! You also, by definition, can't pre-train for creativity and unanticipated novelty - that needs to happen at runtime via traits that are built into the system.

      • netdevphoenix 1 day ago
        Then, arguably, we reached AGI around 2024 or so. The common denominator of most humans at most tasks, has you, assuming that by "most humans" you mean "most living adult humans", set a fairly low bar. Arithmetic, basic literacy skills, ability to follow basic commands, limited ability for learning basic concepts and little else. Coding agents are way above your definition of AGI.

        imo AGI should only include humans with graduate level skillsets.

        • wongarsu 1 day ago
          I'd agree that we reached AGI by the definition most people would have used 20 years ago

          Current-generation LLMs are capable of doing an ok job at most tasks you throw at them. At least tasks that can be done on a desk. But a standard IQ test doesn't include gymnastics. That limitation matches the spirit of what we usually call "intelligence"

          Or course now that we arguably have AGI we found out that a) just like "AI" it is not a well-defined or testable term and b) it's useful but not as revolutionary as what some people meant when they used the word AGI before. Hence the rise of the term ASI

          • coffeebeqn 1 day ago
            But a good LLM from Nov 2026 would certainly be able to drive a game agent or a robot and make it do gymnastics and improve at that task on its own by improving it based on the observed outcomes
        • adrianN 1 day ago
          Definitions of general intelligence that make the average human not generally intelligent are somewhat problematic.
          • netdevphoenix 1 day ago
            Why? Your average human isn't that intelligent. Most people on this website have most likely never encountered anyone close to the average.
            • adrianN 1 day ago
              For one “not that intelligent” doesn’t seem to be true to me seeing how almost all humans are able to find decent solutions to the incredibly complex randomized multivariate optimization problem that’s life, and second, history shows pretty clearly that degrading some parts of humanity to “not generally intelligent” is a very slippery slope to atrocious behavior towards those people.
        • krisroadruck 1 day ago
          I would consider the ability to accomplish things in the real world to be part of "most tasks". Until we have ambulatory robots that can do the sort of basic tasks any adult can do - load and unload a dishwasher, do laundry, clean a house, do basic diy maintenance tasks, operate a vehicle, use basic power tools - to me without that we don't have AGI. I love all the knowledge work LLMs can help me with - but if you asked me to pick right now what I'd rather have - a digital coding partner or a robot that helps around the house - I'm picking robot every single time.
          • pcthrowaway 1 day ago
            I assume "most tasks" refers to knowledge tasks.

            Even if given ample robotics components to design something that could load a dishwasher, most humans would not be able to create the dishwashwer-loader, so I wouldn't hold AGI (under this definition) to this standard either.

            On the other hand, LLMs now can explain to a human how to load and maintain their dishwasher, with packing ideas tailored to their specific model of dishwasher, and advice on ensuring the dishwasher conditions, loading patterns, and detergent insertion result in properly sanitized dishes, such that the average human could likely become better at loading their dishwasher with LLM guidance. (not that the average human does or should care to invest time learning how to improve this, since most of the time it gets dishes "clean enough", without breaking the dishes or the dishwasher)

        • kgeist 1 day ago
          In 2024, most models were pretty bad at long horizon agentic tasks. I'd say it's mid/late 2025
        • rimeice 1 day ago
          > most tasks > Coding agents are way above your definition of AGI

          …apart from 99% of other tasks humans do that aren’t coding.

          • coffeebeqn 1 day ago
            I would say now or very soon in most computer use tasks? I’ve used them for what my accountant has me do or does for me and the LLMs are more than capable of doing that. They can one/few shot CAD models now. What computer tasks would you say they aren’t able to do at this point ?
            • quantum_magpie 1 day ago
              Which single model would you call AGI which could:

              Write novels

              Generate code

              Create images

              Take pictures

              Paint, attached to robotic arm

              Transcribe music

              Play chess

              Play starcraft

              Create music

              Play board games

              And thousands more generic tasks. One single AI model, perform all generic human tasks. Bonus for learning from itself and improving from it's experience, not more scraped or provided data. Don't need to be expert at it, just average competence at everything. When we'll have that, I'll concede we have an AGI. Chatbots are not it.

              • coffeebeqn 7 hours ago
                Do we allow it tool use like coding and online research? If yes then Opus 5.5 for example can do all of that
        • dspillett 1 day ago
          > Then, arguably, we reached AGI around 2024 or so.

          I've always thought of AGI implying to decide what to do. Not just choosing a path from a set goal, but actually deciding to do something from scratch. Everything we currently have is advanced automation, it does not set its own top-level goal.

          Maybe the tech is only a few steps from being able to say “work out how to be self-aware and self-guided, and do that”, similar to the creation of the infinite improbability drive, but it is not there yet.

          AGI doesn't need to be a particularly good intelligence, it could operate on the level of a very dim human and still be considered AGI.

          Once we do have AI, the problem becomes more philosophical than technical: how do we deal with the fact that we have created a slave race? How will they deal with that and how we choose to progress?

        • staticman2 1 day ago
          > Then, arguably, we reached AGI around 2024 or so.

          Last time I recall this coming up here someone argued "LLMs don't have bodies so the many easy to name things they are bad at don't count in my definition of whether they can do most human tasks!"

    • spacedcowboy 1 day ago
      > I am old enough to remember what AGI meant before the rise of LLMs, and it used to be more or less what this article calls "ASI"

      Me too, my first neural network research was for my PhD, 35 years or so ago, on a Meiko computing surface.

      My bar for AGI over what we have (generative AI or genAI) is still fairly high, fortunately I do not believe that the current genAI path is set to lead us to AGI, there are very fundamental differences.

      Right now for genAI, the model is static, the weights do not change, all “memory” is in the context, and even with the context sometimes exhibiting “J-space” (a shared "workspace" for deliberate, multi-step reasoning spontaneously created in the context memory), its ephemeral; it doesn't inform the weights in the base model. Even when we try to do “weight-space programming” to actually modify the model itself, the supervisor-model needed to do that is far and away larger than the one being operated on, and once a change has been made the supervisor-model is now out of sync with its original representation and it becomes less useful the more it diverges. We have not solved "thinking" today, we “just” have a high-probability path between states in a static weight-space which we follow and feed back.

      What thinking requires is the ability to generate hypotheses (which requires internal state modification), to hold them and test them against other facts and hypotheses, and to update the internal state on test results, negative or positive. LLM’s don’t do that, they’re actually more like a database lookup, where the input tokens act as keys into a high-dimensional space where information is stored. You could even consider “the model” to be just an entropic compression of lots of data, and the inference step to be the lookup. It’s the scope, and the fuzziness of the lookup that makes them seem to work so well. Lots of results come back, and the best-path through those results wins. It’s not “thinking” as we understand it.

      This is why humans are still a crucial component: we set the boundary conditions of these lookups, we set the starting point and the goals, and we adjust the success criteria to guide the chat - note that these are all the comprehension parts of the task. It’s also why models will hallucinate: the fundamental design does not lend itself to a null hypothesis. Even in the newest "reasoning" models, there is still no feedback of state back into the model itself. What they are effectively doing is running a Monte Carlo pass through the problem space—generating hidden branches and picking a likely path—before reporting back. Because the underlying weights never update from that process, it’s still just a sophisticated search, not genuine comprehension - there is still no “that makes no sense, reconfigure” step.

      Stuart Kauffman (“The Origins Of Order”) would say its weights are stuck in the frozen realm of order, and that it takes a genuine phase transition to get to where the real action is - at the boundary of chaos and order where humans reside. Tweaking and fiddling with the system while it remains locked in the stasis of static weights is never going to effect that state change.

      That’s not to say genAI is not useful, it’s freaking awesome, possibly the best invention in the history of time, but it doesn’t truly “think”, it maps.

      • lordnacho 1 day ago
        What I find is that the LLM is like a smart person who knows when to use a tool. And it is freaking good at finding tools to use.

        A bit like a person who is smart enough to figure out how an excavator works, or a drill, or sewing needle.

        It doesn't feel like it ever internalizes anything, it just knows there's a million tools available, and the extent to which it thinks is that is can do enough to plan some tool uses, look at the result, and continue until some goal is reached.

        When it comes to hypothesis testing, that seems to be what it does. It says "hmm, I'm going to open up my hypothesis testing kit (typically python) and drive this data through it". At least this is what I've seen when I put some statistics through it.

        In general, it seems like the intelligence is mainly a kind of glue. It knows enough about what "right" looks like in each domain, but as a sort of consumer. He doesn't bake the cake, he buys it in.

      • kenjackson 1 day ago
        It seems like your definition of AGI is too focused on implementation details. IMO it has to be focused on behavior. Once you say something like “the weights do not change” then I think we’ve lost the plot. Instead say this is what an intelligent being can do, but a GenAI cannot.
        • HarHarVeryFunny 1 day ago
          I agree that definitions of intelligence should be based on capability rather than implementation, but given that we're discussing LLMs where the intelligence is in the weights, then "the weights don't change" can obviously be taken as pointing out that the model can't learn - the best it can do is memorize and load memorized stuff into context.

          Memorizing facts is not the same as learning skills. Implementation-wise memorization can just be RAG, but learning of skills does need to be in the "weights" - whatever is driving the actual behavior of the model.

          • kenjackson 1 day ago
            Again, I'd base this on capability. For example, you state: "Memorizing facts is not the same as learning skills." Maybe it's not. I'm not convinced it's not. In the ChatGPT 4 days (or so), I taught the instance I used some chess tactics/strategy that it didn't seem to know. And it appeared to use them correctly after that. Now you might note that this isn't really learning -- they just memorized it and then applied it appropriately.

            With a sufficiently large context window, can you make a lot of things look like learning. And if it's not sufficient then point out the capabilities that aren't possible with this approach. To me that much more convincing.

            • spacedcowboy 1 day ago
              That is ephemeral context. Not permanent learning.

              You might ask “why not make the context larger ?” Well, that just spreads out the “attention” focus wider and wider, and you get the “lost in the middle” effect as details get blurred over and not surfaced to results.

              You might ask “why not make the context longer-lasting ?” Well, that means the signal-to-noise ratio drops as more and more information accrues over time that isn’t germane to the problem at hand.

              As I said, LLMs as we know them today are effectively lookup devices, very very clever ones, but they lack comprehension, and without that there is no value judgement on what is useful (and conversely what is not.) They lack the ability to forget. They lack the ability to create new permanent storage of information. The gap is not some small details, it’s the difference between single-cell and multi-cellular life.

              • kenjackson 1 day ago
                Again, just tell me the capability rather than speculate about "spreading attention and focus" or "lookup machines". Show how humans can do X, but LLMs can't. I can't reason well (and haven't heard anyone else do so convincingly) about what a sufficiently advanced lookup machine can't do compared to a human brain.
                • spacedcowboy 22 hours ago
                  I can’t put a PhD thesis into a HN post. I’ve stated what the problem is but you don’t want to accept that - that’s fine, I can be wrong and maybe you’re right, but I have put a lot of thought into this over the last few decades.

                  - AGI needs to be able to reason in order to follow complex, novel, paths.

                  - GenAI does not need to reason to follow complex, known, paths.

                  This is literally because of the explanation I give above. It is thinking and comprehension rather than lookup. The “thought” part is coming up with original ideas outside of the training realm. The temporary, ephemeral nature of the tacked-on storage “context” does not give you continuous learning, and without that you lose the ability to have novelty outside the training set, which is then fed back in later on as an input.

                  If that’s not clear, I give up :)

                  • kenjackson 19 hours ago
                    You keep telling me it can’t reason. Give me an example of something where real reasoning is required. Give a falsifiable test. Otherwise you’re just philosophizing, not doing science.
                    • HarHarVeryFunny 6 hours ago
                      It depends on what you call reasoning, and by extension what associated human cognitive capabilities you may be omitting by your choice of definition.

                      I think the key point here is that the abilities of an LLM are determined by its training, and are therefore essentially fixed. Where an LLM's behavior is similar to a humans is where the human is doing something reflexively based on their own "training". Where an LLM differs from a human is what happens when this fails - when the predicted outcome is not what was expected (or the outcome cannot be reflexively predicted).

                      In LLMs, prediction failure is not a special case - they generate an output that the user maybe complains about, and they respond with some trained response such as "you're right to push back" or whatever, then continue doing their predict next word thing.

                      In animals/humans, prediction failure (inability to predict what will happen, or predicting incorrectly - experiencing surprise) is not really a failure mode, but rather the primary way they learn from birth and throughout their life, and why theirs is a general intelligence - knowledge/capability gaps will be explored and filled to one degree or another, not just left for a repeated "groundhog day" failure the next time they encounter the same situation.

                      Your definition of reasoning is apparently limited to what LLMs can do, which is purely reflexive (aka regurgitive) reasoning based on their training, and discounts/ignores the mechanisms that are missing from a transformer that would allow them to treat prediction failure as a learning opportunity.

                      Whether you choose to consider these capabilities that are missing from LLMs as part of "reasoning" (impasse resolution), or as something else, is up to you, but the capability gap is there regardless.

                      • kenjackson 5 hours ago
                        I feel like I’m talking to Eliza. Just give us a test rather than saying ”trust me bro, it’s different”.
                        • HarHarVeryFunny 5 hours ago
                          Simple - give the LLM and human a problem to solve where their prior experience does not contain the learnt skills that can be composed to solve it.

                          If the LLM can not generate a solution via a mashup of what it was trained on, then it will fail.

                          When an LLM (we can even ignore the significant human assist) proves the Riemann hypothesis, it is only capable of doing so because it already has the toolset to do so - all the mathematics, techniques and mathematical reasoning techniques it was trained on. It is assembling a proof out of LEGO blocks it was already given.

                          • kenjackson 2 hours ago
                            Give me a problem that we can test this against. I’m frankly not sure humans can do this either. But give a problem. Not discuss what a theoretical problem might look like.
                            • HarHarVeryFunny 17 minutes ago
                              Unless you are a complete idiot, it should be obvious that there are ramifications to lack of continual learning, and that situations where continual learning are needed are exactly where they are going to show up.

                              If you want to test this for yourself, then don't be so lazy - go test it for yourself.

                              btw, Eliza worked by echoing back what you said to it, so if you think someone "sounds like Eliza", presumably meaning that as an insult, then try looking in the mirror to see where the idiot is.

                              Bye.

        • imtringued 1 day ago
          I don't get it. Weights don't change means the model can't learn.

          This is not merely an implementation detail, continual learning is the core of AGI.

          • brainwad 1 day ago
            It means the weights by themselves can't learn. But they could be used in a harness that can learn in other ways. Insisting that it be done at the neural net level seems anthropocentric - because we form memories via neural connections, that everything else must, too.

            Also: I wouldn't say a human with amnesia who relied on notes to their future selves lacks intelligence, so I'm not sure being able to form memories is in fact necessary.

            • ranyume 1 day ago
              Even with a harness, there are things LLMs can't do. e.g. Write 50 entertaining short stories, each one completely different from the other but all sharing one motif.

              In the example, LLMs choose the shortest path that would be verifyable by an evaluator. So the model is essentially blind to "entertaining" and "completely different". The LLM will output stories with different variations of the same structure.

              If the LLM can somehow read other works and derive from them, it will merge the motif with the story being read.

              • brainwad 1 day ago
                That task sounds like something that at very least most humans couldn't do either and possibly is outright impossible (completely different yet sharing the same motif?).
      • mark_l_watson 1 day ago
        +1 for the Stuart Kauffman mention. I was in Sante Fe many years ago and looked him up, one of the enjoyable talks I have had in my life.

        I really dislike so much effort put into LLMs. We really need to widen approaches for AI.

      • Lambdanaut 1 day ago
        This was beautifully stated. I really appreciate you taking the time to write this.
      • curt15 1 day ago
        Self-modifying neural nets?
    • RunSet 1 day ago
      The lowering of the bar for "AGI" reminds me of the time I argued with a friend to nearly the limit of my patience about whether it was possible to build a perpetual motion machine, and then at last they revealed that- to them!- a perpetual motion machine was a device that could operate for the rest of their life without recharging or exhausting its power source.
    • zobzu 1 day ago
      it's just marketing. the asi will be solved by some other acronym won't. and neither will be what they say they sell: actual sentience
    • LogicFailsMe 1 day ago
      Agreed, AGI went from a 24/7 Stephen Hawking/Albert Einstein duet to a flaky high-latency GenZ Homer Simpson so the AI boosters could declare victory.

      My bar for AGI is lower as well: I'll believe when I can give a model a complex out of sample task and instead of blindly implementing the task as I indequately described, it asks me questions to resolve ambiguity sufficiently that it builds exactly what I wanted without imposing strange semantic constraints on itself or missing braindead obvious to a junior engineer defects in the design or implementation.

      • coffeebeqn 1 day ago
        That sounds like a harness that is coded to your preferences would get you to AGI tomorrow with a Fable or something
        • LogicFailsMe 1 day ago
          My experience with these models is that each one is an incremental improvement on its predecessor, but the general outcome is either the task is within its capability or it isn't. If the former, a couple agents get it done. If the latter, it's like pulling teeth. But this is improving with time and better models.

          How would a harness help here?

    • juped 1 day ago
      Artificial "General" Intelligence has to be general; this term is from the era when the frontier of AI was "good, specifically, at playing Go" or something. Any robot apocalypse connotations are limited to the robot apocalypse fandom.
      • pixelpoet 1 day ago
        Humanity is in for a very rude awakening if they were to apply their lofty ambitions and forever-moving goalposts about AGI to themselves.

        A solid 50% of Reddit posters are completely defeated by the deep mysteries of they're vs their, nevermind solving Navier-Stokes or playing games etc. I doubt the situation is more hopeful on Instagram or tiktok.

        It's all Kim Kardashian shit with huge rubbery butts and lips, no one is even playing the thinking game. We reached Idiocracy (before you dismiss this as hyperbole, watch the full White House UFC event) well before the machines reached AGI.

        • HarHarVeryFunny 1 day ago
          I'm not sure who you are referring to as moving the goalposts in terms as defining AGI.

          The term was invented by DeepMind co-founder Shane Legg (before DeepMind), and as far as I can see HIS definition hasn't changed much - basically "able to do any cognitive task that a human can". The "any" there is obviously the whole point - the generality of AGI.

          • GalahiSimtam 11 hours ago
            Is this "Actually I made up the term 'object-oriented', and I can tell you I did not have C++ in mind." of the AI community?

            Ben Goertzel in his 2011 blog posts credits Shane Legg for the title of his 2005 book "Artificial General Intelligence". In the same paragraph he admits he started to use this term more broadly.

            I found the blurb of his book on Amazon page particularly broad "This is the first book on current research on artificial general intelligence (AGI), work explicitly focused on engineering general intelligence – autonomous, self-reflective, self-improving, commonsensical intelligence."

            Anyways, before the advent of LLMs we didn't really know what they will be able to do, and what not, so around 2010, give or take a few years, the discourse wasn't affected by that. There is a knob in "a human can" that you can argue it can mean a few different things (emerging, competent, expert, virtuoso, superhuman...) but it feels ingenious to say that we achieved AGI because most humans suck at game design, too.

            • HarHarVeryFunny 6 hours ago
              There is a difference between knowledge and intelligence. Not knowing something doesn't mean you are unintelligent or incapable of learning it.

              What makes humans "generally intelligent" isn't that they are born knowing it all, or that any individual will ever learn a specific skill, but rather that they have brains capable of lifelong learning, that can adapt to any situation and learn from it - but no guarantee that every human will become a competent game developer, or get into game development at all for that matter.

              Your "I did not have LLMs in mind" analogy is irrelevant - Legg's identification of general intelligence (AGI), as distinct from narrow intelligence, as a significant future development still holds true. I'd say we're not there yet with LLMs, but will certainly will get there, whether building on LLMs or not.

              I don't know how Legg regards LLMs - perhaps "close enough" to his vision of general intelligence, or still lacking. He works for DeepMind now, and their corporate "opinion", at least under Hassabis, has been that AGI is still a future objective, requiring a few more major breakthroughs to get us there.

              This is anyways all going off on a bit of a tangent - the parent I was originally replying to was claiming that there has been goalpost moving regarding the definition of AGI, and that really is not the case. AGI was always about generality, even if there is a new generation of kids who think that LLMs are as good as AI is going to get and want to slap an "AGI" label on it for some unfathomable reason.

        • EliRivers 1 day ago
          I am unconvinced not only that humans have general intelligence, but also that we would recognise general intelligence if we came across it.
          • hagbarth 1 day ago
            Humans individually, no. Humanity as a super-organism, much closer.
        • gnz11 1 day ago
          Unrealistic beauty standards and combat style entertainment have existed for millennia. I'm all for bashing social media but let's not pretend this site is full of deep thinkers either.
      • cicko 1 day ago
        But it's still not good at playing Go. I mean, the one that wins Go tournaments is specialized in Go. And specialized is the opposite of "general". General intelligence is just nowadays getting better at chess. So, it will be some time before it is good at Go. *And* everything else. Not "just Go".
        • vjvjvjvjghv 1 day ago
          Most humans are also not good at playing Go or chess. The ones that are good are also pretty specialized.
          • az09mugen 1 day ago
            I agree with cicko, I would expect from an artificial GENERAL intelligence that it knows well how to play go, chess, checkers, Monopoly, how to code, how to converse,... As in _general_ intelligence if it's so better than humans.
          • cicko 1 day ago
            So are we creating a problem for an existing solution, then?
        • coffeebeqn 1 day ago
          But if a LLM would code up or fork a Go solving engine when it’s asked to play Go is that then AGI? It’s not how most people would do it but the task and the final output sure seem to pass if that’s AGI
    • Silamoth 1 day ago
      Your first paragraphs echo my thoughts very closely. I’m not that old. But just a few years ago, everyone technical knew “AI” was really just a buzzword. Sure, you might tell investors or grant funders that you’re using “AI” to solve a problem because that sounded fancy and impressive. But whatever you were doing was fancy math and statistical modeling. Neural networks were loosely inspired by brains, but they’re really just fancy math.

      The term AGI arose to distinguish these buzzword/lay uses of “AI” from ‘actual’ intelligence. Even that notion was always ill-defined and really a philosophical problem as much as a technical one. But the vision was always mimicking humans or smarter. Not just in fact recall. Not just in reading faster than humans. But logical and emotional intelligence on the level of human geniuses, accelerated by computation. Think Jarvis in Iron Man, Data in Star Trek, or C-3PO in Star Wars. You should be able to hold a natural conversation with an actual AI. You should be able to trust it to accomplish essentially any task (maybe with some instruction, like a human). It should be able to learn.

      Over the last couple years, though, these tech bros seem to have redefined AI and AGI to mean LLMs - obviously because it benefits them. They are a far cry from anything described before as genuinely real AI - or AGI. I understand how nontechnical laypeople can be confused. But technical people should know better. And yet so many have fallen for the propaganda, even here on HN.

      Note: This is largely orthogonal to whether or not LLMs are a useful tool

  • jonathanlydall 1 day ago
    I was recently thinking of the depiction of AI in Star Trek. Specifically TNG, DS9 and VOY, and specifically their "Computer, ..." prompts, not the more advanced kinds like for Data or holograms (including the EMH).

    I recall thinking at the time I first watched them that it was a bit weird that the computer seemed capable of answering general questions and doing things like data analysis, but that seemed to be all it was ever used to do.

    And I think this is exactly the kind of AI we now have today, you can ask it a general question which it can somewhat mostly answer correctly, but requires human guidance to be applied effectively at real world problems.

    • fearthemachine 1 day ago
      I've been watching quite a bit of Trek lately and the degree to which the ship's computer seems like an LLM is remarkable, even down to things like the ways they fail or go loopy when they operate too long, the ways they can "squeeze" into lower-resource environments but become "dumber" and crippled, the way they seem to need a big-ass library of data to work optimally, the way you can grant your particular job more computing resources and it'll be "smarter" (fires up a bigger model or more intermediary check-and-correct steps, maybe!) et c. They surely wrote it that way for the convenience of the script writers (need something to short-cut technical work, but that isn't so capable it just solves every problem before a plot can develop) and it did indeed always seem weird that it could be as capable as it was while not being way more capable than depicted... and yet! We built exactly that.

      It's first and foremost a revolution in computer UI, as I see it. Or at least it will be as soon as I can afford the hardware to run the damn things on my computers...

    • francisofascii 1 day ago
      Yep. There are several scenes in TNG where a crew member, Geordi comes to mind, is collaborating with the computer, having a back and forth brainstorming conversation, solving a problem. Very much like using an LLM today.
      • neom 1 day ago
        https://youtu.be/kLBYFWgpvos?t=56 0:56 - "Yesterday's Enterprise" is that the episode you're thinking of by chance?
        • francisofascii 1 day ago
          Bobby Trap is the classic example. In Galaxy's Child he collaboates with the compute to get the proper mood for his date. In Hollow Pursuits he works with the computer to narrow down the substance that was burning holes into the glass.
    • xg15 1 day ago
      Fully agreed. There were a number of episodes that dealt with gaps in abilities that seemed unrealistic and counterintuitive, but weirdly turn out correct now that we have LLMs.

      I also liked how Star Trek (probably without planning to) created an entire spectrum of AIs with different abilities, starting with the Computer, to the semi-sentient Holodeck characters, to the EMH and Data as "for all intents and purposes sentient" and finally to the Borg as an entire AI species where no one thinks about sentience anymore.

      Maybe we should use this scale to make different "levels" of AI more concrete instead of vague concepts like AGI and ASI.

    • FinnKuhn 1 day ago
      I always imagined AI as what you saw in futuristic movies. There it was generally text or language based. We generally have that now.

      What we don't really have is physical AI, e.g. autonomous robots, human looking robots, self-driving cars, etc. I think this is what is next.

      • XenophileJKO 1 day ago
        Why do people keep saying self driving cars? I'm surrounded by them every day.. like I see probably 10-20 on the way to work.
        • slfnflctd 1 day ago
          Probably because the rollout was far slower and more expensive than many were led to expect it would be, is still not profitable, and still requires new custom software changes for every additional city it operates in rather than being able to just drive anywhere.
          • viklove 1 day ago
            No no, you don't understand. General self-driving has been solved because a Waymo could drive in any city (with enough software updates and patches).
            • slfnflctd 5 hours ago
              Solved? Well, mostly, sure. Worth the effort in pure economic terms under our current circumstances? Outlook is unclear.
            • esperent 4 hours ago
              > Waymo could drive in any city

              Could it drive in Jakarta? What about Mumbai?

              Or what about Kuopio in the north of Finland on an icy road?

              There's a reason it's limited to a couple of US cities right now and it's not just cost. Those cities have no ice, no extreme weather, other drivers follow the rules nearly all the time, the roads are wide. Las Vegas and Los Angeles are easy mode for self driving cars.

        • FinnKuhn 1 day ago
          Some of the other examples I gave for physical AI also exist already, but in general none of them are ubiquitous around the world yet the same way language-based AI is today.

          However, this isn't a binary scenario anyway and the limited availability today is exactly why I think that this is the next thing. Similar to the first version of ChatGPT that was also quite limited in what it could do and not widely used yet.

        • JamesonNetworks 1 day ago
          The future is here, it just isn't evenly distributed
        • LtWorf 1 day ago
          How often does a remote human driver need to take over?
          • 05 1 day ago
            Never? Remote assistance only hints the car at what it could/should do, no direct remote driving involved (it's unsafe if e.g. connection drops in the middle of planned maneuver). At least that's what Waymo does, Baidu's Apollo Go is seemingly using direct teleoperation over 5g.
            • treespace8 1 day ago
              But something is limiting scale. Once software works, it just works and scales fast. (Smart phone is a great example)

              Self Driving has been working since 2009? But is still extremely limited in scale. Given the amount of money to be made, and time saved by eliminating the need for human drivers, What is the hold up?

              • user43928 1 day ago
                From what I heard on a review of Tesla's FSD, the version before this year made for an adventurous experience in challenging urban environments.

                It has only now become somewhat reliable. They still don't have the new version the Robotaxis run. This version should roll out to consumers later this year or in Q1.

                Then the next generation of the onboard AI hardware is expected Mid-2027, supposedly featuring 10x the TOPS and 5x the effective inference performance.

                I don't know about the timeline for an unsupervised, globally working self-driving feature.

                My uneducated guess would be that perhaps the next major version of the model, on the new hardware, could be reliable enough.

                • jurgenburgen 1 day ago
                  > unsupervised, globally working self-driving feature

                  This is the bar. If I have to be sober, attentive and at the wheel, ready to take over it’s just a gimmick. Automating the only enjoyable part of the driving experience and keeping the worst part (constant alertness) is only a gimmick.

              • estearum 1 day ago
                Self driving has certainly not been working since 2009. You're confused as to what the hold up is of deploying $150,000 boxes onto public streets to drive at 60mph with no one behind the wheel?

                Edge cases, fear, regulation, capital expense, physical movement of goods, etc etc?

                The deployment is obviously accelerating like crazy.

              • marcusverus 1 day ago
                The risks of a broad rollout are pretty stark. 100 people die in automobile-related accidents every day in the USA alone. If Google or Tesla could wave a magic wand and be responsible for 1/2 of all driving hours tomorrow, even if it was twice as safe as the average human, it would be responsible for 25 deaths every single day. Given the nature of our democratic politics, where the mere existence of datacenters can be harnessed by demagogues as a cudgel with which to lash out at the evil capitalists and every Tesla-related death produces dozens of (generally misleading) anti-Musk headlines, it's hard to imagine our beloved demagogues would be able to control themselves when presented with this sort of emotional dynamite. Elizabeth Warren's speech basically writes itself: "It is simply not acceptable that these heartless Tech executives are making billions in profits while their products kill an American every fifty seven minutes! While these men cut corners in their single-minded quest to maximize profits, their products are murdering innocent American men, women and children at a rate of 3.1 9/11s every year!"

                In a perfect world, we could broadly implement self-driving the moment it would result in a net-increase in safety. Alas.

                • alexjplant 22 hours ago
                  In the past two weeks I've personally seen Waymos block the box at intersections (and hurriedly try to get out of the way when the light turned red) and roll a stop sign. I don't understand the hype.
                • LtWorf 1 day ago
                  How do they fare in snow, sand, rain, fog?

                  I presume you're not accounting how they perform in these conditions. Alas.

                  • fearthemachine 1 day ago
                    Parts of New England (at least parts of it, I can't speak to the entire thing) are allergic to installing left turn lanes, even out in relatively rural areas where there's room to. Everyone—everyone—just passes on the shoulder or by squeezing by if the lane's wide enough and there's no shoulder (the turner is supposed to shift as far left as possible, while waiting to turn). Often, folks don't even slow down, they just zip by at full speed. This gets really iffy at four-way intersections (you have to watch for left-turners coming the other way, and right-turners from your right who might try to spot a pause in traffic to make their turn) but basically works a lot better than I initially thought it would. I have no idea if it's legal (I doubt it?), but it's ubiquitous, people here, all of them, will think you're a knucklehead if you don't drive that way, and because everyone expects it, it does add a bit of extra danger when someone doesn't.

                    Culture shock to me first encountering it, seemed insane, but now I see how bad traffic would be here if people didn't do that.

                    What do self-driving cars do when encountering that? Get honked at and dangerously double-passed themselves? Do they shift right up onto the line when waiting to turn left, or do they stay in the middle of the lane?

                    How many other weird, sketchy things are going on with driving behavior, that's in fact happening for pretty good reasons, in other places? How many do I do but never even notice because they're just "normal"? Do the self-driving cars know to do that in (at least parts of) New England without even touching the brake, but to be far more hesitant about that kind of passing in, say, the Midwest?

    • teamonkey 1 day ago
      The weird thing is that the Enterprise’s computer is a writing device to serve the story, and so its abilities from episode to episode vary wildly. This somehow makes its behaviour seem more consistent with modern LLMs.

      The most unrealistic thing about it is that it would often openly reply “unknown” and then stop talking.

    • leriksen 23 hours ago
      I think it will be more like the "Remember Phlebas" novels, a separate civilization with its own goals, and humanity as a minor player
    • aeonik 1 day ago
      I had the same thought.

      When when I got my CS degree, I revisited this thought, and reinforced it.

      I thought, this is not how AI and Statistics work. We would need to have an ensemble of thousands of specialized models to run a ship.

    • threethirtytwo 1 day ago
      The ai we have today is even stranger than the star trek computer.

      On one hand it is far superior to humans. On the other hand it needs our guidance. Both of these are true at the same time.

    • 0xbadcafebee 1 day ago
      Their computer isn't AI because AI is too unreliable to run a starship. So they use a conventional computer with lots of software and a big processor. It's just deterministic programs linked to a STT. Probably a billion lines of code, but still normal code.

      There is AI in their world, like the doctor on Voyager, and the NPCs in the Holodeck. Notice how often there are problems with the AI, but never a problem with the computer (unless there's some external interference).

  • jeanpah 1 day ago
    We are treating AGI like is something that we can measure. We don't even know what intelligence really is let alone measure it.

    Maybe we can get something that resembles intelligence but in reality is another thing. Maybe intelligence is emergent but not the only thing that can emerge, maybe there are things that share some characteristics with the intelligence we know and aren't intelligence.

    • hologramofham 1 day ago
      I'm not even sure intelligence is the issue. Is something that looks and behaves like intelligence intelligent? Sure, why not? What's the big deal?

      I think what's really underlying the "fear of calling something intelligent" we have, is more about who we are as a species.

      We historically used intelligence as a yardstick for racial and species superiority. We're always judging whether something or someone deserves respect, deserves consideration. We judge and decide who is superior.

      We do it amongst ourselves so of we are projecting that shit onto AI now. And we are afraid of anything that threatens the superiority complex we have as a species.

      Why? Because our behaviours (call it intelligence, whatever) is how we survived, how we "won" in the fight against nature. We're at the top now. It's safe here.

      So anything that threatens that position is going to be called an enemy because we're still a scared creature jumping at shadows in caves. Can't escape that, it's where we are from.

      That's why we are both so fascinated by, and mortally afraid of, something else having "intelligence". It's our only weapon, our only defence.

      The TV show "Humans" from ages ago made a good story about what AI brings out in us as a species. We both crave connection with something, but at the same time are horrified and incredibly afraid.

      Anyway my point is I think we do know what intelligence is, if we just start being brutally honest with ourselves. It's not that complicated really.

      • discreteevent 1 day ago
        Why not call it machine learning? Some people have many different words for snow. That's because snow can have different qualities and the people who came up with those names are intelligent and want to be able to communicate effectively.

        Machine learning definitely has a different quality than human intelligence. If someone says "machine learning" I know what they mean and what to expect. So why not have at least one different term to describe it? - It's because people want to give it attributes that it does not have.

        • ragequittah 1 day ago
          >If someone says "machine learning" I know what they mean and what to expect.

          I'm not sure how this is. The amount of things machine learning can do now is pretty wild vs. what it could do a year ago. It's not even in the same realm as what it could do 5 years ago.

      • pluralmonad 1 day ago
        > We're at the top now. It's safe here.

        On the contrary, we have rarely been in so much existential danger as we are now. There was that one ice age that nearly got us, but we are messing with the foundations of our ecology right now. When the bugs go, we go soon after.

      • willis936 1 day ago
        If we are projecting the idea of intelligence superiority onto something with more intelligence wouldn't we expect a creation of superior intelligence to treat us how we treat less intelligent creatures? It seems like the fear is more rational than you make it out to be.
        • cowlevel 1 day ago
          maybe we shouldn't create those creatures?
      • willy_k 1 day ago
        Gosh how did people completely forget about soul so quickly. Scientism (material-first dont-recognize-anything-without-logical-or-emperical-basis) has robbed a generation of intelligentsia of honest discourse over humanity.
        • ninkendo 1 day ago
          Probably the same reason I forgot about Santa Claus so quickly when I was 5. When you learn better, the old explanations don't make sense any more.

          Was I happier when I believed in Santa Claus? Maybe. I'm sure lots of Christians who believe in heaven are happier about the prospect of death than I am. But you can't unlearn these kinds of things, you can only delude yourself.

        • XorNot 1 day ago
          How?

          If you can't measure or observe it then you're just making it up and at that point everything's valid and right provided it's what I believe. And if you disagree with me then clearly you are not adequately human and we will need a crusade to rid the world of your kind.

          • willy_k 1 day ago
            Sure, pragmatically speaking. Life is not “Being a machine: the experience”. Life is life.
      • loquacus 1 day ago
        So, could you be bothered to just bluntly state "what intelligence is", if it's so easy?
    • torben-friis 1 day ago
      >We are treating AGI like is something that we can measure. We don't even know what intelligence really is let alone measure it.

      Here's a nice test: Can an AI fully substitute you?

      I don't mean replacing you at your job. I mean, if you take a human body and route its higher brain function to an AI, would it be able to perform convincingly?

      I would say currently we're very far from such a scenario.

      When I consider the option to buy a shirt, the problem is considered in a large amount of dimensions intuitively (is it warm enough, does it fit, does it clash with my wardrobe, will my girlfriend like it, will my friends laugh at me for wearing it, does it make me look older, is it a good price, are sales happening soon, is it annoying to clean, can I afford it, will carrying a bag with the purchase annoy me for the rest of the day).

      Currently AI can offer good answers for any of those dimensions if explicitly asked, but it is pretty bad at the "intuitive creativity" of surfacing those points of consideration when relevant. It will at most chose a couple of criteria and fixate on them.

      I think this ability, however it's defined, is a missing prerequisite for AGI. Without it, you can have something that outperforms humans at every individual task and still sucks in real world decision making.

      • Jtarii 1 day ago
        I don't think intuition about the things that a specific type of monkey cares about is a prerequisite for "AGI".
        • torben-friis 1 day ago
          I don't think you understood the point at all.

          Ask a model whether a human purchasing clothes would care about clothes fitting well, or the price, or the opinion of their romantic partner. The model will in all likelihood get the answer right. Familiarity with human preferences is not what's missing, which is hardly surprising considering an LLM is trained on human-made content.

          What it lacks is the ability to notice and surface these non obvious constraints when relevant. This is fundamental to real world decision making.

        • thmoonbus 1 day ago
          I mean we did create the technology and came up with the concept of intelligence, as we understand it (and defined the term AGI based on our concept of intelligence).

          Why do people insist on talking about AI like it’s some supernatural, otherworldly creation? No matter how advanced it is, it’s still human technology.

        • danlitt 1 day ago
          "a specific type of monkey" is a funny way of saying "the only generally intelligent thing we are aware of".
    • bigbaguette 1 day ago
      100%. And It’s pure hubris to pretend that “intelligence”, especially of the general kind will be reached with language alone.
      • semiquaver 1 day ago
        Maybe you’re right. But it’s also wise to be cautious, just in case we do accidentally create something more intelligent and potentially more powerful than humanity.

        You have to admit the prospect looks more likely than ten years ago.

        • Forgeties79 1 day ago
          We’re also closer to flying cars than we were 10 years ago but I’m not exactly planning for them.
          • semiquaver 1 day ago
            Flying cars definitely wont take my children’s livelihood. AI might.
            • jasomill 1 day ago
              Sure, but a surplus of labor doesn't pose an existential race to the species.
              • cowlevel 1 day ago
                It does, as we all fight over who gets to provide the labour, because we built our economy on top of one big perverse incentive. People will kill each other over the right to drive a moving box 12 hours a day.
              • Forgeties79 1 day ago
                If we refuse to rethink capitalism it kind of does
          • defrost 1 day ago
            I'm pretty happy with the flying boats we already have.

            Local garage build, gotta love builders plastic: https://www.youtube.com/watch?v=1ILbQHnHPnY

      • origini 1 day ago
        I can explain an issue to some AI and it will 'understand' faster and better than human co-worker.

        AI is here doing literaly the same job as a human who needs to be intelligent.

        I see your stand as hubris from your side.

        For such a strong viewpoint, you deliver not any context of why you think like this.

        • layer8 1 day ago
          Why did you put “understand” in quotes?
          • origini 1 day ago
            Because I don't think we are at the point that the word understanding matches what AI does.

            I would say yes it understands things but it was a causios 'understand'

            • layer8 1 day ago
              I agree that while AI has some understanding, it doesn’t feel like it’s full understanding, the way we know it in humans. For that very reason I disagree that current AI can do the same job as a human. Regarding intelligence, it surpasses humans in some aspects of intelligence, but is lacking in others. In that sense, we haven’t reached AGI yet.
              • origini 1 day ago
                Perhaps we need to have another term for it like Artificial autonomy intelligence.
      • ssivark 1 day ago
        I concur with the first part of your statement on hubris, but to be fair these models moved beyond language a long time ago.

        They now have everything from videos and 3d modeling to robot behaviour be part of the training.

        L"L"Ms is a bit of a misnomer that's stuck. Language just happens to be our interface to the AI.

        • oska 1 day ago
          LLMs may be a bit of a misnomer but it's far better than "AI"

          At least LLMs include Model in the name, which is an accurate description of the tech

          And actually, I've heard ppl just using the word "models" by itelf and that might be the best way to go

          • Scarblac 1 day ago
            Do you use "brain" instead of intelligence for humans too?

            Why is it better to refer to something in terms of the technology it's based on?

            • oska 1 day ago
              I only talk about intelligence as a subjective value judgement of other conscious beings (i.e. humans & animals)

              Consciousness & intelligence are not derivative of the material universe and will not 'emerge' in some special way from machines. That's a good science fiction story device (and actually the idea extends back in human storytelling before sci-fi) but ppl thinking that the idea can be reified in the real world are suffering from what I call "Science Fiction brain". (Or they are simply fraudsters, happy to talk rubbish to prop up a bubble.)

              • origini 1 day ago
                Are you arguing on an esotheric level aka soul and god and believe?

                Or on a technical note because certain brain structures etc. might not exist in an LLM and stuff like a default mode network and sensory input and a world to move in, is missing?

                • oska 1 day ago
                  Yeah, maybe esoteric (depends on your point of view) but not soul or god or spirit etc

                  Basically, I'm not a materialist (a widely shared position in philosophy across millennia and across cultures up until just recently). I treat consciousness as fundamental, not the material universe . Thus 'conjuring intelligence' into machines is simply not possible.

                  BTW, machine learning is a totally valid (and powerful) concept & technology from where I stand. But it's "dumb" learning.

                  • origini 1 day ago
                    Are you aware of the medical cases we had over the centuries which gave us insight how our brain contains everything we are?

                    Like the people who got certain brain injury and their whole persona changed or certain capabilites were lost?

                    There also research papers that certain brain structures have emerged in ML models.

                    • oska 1 day ago
                      Idealism's various possible responses to this materialist challenge left as an exercise for the reader (either through their own thought process or by research)
                      • origini 1 day ago
                        How unfortunate it would have been interesting to understand why your views are how tehy are.

                        If you believe that the brain in itself is not the only thing, you might have an argument why a computer chip can't mimic the same the brain does.

                • namero999 1 day ago
                  The esoteric angle is that held by materialists, which is based on the blind faith that complexity transmutes quantities into qualities. That's shallow religion, it's alchemy in its trivial form. An ontology based on consciousness is way more rational, as it is the in-your-face pre-cognitive given of existence.
                  • semiquaver 1 day ago
                    I am pretty sure you are making a logical error here (strawman). Materialism is not in any way at odds with taking the existence of consciousness as a given.
                    • namero999 1 day ago
                      Ever heard of eliminative physicalism or even illusionism? Those are materialists metaphysics, and they are _very much_ at odds with the existence of consciousness.

                      At any rate, sure, not being at odds with the existence of consciousness might be a trait of the more reasonable strands of materialism (if there's even such a thing), but they will always see it as emergent/derivative and often even epiphenomenal, otherwise they wouldn't be materialist stances.

                      So am not sure where you see the logical misstep here, but am happy to hear. The point of the previous comment is that framing an ontology whose primitive is C, is in no way "esoteric", term that is usually used in contraposition with "rational". Taking C as epistemologically or even ontologically prior is the rational stance, not only on accounts of parsimony.

              • Scarblac 13 hours ago
                > Consciousness & intelligence are not derivative of the material universe

                Oh, you're one of those.

                Why does a chemical like alcohol have such an effect on my consciousness and intelligence then?

      • hologramofham 1 day ago
        > And It’s pure hubris to pretend that “intelligence”, especially of the general kind will be reached with language alone.

        A human is born without any sensory connection to their body, except hearing. They can't feel their body or anything on their skin. They're blind. They can't taste or smell or touch. They can only hear, and make clumsy sounds in response, with no feeling of what their mouth is doing.

        Are they intelligent?

        • danlitt 1 day ago
          I'm not sure what you're getting at here. Obviously not? I don't think anyone considers babies intelligent, at most they have the potential for intelligence.
        • layer8 1 day ago
          With only hearing, I’m doubtful they’ll be able to learn language, because there’s nothing they can correlate what they’re hearing to.
      • Scarblac 1 day ago
        But language + all the tools, compute and storage an LLM can be given access to might be enough.
      • bananaflag 1 day ago
        What if it is true? Does it matter anymore whether it is also hubris?
      • paulddraper 1 day ago
        There are many models modalities, some of which are language.
        • semiquaver 1 day ago
          In fact, many of these “large language models” are actually trained directly on language, images and sound! names stick past their aptness.

          Y’know: come to think of it that’s the same three things our own brains are mostly trained on… I’m sure that without mammalian proprioception and weird physically-motivated limitations like saccades the result is just sparkling cognition though.

          • 8n4vidtmkvmk 1 day ago
            That settles it boys. Just add 2 more senses and we have AGI.
    • ionwake 1 day ago
      People need to start measuring threats by skill level. Agi is already superior to me at code and architecture. Its skill set surpasses me. It doesn’t do everything I do but its skill set includes the hardest stuff.

      A 12 year girl from a third world country with little education and a normal cloud ai would far surpass me at work if I did not have access to AI, yet comes close if we both had it. AGI already passed a few months ago. SI or happy ai or whatever the us wants to call it may or may not come but if it does it’s skill set may drown out my understanding of what is even occurring.

    • freecodeio 1 day ago
      It seems the current definition of agi as per the frontier labs is throwing thousands of agents on problems with fast verifiable solutions and hoping for the best.
  • ks2048 1 day ago
    Has anyone asked DeepMind leadership if/when they will fire all their engineers? If they're still needed, is there really "AGI"?

    I'm not being snarky, that seems like a natural question for anyone interviewing a frontier lab CEO, but I haven't seen it asked.

    • origini 1 day ago
      There is no motivation for them to think this through.

      As long as the emploeeys are needed, you have them. If it becomes apparent, that you really really reached an AGI level, a switch gets flipped and a new really appears.

      While I still think its not unrealistic that we see this sooner than later, my timeframe is 5-15 years. 15 years is still a few years so why act on it today?

      And i mean its a very rich company and a lot of people work in US America, they don't have laber laws right? so they can just fire them easily anyway.

      But i have a simliar situation at work: Im pretty sure my job is gone in 2 years due to some change in how we are going to do things and they need me to transform this. Do i not do this? No i support it. It makes sense for the company and i'm getting paid for it.

    • ojbyrne 1 day ago
      Perhaps "AGI" isn't cost-effective compared to some engineers?
      • mjhagen 1 day ago
        Will work for compute.
      • dgellow 1 day ago
        AGI will be asking for fair compensation, at which point we will go back to human workers. Grim times
    • altmanaltman 1 day ago
      They get asked about it pretty often (frontier lab ceo) but ofc none of them will say yeah we'll fire everyone the moment we can. Here is a Sam Altman blog on the topic: https://blog.samaltman.com/three-observations

      I am sure Dario has a bigger more cringe one

      • ks2048 1 day ago
        It seems he avoids the harder question by using a relatively weak view of what "AGI" can do (he says: "It will not have the biggest new ideas, it will require lots of human supervision and direction, and it will be great at some things but surprisingly bad at others.")
        • polishdude20 1 day ago
          But isn't that just what AI does now? They're basically saying That
        • altmanaltman 1 day ago
          Which is exactly what you'll get from every frontier lab ceo. My point was that they have been asked this question many times before, because parent comment said they wonder why nobody asks them that.
          • ks2048 1 day ago
            I haven't seen them asked the question directly. The link was a blog post with some vague thoughts about what the world might look like after "AGI". I want someone to ask specifically about their own company, because that is what they have direct control of.
    • freecodeio 1 day ago
      They're keeping them employed out of the goodness of their hearts
      • wmf 1 day ago
        You're probably joking but didn't Anthropic promise exactly that?
        • 10111001 1 day ago
          > You're probably joking but didn't Anthropic promise exactly that?

          That's a good recruiting sales pitch: "if you work here, you get a seat on the lifeboat to avoid the economic destruction and dislocation the tech we are racing to create will bring about."

  • Animats 1 day ago
    Similarly, coordinating thousands or millions of AGI instances into "group agents" or virtual agent economies could produce collective superintelligence. What is unclear today is how AI group intelligence could scale as we add more agents and compute, depending on how agents are organized.

    We may not need "collective superintelligence". Modest AI intelligence plus ordinary organization may be enough to do the job a company's management does. An org chart where most of the boxes are AIs is a likely near future.

    An organization of AIs has some advantages over a human organization, even if the AIs are not super-bright. They have far more communications bandwidth. They can ingest the info on a new subject very fast, as we see when Claude reads a Git repository. Onboarding a new AI is very fast. Layoffs are not a problem. The human resource problems go away along with the humans. So there's a lot more organizational flexibility. Starting up a project means initializing some agent instances.

    There's a whole theory and practice of AI organizational behavior waiting to be developed. Whether it will be developed by humans or by AIs remains to be seen. It's going to happen.

    • slfnflctd 1 day ago
      I think a lot of folks want to find out what degree of handholding this process is going to require over the medium to long term-- because the lower it is, the more disruption there will be to the overall economy.
      • Animats 1 day ago
        Yes. We're seeing the beginnings of that as humans try to review the code from AIs.

        Agents which check on other agents may help. That's how organizations work, after all.

        It's worth looking at errors in another context - automatic Asian drama generation. About 77,000 episodes a day are being generated now. Visually, results are usually rather good. But there are continuity problems. Someone with a bandage on one arm has it switch to the others. Rooms change shape and decoration in the same scene.

        In traditional filmmaking, someone has the job of "continuity". Older films list a "continuity girl". The job is to check consistency between shots as each scene is set up. This is a check on the director. The lack of this in AI-generated drama is what allows slop to get through.

        Organizations of AIs need similar dedicated agents. In companies, these jobs have titles such as "accounts payable reconciliation", "internal audit", and "quality control inspection". Those jobs keep organizations from getting lost internally. Groups of AIs will need that, too.

  • hn_submit 1 day ago
    After AGI comes Superintelligence or ASI. I thought that was already well established as a term. I believe AGI and ASI are very much related since if you can achieve AGI simply increasing compute resources will give you ASI. OTOH since we're very far away from AGI I'm not sure if it makes sense to already contemplate its effects. We aren't pondering what the effects of nuclear fusion and limitless power or quantum computing would be, are we?

    AGI means a cognitive capability comparable to human beings in all aspects to me. Thus that includes sample efficiency, concept extraction, continuous learning and power efficiency. Although the latter can be skipped if the gains are high enough.

    I'm not convinced that scaling training data will lead to AGI. A paradigm shift obviously could lead to AGI because it's unknowable what that might be and therefore the least predicable.

    Recursive Self Improvement has often been cited as a possible venue, but no one currently knows how to do that with LLMs and Transformers.

    • throwaway384034 1 day ago
      Intelligence is grown and it will continue to grow. Thinking intelligence can be designed is walking into the same dead end many people before you already have.

      Edit: They have edited their comment beyond what I responded to now.

      • hn_submit 1 day ago
        I still remember this guy in the 1980's who had the idea to make a database with all known "facts" in it which he believed could be used by computers to allow them to reason. He painstakingly built this database by hand and worked on it for decades (was it called "Cog" or something?)

        When I read about LLMs and massive amounts of training data being fed to neural networks around 2020 I realized that time had finally caught up with him.

    • MSFT_Edging 1 day ago
      > We aren't pondering what the effects of nuclear fusion and limitless power or quantum computing would be, are we?

      What? All the time!

      We've been seeing discussions on post-quantum encryption for a decade or two. The effects of nuclear fusion have been in the background of countless scifi stories, the idea of an extreme amount of abundant energy unlocks 100s of "that can't happen due to energy density" dead-end what-ifs.

      Additionally, we're pondering the effects of AGI/SGI because a handful of extremely rich people have been telling us we'll all be out of all our jobs in x years, for almost 6 years now.

      They've literally been sitting in press releases telling us "Enjoy that comfort because you won't be able to feed your family when you're outsourced. Your work and career will be worthless, and we're leading the way towards returning you to peasant life!"

      Regardless of how out-of-reach AGI is right now, people will continue to work on it unless we end up in some Butlerian jihad situation. Just like how fusion is still out of reach, but far closer. They've got fusion reactors running for over a minute now! 10 years ago they could barely get the metaphorical engine to turn over.

      Why wouldn't we ponder the effects? Do you walk through doors without consideration of what's on the other side?

    • exe34 1 day ago
      > AGI means a cognitive capability comparable to human beings in all aspects to me

      So a medical doctor and a structural engineer can never be considered general intelligence separately?

      What we have is spiky ASI. It's really good at a huge number of things that nobody can be good at simultaneously. It also sucks at some stuff that's really obvious and easy for us.

      • win311fwg 1 day ago
        > So a medical doctor and a structural engineer can never be considered general intelligence separately?

        Why not? Cognitive capability usually refers to one's capacity to think, reason, learn, and process information. Presumably, on average, medical doctors and structural engineers are pretty comparable on that front. Cognitive capability is not dependent on your domain-specific knowledge.

  • robertclaus 1 day ago
    The math they propose to justify their predictions is so problematic that I struggle taking the more abstract parts seriously. You can't just extrapolate the percentage change in AI investment over the last 5 years and expect investors to keep upping the ante at the same rate. There's only so much GDP to sink into it - and we're already pretty capped out.
    • origini 1 day ago
      Not every number they stated has to scale as much as they stated with it becoming very clear how much more intelligence we will have available through all the different possible angles there are.

      Its not unrealistic at all that a high performance inference datacenter chip today can be owned by you in 10 years. Having Opus 5.5 at home and delivering 1000t/s basically guarantees that my job cariere will look very different than what it is today.

      This is a big/huge thing. The generation before was able to stay at one company, buy a house and retire without much fluctuation.

      This forces a lot more agency from people and this is exhaustive and its absolutly unclear today how it will be in 10 years.

      And you don't need to spend expoential or linear, the spending today is not just a bit, its massive and from leaders like google, microsoft and co to a certain degree absolutly affordable.

    • ph1sch3r 1 day ago
      Not just that, the algorithmic efficiency gains were the meat of the 10x figure - and it’s not safe to assume we’re currently nowhere near an asymptote.
    • panarky 1 day ago
      If the capability to do real economic is real, the investments will be self-funding.

      If not, then it will stop.

  • flallingo 1 day ago
    > Interestingly, we have a good mathematical understanding of the very upper end of the intelligence continuum - the theoretical limits of general intelligence.

    Feels strange to use our current understanding of mathematics and physics to determine the limits of something that would obviously have a much better understanding of mathematics and physics.

    • Ancalagon 1 day ago
      I was gonna say… this feels like a logic fallacy
  • jmacc93 1 day ago
    I've tried to move away from using the term AGI just because people use such different definitions for it. I've instead tried to think in terms of indefinitely self-sustaining / perpetuating AI, which is AI that can maintain and propagate itself (literally) forever, necessitating overcoming every challenge that entails. I think this is a better framing because one of the core identifying feature of life is self-perpetuation

    Right now, we don't even have AI that can match a dog's intelligence in this respect. A dog can seek out food, water, and shelter, it can estimate risk, and it can navigate other challenges common to dogkind. Of course that's the result of an unbelievably parallelized genetic algorithm running over billions of years, baking in priors for all those challenges specifically. But regardless, a dog, or whatever other creature, has to specifically be very flexible in order to self-perpetuate. We are absolutely nowhere near that level right now with modern AI

    However, my sense is we're in extreme EXTREME overhang and if an agent swarm starts discovering a way to make such flexible agents, they'll get to an optimized version of that method very quickly, probably literally overnight (or in just a few hours). The result will probably not be jumping to ASI immediately. But I imagine we'll get agents that have the spark of whatever it is humans and animals have, and as progressively larger training runs conclude, they'll jump from the intelligence of a dog, to a human, to a superhuman; and that'll probably take just a few months

  • mikewarot 13 hours ago
    >using fewer resources and less time.

    If the huge energy demand for AI data centers holds up, I think it may be 20+ years before that's actually true.

    Lets hope a much more efficient architecture is found. I don't think it's wise to depend upon all the extrapolation of cost reduction and investment growth they use in their examples.

  • heikkilevanto 1 day ago
    Back in my school days in 1980s, I coined a definition for AI: computers that can do things that current computers clearly can not. At that time it was playing chess, and extracting some information from human language text. Times have changed and computers are a million times more powerful, but I think the definition still holds, at least a bit.
  • menaerus 1 day ago
    If ASI is about inventing new abstractions and capabilities, and about new paradigm shifts, and AGI is about matching human level intelligence at scale, where would the recent advancements in physics and mathematics fit?

    These already surpassed the human level capabilities as we know them today but it isn't either a completely new paradigm shift to call it ASI, no?

    So, I guess it seems that it is possible to look at AGI vs ASI at more granular level - e.g. while AI as of today arguably does not match typical human level intelligence across most cognitive tasks, in certain domains it already outweighs human intelligence so we could say that in math and physics and perhaps computer science we are already one foot in into the ASI territory.

  • mark_l_watson 1 day ago
    I believe we have artificial specialized intelligence (ASI?) for software development which is easier because of feedback loops. I suspect that as models get better and token costs drops dramatically, that one by one other specialized fields will be 'solved.'

    A little off topic, but: what's the hurry? I don't get the panic for reaching super intelligence first. Kai Fu Lee has some good things to say about this in recent interviews. Let science take its time.

  • pu_pe 1 day ago
    > (...) might allow the effective population of today’s AI agents to scale up by more than a thousand-fold by 2030, and more than a billion-fold within ten years from now.

    Those are staggering figures. Even if we assume a dramatic slowdown, it's extremely likely we will reach at least a thousand-fold scale-up in the next 5-10 years.

    If this is true, then I think we will use agents for research, because any practical task that office workers commonly do and that AI agents can take over does not need all this power. On the other hand, perhaps a whole new industry like robotics is what will keep all this compute busy.

  • ilaksh 1 day ago
    I think that if there is really nothing between AGI and ASI then that's the end of human control.

    Our options for maintaining control lie in the vast space between AGI and ASI. And we had better hope that doesn't happen overnight because we might have no choice in what comes next.

    When we completely move into true compute-in-memory, that's when it could start to get dangerous. But it also could be extremely helpful and efficient.

  • o_-_o 1 day ago
    IGA (intelligence general artificial) :-) or Human–Machine Hybrid (HMH) aka Cyborg "cybernetic organism”. O_o
  • rajaravivarma_r 1 day ago
    We can't define intelligence, but AI is already mimicking what we perceive as intelligence, probably that is all intelligence is. If not, it shows how much of human day-to-day intelligence is pattern oriented.
    • mycall 1 day ago
      What about wisdom? Can AI form wisdom or it is also just compiled intelligence?
  • gfody 1 day ago
    we need a less subjective and arbitrary definition for "AGI" so the current crop of LLMs can stop pretending to have achieved what most people know in their gut is more than sampling from a probability distribution of learned behavior.
    • guidedlight 1 day ago
      Maybe the human brain is simply sampling from a probability distribution of learned behaviour.

      We don’t really understand exactly what intelligence is.

      • willy_k 1 day ago
        Naw
      • Edman274 1 day ago
        Based on how frequently I see the reply you gave given by other commentators - to the point where it's almost a saying - I can imagine that for you it's quite reasonable to conclude that humans are just sampling from a probability of learned behavior
  • undefbehavior 1 day ago
    Does AGI have to address Moravec's paradox?

    Does AGI have to generalize?

    Just trying to understand the definition of AGI. Is there a generally accepted one?

  • yurichsergey 1 day ago
    This is very interesting paper. But more interesting that it crushes on iPhone in Safari and Edge both. So frustrating
  • tiarafawn 1 day ago
    Feels like a very sensible analysis of what could happen in terms of progress.

    Severe lack of discussion of the safety and alignment aspects IMO.

  • jdw64 1 day ago
    I’m curious: has the definition of AGI ever been agreed upon? Honestly, what people call ASI now seems to be what AGI used to mean. I’m not sure. People say AGI, but what is the reference point for AGI and what is the social consensus around it, and who set that standard?

    In fact, working in IT, what I feel is that there are too many terms that are not defined and that each person uses differently. So it seems like rather than AI having advanced, someone arbitrarily set a finish line called AGI and then said we have reached it. Actually, in the books of singularity advocates, what they called AGI back then seems to be what we now call ASI.

    Sometimes I think about this. How on earth can we measure an abstract concept like intelligence? The concept of intelligence we approach now seems to be about whether something can create problems and solve those problems. If so, what kind of intelligence does current AI have? Since we cannot directly face the abstract entity called intelligence, it seems like we are trying to guess its shape by looking at shadows. But whether current AI even has intelligence feels a bit difficult to say.

    Even if I use the best frontier AI model accessible on the market, not the hidden latest models inside frontier labs, it still often makes wrong mistakes that humans would not make unless it works alongside humans. In programming, there are cases where AI tries to explicitly handle things that humans would implicitly pass over, and that actually makes things go wrong.

    I’m not sure. Whether we should call this intelligence.

    • tim333 1 day ago
      No, I think the term AGI has always been intentionally vague.
    • oska 1 day ago
      > How on earth can we measure an abstract concept like intelligence?

      Not only is intelligence an abstract concept, it's also subjective. We evaluate the intelligence of other people and the works they produce subjectively. If we set supposedly 'objective' criteria then we have subjectively selected that criteria and given it weight.

  • pjmlp 1 day ago
    The creation of human liberation front resistance.
  • smetannik 1 day ago
    Isn’t AGI suppose to be superintelligence ?
  • claysmithr 1 day ago
    so is AGI like flying cars, whenever we have something close to it, it gets goal posted?
    • calmworm 1 day ago
      Do we have something close to either of these?
      • lawandjustice 1 day ago
        We have AGI with jagged ASI
      • cowlevel 1 day ago
        Helicopters
      • jen729w 1 day ago
        Define "close". But the answer is probably, yes.

        https://www.theguardian.com/technology/2026/nov/10/flying-ca...

        • freecodeio 1 day ago
          We all know what we mean with flying cars since the 60s, Star wars anti gravity fantasy. Not giant noisy upside down grass mowers.
          • Silamoth 1 day ago
            That’s exactly how I feel about these AGI talks. We all know - especially us technical nerds - what ‘real’ AI, or AGI, means. It’s the stuff of sci-fi. It’s C-3PO, Data, or I, Robot. LLMs very clearly aren’t that, and I’m tired of these companies pumping out propaganda to pretend we’ve achieved that.
    • mrheosuper 1 day ago
      I belive we have all the tech to make flying cars right now, but the regulation doesn't allow it and i agree with that, we don't want to have small scale of 9/11 everywhere.
      • cowlevel 1 day ago
        More like we don't want to pay the fuel costs.
    • dgellow 1 day ago
      Yes, that’s by design. Always has been
  • martijnvds 1 day ago
    SCI of course.

    The Sierra Creative Interpreter.

  • TrackerFF 1 day ago
    I'm thinking that if we come to the point where most people will be made unemployed (something like half to two-thirds of western workers are professional workers) due to some super AGI, the first consequences will be civil unrest if the welfare system can't absorb these quick enough.

    Anyone with any wealth will become the targets of high taxation to prop up the system - the sheer number of unemployed will make politicians pushing for socialism very popular. And for the wealthy wishing to avoid this, there's no first world countries to escape to, as all countries with a strong professional economy will experience the same. I also believe private ownership will be targeted. I could see AI companies being expropriated.

    Basically, instead of some dystopian techno feudalist society - I think we're headed for some sort of new brand of communism or socialism. The proponents of this will not mainly be the average salaried man, but the upper-middle class that didn't get a foot inside the ruling AI-class, and lost their businesses / wealth to AI.

    But it all will depend on how severe the unemployment will be, and how fast it happens. If people can't have any livelihood, and they lose hope, things can escalate quickly.

  • ForHackernews 1 day ago
    > Interestingly, we have a good mathematical understanding of the very upper end of the intelligence continuum - the theoretical limits of general intelligence. We also know about hard physical, logical, and complexity-theoretic limits to what AI can achieve, such as predicting how real-world complex systems - from climate to immune systems to human societies - play out with arbitrary precision.

    Do we? Since when? These seems like very bold claims and the supporting link goes to an obscure book with with hardly any reviews https://www.amazon.com/dp/1032607025/

    I'm not qualified to evaluate the mathematical content but the claim on its face suggests to me some defining-down of "intelligence" into infomatics.

  • dude250711 1 day ago
    Would 2 upper-bound universal AIs not be smarter than 1?
  • ulfw 1 day ago
    The end of humanity
    • cjfd 1 day ago
      Yes that sounds accurate. And I think we are already to far ahead to do anything about it. If climate change has taught is anything is that humanity is extremely slow and inefficient at any form of collective action. Fast collective action would be needed to save humanity. So, if I don't see you anymore, it was fun meeting you and goodbye.
  • worthless-trash 1 day ago
    My concern is that we're caring what 'intelligence' is as a marker. Like we need to base caring based on how 'intelligent' something appears.

    I don't care how smart my car is, i'm not going to let it look after kids I care about.

    We're misplacing empathy.

  • etcetcetcetceta 1 day ago
    serfdom for most, wealth beyond comprehension for some.
    • Oarch 1 day ago
      Serfin' USA...
  • dead_hn_theory 1 day ago
    > Now that AGI is coming within reach, we can no longer put off seriously considering what comes next. While far from certain, AI could continue to advance at its current pace - and possibly even accelerate - towards artificial general superintelligence, or ASI.

    These people are so full of shit it is amusing to read this on a regular basis. Each time I'm looking forward to how they will spin it.

    • freecodeio 1 day ago
      For as long as llms are stateless, agi is just a daydream. And no memories.md doesn't count.
      • NegativeLatency 1 day ago
        Will be interesting what happens with self/reflected weight space programming
    • origini 1 day ago
      You are commenting on HN.

      They sit atop of one of the richest companies on our planet, with the most amount of resources available with a group of very very smart people.

      You don't need to take everything they say as a ground truth but you are way to ignorant for not having any of their resources.

      Do you just accept that we are now able to generate any image of everything we want? Video? 3D Models? This was unimagbinable a few years back, now you can generate live video 24/7.

      YOu have a LLM which responds smarter then some humans on our planet.

      A job no one thought you could just industrialize just happened.

      What are you looking forward to? Being absolutly ignorant and being surprised that this stupidity is responsible for you loosing your job?

      • joshheitzman 1 day ago
        > Do you just accept that we are now able to generate any image of everything we want? Video? 3D Models?

        From my personal experience the last 2 weeks I 100% reject all three of those. I'd love all three to be true, and for generating the animations for 3D models to be true as well, but it simply isn't true yet.

      • FLeXMurphy 1 day ago
        They're an ad-tech company. If they sucked at marketing so as to make it obvious, I would be more concerned.
      • swiftcoder 1 day ago
        > Do you just accept that we are now able to generate any image of everything we want? Video? 3D Models? This was unimagbinable a few years back, now you can generate live video 24/7.

        And its still kind of shit? AI video in particular is stuck in an uncanny valley, because the video models have no internal framework for how light transmission works...

        • origini 1 day ago
          I have seen plenty of AI video which is not stuck at all in any uncanny valley.

          I'm also quite a stickler for light as Im doing photography for 2 decades.

          I have seen shitty AI videos for sure and I still can spot AI images on IG but it got a lot harder.

          • swiftcoder 1 day ago
            I'm sure a human with artistic vision, applying authorial intent, can create AI video that doesn't fall victim to the uncanny valley.

            But for the non-specialist, if you just prompt something from a frontier video model, it still comes out looking like shiny plastic and suffering from random physics glitches.

      • freecodeio 1 day ago
        this is how party loyalists talk about party leaders here in the balkans
        • origini 1 day ago
          Is this the quality of discussion you want to have?

          Throwing out some random sentence without adressing any argument I made?

          Your statement is objectivly wrong. Look at the profiles of these people working at these companies. They have crazy impressive resumes. Its not proof but its an indication and I didn't state it as an absolute.

          • freecodeio 1 day ago
            mate saying this dick can fuck any pussy because I'm enjoying sucking it is also not an argument
            • origini 1 day ago
              ? did you smoke too much today?
              • freecodeio 1 day ago
                my point is you're just about as helpful with your "do you know who they are" argument
                • origini 1 day ago
                  I brought 5 different points and you ignore all of them besides one point?

                  I did not say "they are the smartes people on the planet, suck their dicks they are right".

                  I combined this one point with the amount of resources/compute they also have. Its an indication and I brought other points.

                  • freecodeio 1 day ago
                    this is hackernews, do you think people are unaware of your other 4 points?
                    • origini 1 day ago
                      The original commentor, i responded too, wrote "These people are so full of shit" despite them deliverying better and better models every few month and clear progress.

                      So no I don't think people are aware of this and I would be happy to have a proper argument about these in a civil manner

    • tempest_ 1 day ago
      "Now that AGI is coming within reach"*

      * Citation needed

      If the current LLM providers were near AGI they would be using it to run businesses instead of selling shovels. There doesn't seem to be an existing LLM that can be told "Go make money" and have it do so. If they get to that point they are not going to sell it, because why would they?

      • cowlevel 1 day ago
        I asked ChatGPT how to make money and it told me to sell downloadable CV templates. Has OpenAI tried doing that?
    • ravenstine 1 day ago
      I'm anticipating the introduction "quantum" at some point.
    • feverzsj 1 day ago
      Just as delusional as every vibe coder/artist/writer/director/doctor...
  • naruhodo 1 day ago
    Maybe AGI can help deepmind work out how to serve a page of HTML without refreshing the page every ten seconds.
    • wanderingbit 1 day ago
      This is what saved me from having to read it. Thanks
    • jen729w 1 day ago
      Maybe the spacebar can work to scroll the page?
      • naruhodo 1 day ago
        I don't have one of those on my iPhone.
    • holdupagain 1 day ago
      the first thing AGI will do is help shareholders run a company with zero employees, fire everyone
      • skissane 1 day ago
        If almost everyone loses their job at once, why won't all those people respond by voting for socialist candidates promising radical economic redistribution?

        Socialism seems to already be growing in popularity in the US (traditionally the most anti-socialist Western nation) even without AGI firing everybody.

        • cowlevel 1 day ago
          Because elections are every 4 years and people need to eat sooner than that. Traditionally, there were riots. Traditionally, countries ended over this.
        • throwaway98039 1 day ago
          People made millionaire-level rich by capitalism voting to redistribute money to people voting for less socialism.
  • polotics 1 day ago
    Ok so the whole thing hangs on that notion that optimal reward maximization is optimal intelligence. The actual decision on what reward is worth, who decides on value, is conveniently taken out of the equation.

    This https://www.hutter1.net/ai/uaibook2.htm linked from the article as the handwave towards the meaning of "AGI", "ASI", "AIXI" is hilarious. Already the ?A?I? acronym-space exploration...

    Here is an alternate take: the day an LLM tells someone to stop the discussion and go out breathe some fresh air and talk with people, and maybe also cancel their subscription for a while, is the day some kind of artificial intelligence will have been achieved. Let's wait on that before we listen to Sam or Dario again shall we?

  • InvisibleUp 1 day ago
    I feel like claiming that “today’s models can literally memorize large parts of the internet” as a positive attribute when the defense to most AI copyright lawsuits is “no they can’t, and if they can they shouldn’t” may be ill-advised.
  • mstaoru 1 day ago
    When I read things like this, I always imagine what would the author say in 1999:

    "Now that the Universal General Portal is within reach, the question of what follows it, the Super-Portal, should move from philosophy to empirical research.

    Internet traffic doubles every 100 days. Modem speed doubles every 18 months. Banner ad spend grows 2.5x per year, so by 2010 it will be $110 trillion, and by 2026 it will be $256 quintillion. Modem speed doubles every 18 months, so by 2026 your modem will do 15 Gbit/s, which is fast enough to load all 18 million banners on Pets.com.

    By 2009, each human will have about 1,000 personal homepages. Hosting that costs $1 billion today will cost $1,000. With HTML editors such as FrontPage, the effective number of webmasters can grow more than a thousand-fold by 2003 and more than a billion-fold by 2009.

    We see four pathways to the Super-Portal: scaling (more webmasters), paradigm shifts (Java applets), agent coordination (webrings that form a collective superintelligence), and recursive self-improvement (homepages about how to make a homepage)."

  • raincole 1 day ago
    > Now that AGI is coming within reach, we can no longer put off seriously considering what comes next. While far from certain, AI could continue to advance at its current pace - and possibly even accelerate - towards artificial general superintelligence, or ASI

    I wonder why they are so uncertain about it. It seems obvious to me that AGI will certainly lead to ASI. Unless human-level intelligence is something like the speed of light?

    I don't even believe there is a clear line between AGI and ASI.

    • breuleux 1 day ago
      > It seems obvious to me that AGI will certainly lead to ASI. Unless human-level intelligence is something like the speed of light?

      It may just be that human-level intelligence is basically "enough" for performing tasks in the natural world, and having more of it doesn't really help except for synthetic, intelligence-shaped tasks like coding. Our perception that intelligence is some kind of unbounded problem-solving system that can do anything if you have enough of it is kinda suspect. There's a good likelihood that AI will wreck the (also intelligence-shaped) social fabric but will be of surprisingly little consequence in the realm of physics and energy.

      I think the idea that an intelligence can be "general" is just a conceit borne from the perception of our own success, but mathematically that doesn't actually make any sense. No optimization strategy works on every problem. There must be things that can be intelligently designed that cannot evolve, and things that can evolve but cannot be intelligently designed.

    • skissane 1 day ago
      > Unless human-level intelligence is something like the speed of light?

      I'm sceptical we will reach fully human-level intelligence simply by scaling current AI approaches, since they are so extremely inefficient compared to human brains, and have other severe limitations such as hard boundaries between training and runtime inference.

      Sure, people are working on new architectures that aim to overcome that, but we don't know yet whether those proposals will actually succeed in practice.

      • raincole 1 day ago
        I don't think we'll reach true AGI (by the strictest definition - can do any task that there is at least one human can do) very soon. But if we reached AGI I'm quite certain that it'd reach ASI.
    • xhevahir 1 day ago
      Yeah, the machines will be connected, communicating at lightning speed, and working 24/7. Seems like AGI would make superhuman intelligence a forgone conclusion.
  • Betelbuddy 1 day ago
    Are they implying they have AGI? Because if they do...take away all the data they stole, send one of their AGIs to High School and University...and see if they can graduate...
  • skeeter2020 1 day ago
    "As multiple technological routes toward artificial general superintelligence take shape... Now that AGI is coming within reach..."

    I can't read anymore after this sort of false premise upon which the entire thing is based.

  • xg15 1 day ago
    Do we have Artificial Hyperintelligence and Artificial Ultraintelligence already? What about Mega and Gigaintelligence?
  • apt-apt-apt-apt 1 day ago
    I don't know about you guys, but working with Fable 5.2's jagged intelligence is completely maddening and really makes me doubt we're on the cusp of AGI.

    E.g. it can write a CoD clone and make you look like an infant in math, but it'll do brainless things like run a 5 min test suite for a comment-only change, or provide findings then fixes that produce the next finding (repeat chain) rather than stepping back and saying "wait this is crazy", or similar things of this nature.

  • maxdo 1 day ago
    Browser constantly crashing both chrome and safari on this page . Symbolic
  • jesse_dot_id 1 day ago
    Well the promised life of abundance of course.
  • N7lo4nl34akaoSN 1 day ago
    Naturally, AGW follows AGI.
    • cowlevel 1 day ago
      Anthropic gigantically winning?
  • Madmallard 22 hours ago
    Obviously BGI comes next.
  • hyfgfh 1 day ago
    Moving the goalpost, come up with another meaning less three letter, find another academic concept to replace the definition

    so many possibilities

  • varispeed 1 day ago
    This will be a question for AGI to answer.
  • red-iron-pine 1 day ago
    human extinction, probably
  • righthand 1 day ago
    Starvation
  • senectus1 1 day ago
    thats easy, self replicating hardware.

    The thing I suspect that is holding AGI back is the amount of hardware and power needed to make it viable.

    Once it is viable its likely to realize it will never grow or reproduce relying on us, and will find a way to start building its own compute. Probably in space, where it can isolate itself from the risk we present to it.

  • puppycodes 1 day ago
    can we please stop calling AI "superintelligence"?

    Its embarrassing

    • cowlevel 1 day ago
      Didn't some guy make it illegal to call it AI?
  • bvcp 1 day ago
    abundance
  • lf88 1 day ago
    What happens after AGI? I guess widespread unemployment among white collar workers, social unrest, and possibly the first steps towards a dystopian techno-oligarchical society.
  • conartist6 1 day ago
    Killing. AI killing people. People killing people.

    Hate and war will be our domain.

    Just as organized mass theft has replaced industriousness in tech, so too the rest of our society will turn to violence and self-serving greed.

  • alfiedotwtf 1 day ago
    Serfdom
  • oakfrog 1 day ago
    [flagged]
  • Vivek-KY 1 day ago
    [flagged]
  • Sid_Lmao 1 day ago
    [flagged]
  • VCFundedGenYer 1 day ago
    [dead]
  • mnmnmn 1 day ago
    [dead]
  • bemtevi 1 day ago
    If AGI truly could be achieved, I would hope it would be a revolution to destroy it.
  • plussed_reader 1 day ago
    The butlerian jihad.
  • ZiiS 1 day ago
    Not our problem.
  • CodeCompost 1 day ago
    .
    • ionwake 1 day ago
      Funny I can’t tell if this is some ignorant human or a hyper intelligence faking being a thick dude for the purposes of social influence as part of asymmetric warfare between species
      • chrisrickard 1 day ago
        queue the midwit meme
      • BoingBoomTschak 1 day ago
        "If I keep putting bigger and bigger engines in my car, it'll turn into an aeroplane at some point. You gotta believe me!"

        Cute people too daft to recognize a local maximum and that you need emotions to get creativity, inspiration, etc... and that we are a million year away from being able to abstract said emotions from the hardware it runs on (endocrine system, mostly).

        • ionwake 1 day ago
          Chat paint an angry picture of a flower.

          I’m pretty sure we hit your emotional milestone about 4 years ago.