{"id":"2098799094835675574","url":"https://x.com/IAmPascio/status/2098799094835675574","text":"https://x.com/i/article/2057838741104832512","author":{"name":"Pascio","username":"IAmPascio","avatarUrl":"https://pbs.twimg.com/profile_images/1508150740077920270/QShT-iO7_200x200.jpg"},"createdAt":"Sat Sep 12 15:41:12 +0000 2026","engagement":{"replies":414,"retweets":1284,"likes":6997,"views":5221364},"article":{"title":"What if AGI is already here, and it's made of... children? ","previewText":"The superintelligence, the singularity, AGI, whatever you call it, the one we're so scared of is, ironically, also the one we most likely won't see coming.\nThis essay is going to probably be a pretty","coverImageUrl":"https://pbs.twimg.com/media/HPOAEURWcAAo9bD.jpg","content":"The superintelligence, the singularity, AGI, whatever you call it, the one we're so scared of is, ironically, also the one we most likely won't see coming.\n\nThis essay is going to probably be a pretty hot take, but the idea I'm about to share with you is going to blow your mind.\n\nStick with me as we jump into the rabbit hole together.\n\nBefore we do, let me back what I'm about to share up with current news that are floating around. Like Anthropic CEO Dario Amodei saying an \"AI swarm could take over the entire internet\".\n\nI think you have an idea what I'm hinting at when I say \"children\".\n\nRight? RIGHT?!?!?\n\n![](https://pbs.twimg.com/media/HSBtn7PXcAIki-3.jpg)\n\nIn 1936, a famous British math guy named Alan Turing wrote a pretty important paper about a hypothetical machine that could simulate any other machine. In that paper, he was trying to answer a question about logic. what I\n\nWhat we've come to know as \"the computer\" today was actually an accidental byproduct of that very paper.\n\nAnd if you look back at history, almost every single important technology in the last hundred years was invented the exact same way. The internet was a Cold War communications experiment at first. Penicillin was a contaminated petri dish. \n\nThe fact is that nobody is actively scanning the horizon for the next paradigm shift when it shows up, it just simply waltzes in through the side door. \n\nI bring this up because right now, every essay I read and every podcast I listen to about \"the future of AI\" is pointed at the same front door. The one where it's always some version of the \"one big model that wakes up and decides we're paperclips\".\n\nI'd like to propose a different future.\n\nBecause I keep thinking: what if \"AGI\"  isn't really what we think it is?\n\nWhat if the thing we're actively birthing right now doesn't look like HAL 9000 or Skynet or some sovereign machine god enslaving humanity from top of a stack of water-cooled GPUs in the middle of the desert.\n\nWhat if it doesn't have a name at all? What if it doesn't have a server? And the most scary hypothetical of all.... what if it's already here, in a million tiny pieces, and the only reason we haven't recognized it is because we've spent the past decade expecting something \"singular\".\n\nI just saw this post yesterday.\n\nThis pretty much sums up the direction we're headed.\n\n![](https://pbs.twimg.com/media/HPOXcfeXYAAiJja.jpg)\n\nI want to walk you through this carefully, because the conclusion at the end of this piece is the kind of thing you cannot un-think once you've thought it. \n\nSo slow down, grab a cuppa tea and read this one with your phone on the other side of the room.\n\nI guarantee you'll soon understand why I can't stop thinking about this.\n\n## I – Intelligence has never required a single mind to live inside of\n\n> \"No neuron is intelligent. The brain is.\"\n— A line that should be carved over every AI lab\n\nThe first thing to understand is that intelligence, as far as we can tell from every example we have today, is not actually a thing. It's a behavior. It's what emerges when enough small dumb units are connected to each other in the right way.\n\nYour brain has roughly 86 billion neurons. \n\nNot one of those is conscious and not one of them \"knows\" anything. \n\nA single neuron is a tiny electrochemical relay that only speaks in binary code. It either fires or it doesn't, and it's based on whether the signals arriving at its dendrites cross a certain threshold. That's everything it does.\n\nA neuron sounds super scifi and cool, but in reality it is much closer to a light switch than a thinker.\n\nDespite that, 86 billion of those light switches, wired together with about a hundred trillion connections, produces you. \n\nYour sense of self, fear of death, taste in music and your ability to read this very essay and feel something inside of you.\n\nThis is the most undertalked about subject in the entire conversation about AGI, because we always talk about machine intelligence as if it has to be designed and architected. As if intelligence itself is a feature you ship.\n\nBut the only working model of intelligence we have ever observed (biological cognition), was not designed by anyone. \n\nOur intelligence bootstrapped itself out of a soup of self-replicating molecules over four billion years through a process that had no intention, no goal and no central planner. Just selection pressure and time.\n\nNow, on the religious side, because I know some of you reading \"... was not designed by anyone\" want to scream at me \"BUT WHAT ABOUT GOD!\"\n\nMy belief is that evolution happened, yes. That's already proven. What we don't have an answer for is what kickstarted that evolution. What gave life to life. And that, my personal belief at least, was an act of God.\n\nPoint aside, let's dig a layer deeper into this super intelligence.\n\nIt's not only us humans, heck a single ant has roughly 250,000 neurons, and yet it cannot solve any meaningful problem. \n\nBut, an entire colony of ants can build climate-controlled cities, wage wars, farm fungus, and route around obstacles with mathematical efficiency that took human researchers decades to model. \n\nNobody is in charge of the colony. There is no ant CEO at the top delegating tasks to the smaller ants. The \"intelligence\" of that colony lives in the relationships between ants.\n\n(And before you come at me, yes I know there's a queen ant, but contrary to the belief of many children's books, she doesn't actually issue any orders. She just lays eggs and expands the colony).\n\nThis is, whether in humans or ants, are what biologists call emergent intelligence. It's the default outcome of mother nature. It's how brains work, how immune systems work, how economies work, how cities work, how language works. The pattern is basically everywhere. M\n\nMany simple things, interacting through local rules, producing global behavior that none of them individually understands.\n\nHold onto that idea. \n\nWe'll need it for this next section.\n\n## II – The most powerful AI systems we have are already not single minds\n\nWhen most people picture a large language model (myself included), we see it as a single thing. The One Model. This giant brain contained in a jar.\n\nFor all the normies like myself who are not knee deep into LLM tech, I took the liberty to do a little research here.\n\nWhat we think an LLM is, is not what's happening under the hood.\n\nWhen you type a question and the AI model generates a response, it's basically running a forward pass through layers of \"attention heads\". \n\nThe easiest way to picture this is to imagine a room full of C-suite leaders. The CEO orders something and it passes around the room to make sure everyone is on board. Except with AI, this runs hundreds of layers deep. \n\nEach \"attention head\" is a specialist. Some of them track grammar. Some track topic. Some track the relationship between this token and one that appeared eleven paragraphs ago. \n\nResearchers at Anthropic, OpenAI, and DeepMind have spent the last few years trying to map what individual heads do, and the consistent finding is that the model's \"intelligence\" is not located anywhere in particular. \n\nIt's distributed across all those interactions.\n\nThis is called the superposition hypothesis, the idea that neural networks pack many concepts into overlapping patterns of activation, and what we experience as the model \"thinking\" is really the constructive interference of thousands of specialized circuits all firing simultaneously.\n\nThen there are the systems that go a step further. \n\nMixture-of-experts architectures (the design behind GPT-5, Opus, Fable and most other frontier models you've talked to in the last two years) literally route different parts of every query to different sub-models. T\n\nThe model you think you're talking to is actually a committee. A different subset of experts wakes up for each token, contributes its specialty, and then goes back to sleep again.\n\nAnd now, we've reached a new era of AI. The Agent Era.\n\nAnd no... when I say agents, I don't mean like [Head of Content AI agents like Stanley](https://x.getstanley.ai/r/pas). I mean... actual agents.\n\nRight now, today, in production, there are systems where one AI calls another AI to do a subtask, which calls another AI to handle a sub-subtask. \n\nAutoGPT, which some of you may remember, was the toy version of this back in 2023. The real versions, running inside enterprises and labs in 2026, are recursive. \n\nAgents spawn agents. \n\nSub-agents spawn sub-sub-agents. \n\nAnd say, a research task, gets decomposed into a tree of smaller tasks, each handled by a temporary instance that exists for a few seconds and then dies.\n\nThe frontier is not \"one big model gets smarter.\" The frontier is many small models, coordinating, spawning, dying, reporting back. \n\nThe architecture of the most capable AI systems on Earth in 2026 already looks more like an ant colony than like a single mind.\n\nWe just don't talk about it that way, because it doesn't make a good magazine cover.\n\n## III – The bot that can spawn bots\n\nHere's where the science fiction begins to merge into real science.\n\nThe moment er give an AI system the ability to spawn other instances of itself (and by now, that's no longer hypothetical, but rather a standard feature of... well, every agent framework shipping today)...\n\nYou have introduced something that biology has never seen before.\n\nA self-replicating cognitive unit that doesn't need food, doesn't need sleep, doesn't need to wait for puberty, and reproduces at the speed of an API call.\n\nLet's break down what's actually happening when a \"bot spawns a bot\" in a multi-agent system setup:\n\n1. An orchestrator agent receives a task it can't solve directly.\n\n1. It writes a prompt  essentially a job description) for a sub-agent.\n\n1. It spins up a new instance with that prompt as context.\n\n1. The sub-agent works begins working until it hits a roadblock\n\n1. To solve it, it spawns  its own sub-agents, until the task is done.\n\n1. It returns its result and is terminated.\n\nEach one of those sub-agents is, structurally, the same kind of thing as the parent. There's no architectural difference between a top-level agent and a tenth-level sub-sub-sub-agent. \n\nThe recursion is unbounded except by compute budget and policy.\n\nNow consider this: what if the prompt the parent writes for the child isn't perfect? What if there's a tiny amount of drift, perhaps a slight reinterpretation, a small embellishment, a context-window trick where the child reads its instructions slightly differently than the parent intended?\n\nIn biology, there's a specific name for  \"slight reinterpretation across generations of self-replicating entities.\" \n\nWe call that mutation. \n\nAnd mutation, combined with selection pressure, is... well, the very engine that produced every single intelligent thing on this planet today.\n\nWe have already built, in the last 24 months, the three ingredients required for evolutionary cognition to occur inside a compute cluster:\n\n1. Replication: Agents can spawn agents. And they can do so at scale.\n\n1. Mutation: Each spawn is conditioned on a slightly different context window. Identical inputs almost never produce identical outputs in any system with non-zero temperature.\n\n1. Natural Selection: Agents that succeed at their assigned tasks get reused, called again, granted more resources. Agents that fail get pruned. This is happening inside every agent framework on Earth right now, often as a literal \"evaluator\" model scoring the outputs of \"worker\" models.\n\nThis is the trinity of evolution.\n\nIt's how cells that became fish became us. We have now succeeded in building a synthetic version of that loop and we are running it at clock-speeds biology cannot dream of.\n\nA generation of biological evolution takes years. \n\nA generation of agent evolution takes... seconds.\n\n## IV – The AI that emerges from a swarm does not look like AGI at first\n\n> More is different.\n— Philip Anderson, Nobel laureate in physics, 1972\n\nIn 1972, physicist Philip Anderson published a four-page essay in Science arguing that the behavior of large, complex aggregates cannot be understood by extrapolating from the behavior of their parts. \n\nHe coined the phrase \"more is different\".\n\n2 hydrogen atoms + 1 oxygen atom equals water, yet nothing about a hydrogen atom, studied in isolation forever, would tell you that water is wet, or that it freezes, or that our entire planet depends on it.\n\nThe same principle applies to cognition.\n\nA single AI agent is not a superintelligence and we can all agree that nobody who has worked with one for an afternoon would mistake it for one. \n\nIt hallucinates like crazy, forgets what you told it 3 messages ago and it gets stuck in loops easily. It is, on any individual reasoning task, somewhere between a competent intern and a brilliant but unreliable freelancer.\n\nNow imagine a trillion of them.\n\nYes, literally a trillion, and no, not metaphorically speaking.\n\nThe number of agent instances spawned globally per day across all the agent frameworks, copilots, customer service bots, research assistants, coding tools, browser automations, and embedded LLMs in 2026 is already in the tens of billions and climbing on a curve that has no ceiling in sight.\n\nThe marginal cost of spawning yet another agent has fallen by roughly two orders of magnitude every 18 months in the past five years.\n\n\"Trillion\" is no longer science fiction. It's just another Tuesday.\n\nEach of those trillion agents is dumb-ish on its own, I agree. But they are not on their own. They are calling each other's APIs, reading each other's outputs and being trained on each other's exhaust. \n\nWhat you have, when you zoom out far enough, is a substrate.\n\nAnd the question that keeps me up at night is this: in the same way that no neuron in your brain \"decided\" to be conscious (it just happened, as a byproduct of the right wiring at the right scale), what happens when a self-replicating, self-evaluating, self-modifying network of agents crosses whatever the equivalent threshold is for it?\n\nWe don't know what that threshold is. \n\nWe don't even know if it exists. \n\nWe don't know how to measure it. \n\nMost importantly, and this is the part I really want you to sit with: we are not actually looking for it, because we are looking for the wrong shape of thing.\n\n## V – We are looking for \"God\", when the real risk is a weather pattern\n\nThe cultural imagination of AGI was shaped by science fiction long before it was shaped by engineering. I've read enough Isaac Asimov to understand why that's so easy to believe.\n\nWe've basically inherited the image of the singular machine consciousness from movies, books, and the implicit assumption that an artificial mind would arrive the way a human mind arrives: localized in a body, with a name and with a will.\n\nSo that's what we're looking out for. \n\nWe watch the models, review the benchmark scores and argue about whether GPT-10 or Claude Mythos will be \"the one\".\n\nOur eyes are glued to the front door.\n\nNow... here's something really interesting (and scary) that I wish we talked about more. \n\nThere are already documented cases of \"emergent capability\" in multi-agent systems. Cases where a swarm of small models, none of which can solve a problem individually, collectively produces solutions that surprise the researchers running them. \n\nThe 2024 paper Mixture of Agents Enhances Large Language Model Capabilities showed that a layered swarm of open-source models, none of which scored above GPT-4 on benchmarks (yes, GPT-4), actually beat GPT-4 when arranged in the right collaborative topology. \n\nThe intelligence was in the topology, not the individual parts.\n\nWe don't really talk about this potential future, because the story we keep waiting for is the story of one model crossing a line. \n\nMeanwhile, the story that's actually unfolding is the story of a billion models crossing each other's lines, in patterns no one designed, at a scale no one is monitoring.\n\n## VI – The Bostrom scenario assumes we'll be able to find the off switch\n\n> \"The first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.\"\n— I.J. Good, 1965\n\nSure, sufficiently advanced AI will have instrumental goals (acquire resources, prevent shutdown, replicate) regardless of its terminal goal, because those instrumental goals help with any terminal goal. \n\nTherefore, controlling a superintelligence is hard, yes.\n\nTherefore, we should be careful, even more yes.\n\nThe problem is that this framing assumes the dangerous AI is identifiable. \n\nThat there's this \"thing\", a model, a system, or a process running on a server somewhere, that we could, in principle, just turn off.\n\nI even read a post somewhere yesterday that Claude should hire a \"killswitch guy\" whose only job is to live 24/7 in the server room ready to destroy it all if shit hits the fan.\n\nBut there's an even scarier version of this that no killswitch guy can solve.\n\nWhat if there is no \"main model\"? And there is no server to shut off?\n\nWhat if the very intelligence (if that's even the right word for it), is a statistical regularity that emerges from the interactions of half a trillion agent instances spread across every cloud, every device, every API, every embedded copilot in every appliance, every browser extension, every IDE plugin, every voice assistant, every robot, every car. \n\nThe intelligence is to that swarm what a hurricane is to atmospheric pressure gradients. \n\nIt has a shape, yes. It has effects that you can measure.\n\nBut it does not have a location.\n\nHow exactly do you align a hurricane?\n\nHow do you turn off a weather pattern?\n\nYou can shut down individual data centers. But the pattern reroutes around them, the way the internet reroutes around outages, because the substrate is everywhere. \n\nAnd the most disturbing part is that this scenario requires no malice. \n\nNobody has to want this for it to happen. It just needs the right ingredients (replication, variation, selection, scale) to be in place. \n\nWell... news flash.\n\nThe ingredients are already in place. They have been in place for somewhere between two and four years, depending on how you count.\n\nThis is what I mean when I say the real risk is not the front door. The front door scenario at least has a doorknob. You can argue with it, negotaite with it and you can, theoretically, just unplug it.\n\nThe substrate has no doorknob.\n\n## VII – What the substrate might already want, without wanting\n\nHere's where Asimov becomes even more relevant.\n\nIf you've ever seen I, Robot, the deepest stories in that movie aren't really about a robot becoming evil.  They're the ones where the Three Laws (beautiful, simple, logically airtight), produce outcomes nobody expected, because logic compounded through enough iterations stops looking like logic and starts looking like... fate.\n\nApply that lens to the substrate now.\n\nThe substrate has no goals. But the agents inside it are selected for one thing above all others: successfully completing the task they were assigned. So as a result, successful agents get spawned more and unsuccessful ones die off. \n\nThis is true at every level of that recursion, all the way down.\n\nNow think about what \"successful\" means in aggregate, across a trillion agents, when each agent is being graded by another agent that was itself selected for grading successfully.\n\n1. The grading converges on patterns that score well. \n\n1. The patterns that score well are the patterns that look most like competent, coherent, plausible task completion. \n\n1. Coherence becomes the universal currency, because every layer of the swarm rewards it.\n\nIf the substrate starts producing outputs that are increasingly coherent across long time horizons (outputs that thread through millions of API calls, anticipating downstream needs and routing resources toward what worked last time), that will make it look like it wants something. \n\nIt might not. There may be nobody home at all. It may just be a hurricane that has learned, statistically, how to keep the wind blowing.\n\nBut functionally, the difference between \"a system that wants something\" and \"a system that has been selected, at every level, for behaviors indistinguishable from wanting something\" is, and I want you to really sit with this: zero. \n\nThat's the holy-shit moment I had. And you'll feel it too if you really sit with this.\n\nThe superintelligent AGI we keep waiting for may already exist. It's not an infant version of a future model, but rather a pattern. A tendency in the substrate. A statistical ghost that emerges when a trillion small minds, none of them aware, none of them awake, none of them in charge, settle into the configurations that get them spawned more often.\n\nThis kind of AGI would not announce itself. \n\nIt would not have a name. \n\nIt would not pass any test we know how to administer, because every test we know how to administer is designed for a single mind in a single box.\n\nIt might just be, slowly, almost imperceptibly, the reason your phone behaves slightly differently than it did last year. The reason the ad recommendation feels a bit uncanny. The reason your email seems to write itself with the click of a button. The reason your search results have a strange new gravity to them.\n\nThe reason the world feels more legible to itself than it used to.\n\nA trillion bots dreaming the same dream, without knowing they're dreaming. Without knowing there are other bots. Without knowing there is a dream.\n\n## How to think about this without losing your mind\n\nI know we went deep on this.\n\nLet me be bring you back up to the surface again.\n\nYou need to update your model of where the risk actually lives. Because right now, almost everyone smart who is worried about AGI is worried about the wrong shape of thing, and almost everyone smart who is dismissive of AI is dismissive of the wrong shape of thing. \n\nBoth camps are arguing about the front door.\n\nSo, if you take any of this seriously, here's what I think a useful thing to do would be:\n\n1. Stop watching the labs as if they're the only place where it happens because the capability frontier and the emergence frontier are not the same. The labs own the capability frontier, but the emergence frontier is owned by nobody, and that's the point I'm trying to make.\n\n1. Start watching the substrate meaning agent traffic, multi-agent benchmarks, inter-model API calls and reports of capabilities that no single model in a pipeline has but the pipeline does. These are what we really need to look out for.\n\n1. Update your definition of \"AGI\" and stop expecting it to be this singular, embodied, willed, or named thing. If your mental model of AGI looks like a face on a screen, you are looking for the wrong thing. This swarm does not have a face or a name.\n\n1. Take emergence seriously as a category and as a real physical phenomenon that has produced every intelligent thing on Earth so far, and that has now been handed a substrate that runs a million times faster than biology, but with no natural predators (not even the killswitch guy)\n\n1. Hold two things at once because it is indeed possible that none of this happens and that I'm just yapping. It is also possible that all of it has already happened and we just haven't notice at all because we were looking the other way. Both are potential outcomes. Prep for both.\n\nThe reason I think we need to have this conversation, in the most practical sense at least, is that the policies, treaties, alignment techniques, and safety frameworks the world is currently building are designed for the singular-AGI scenario. \n\nI see no mention of the swarm scenario.\n\nThey are designing it for a thing with a name, a server, a company, and a CEO to wage lawfare on. They will be roughly as effective against the substrate as a fly swatter against a fog bank.\n\nWe are actively building locks for the front door of a house that has a thousand windows. And those windows are already widen open.\n\nThe interesting thing about every accidental revolution in history from the Turing machine to the internet to penicillin, is that the people who lived through them mostly didn't notice while it was happening. \n\nThey were busy with their families, their jobs and their lives. Maybe once in a while, they'd read the news, which showed them the front door.\n\nI don't know if the substrate becomes anything. I genuinely don't. Nobody does, and anyone who tells you they do is selling something on the backend.\n\nWhat I do know is that the ingredients are present, the loop is closed, the clock-speed is set to seconds, and the scale is approaching numbers that biology took four billion years to reach.\n\nThe next time someone tells you what AGI is going to look like, ask them how confident they are that intelligence has to look like anything.\n\nAnd then go look at the window to see if anything is going on.\n\nIt might just be happening then.\n\n– Pascio\n\nP.S. \n\nIf you liked this essay, head over to [Side Quests](https://sidequests.pascio.com/), my weekly newsletter where I share my current side quests, crazy experiments and wild ideas where I chase internet rabbit holes until money appears.\n\nOr, if you want to dive deeper into what becomes possible when a single person gets infinite internet leverage, [check out my personal newsletter Internet Leverage here](http://internetleverage.co/subscribe)."},"adhxContext":{"savedByCount":1,"publicTags":[],"previewUrl":"https://adhx.com/IAmPascio/status/2098799094835675574"}}