[{"data":1,"prerenderedAt":237},["ShallowReactive",2],{"post-\u002Fblog\u002F2026\u002F2026-07-30-what-agi-needs-may-not-be-a-larger-model-but-a-set-of-theories":3},{"id":4,"title":5,"body":6,"cardimage":222,"description":223,"draft":224,"enableComment":225,"extension":226,"image":216,"meta":227,"navigation":225,"onday":228,"path":229,"seo":230,"stem":231,"summary":232,"tags":233,"__hash__":236},"blog\u002Fblog\u002F2026\u002F2026-07-30-what-agi-needs-may-not-be-a-larger-model-but-a-set-of-theories.md","Before Building AGI, We Need to Define It",{"type":7,"value":8,"toc":215},"minimark",[9,18,21,24,32,35,40,47,52,55,62,65,69,72,75,91,98,101,105,112,120,123,130,133,136,150,153,158,169,172,175,178,181,186,189,192,195,200,203,206,209,212],[10,11,12,13,17],"p",{},"Recently, I watched Liang Wenfeng’s talk on AGI, in which he pointed out that the ultimate goal of artificial intelligence is ",[14,15,16],"strong",{},"Artificial General Intelligence (AGI)"," — a form of general intelligence capable of solving a wide variety of problems in a human-like manner.",[10,19,20],{},"Yet I increasingly feel that the current discourse surrounding AGI across the AI field remains at an empirical level.",[10,22,23],{},"We discuss model scale, parameter counts, reinforcement learning, agents, and multimodality, but rarely does anyone address a more fundamental question:",[25,26,27],"blockquote",{},[10,28,29],{},[14,30,31],{},"What exactly is AGI?",[10,33,34],{},"To this day, there is still no widely accepted formal definition. Most discussions describe phenomena rather than capture the essence.",[36,37,39],"h2",{"id":38},"theoretical-framework","Theoretical Framework",[10,41,42,43,46],{},"What has truly changed the world historically is not any single machine, but the theory behind it. To study mechanical computation, Turing proposed the ",[14,44,45],{},"Turing machine"," model. Subsequently, the Church–Turing thesis stated:",[25,48,49],{},[10,50,51],{},"Any function that can be computed by an effective procedure can be computed by a Turing machine.",[10,53,54],{},"Almost all modern computers are engineering realizations of this model: a finite-state controller combined with external storage. More importantly, Turing further proved that the halting problem is undecidable. This meant that, for the first time, people realized:",[25,56,57],{},[10,58,59],{},[14,60,61],{},"Not all problems are computable.",[10,63,64],{},"In other words, the computer acquired, for the first time, a clearly defined theoretical upper bound. The significance of this discovery is no less than that of the second law of thermodynamics in physics.",[36,66,68],{"id":67},"limitations-in-theory","Limitations in Theory",[10,70,71],{},"Today’s AGI is, in essence, at a stage comparable to the period before the invention of the computer.",[10,73,74],{},"Everyone is building ever-larger machines, yet the following questions remain unanswered:",[76,77,78,82,85,88],"ul",{},[79,80,81],"li",{},"What is intelligence?",[79,83,84],{},"Can intelligence be formalized?",[79,86,87],{},"What are the necessary conditions for general intelligence?",[79,89,90],{},"Does intelligence have theoretical limits?",[10,92,93,94,97],{},"If someone were to propose, in the future, a set of ",[14,95,96],{},"AGI axioms"," analogous to the Turing machine’s role in computation theory, its significance might far exceed that of training an even larger model.",[10,99,100],{},"Only after such a theory is established can we truly discuss: what can be achieved, and what can never be achieved.",[36,102,104],{"id":103},"inherent-boundaries-of-computationism","Inherent Boundaries of Computationism",[10,106,107,108,111],{},"Virtually all mainstream AI approaches today are, at their core, built upon ",[14,109,110],{},"computationalism",".",[10,113,114,115,119],{},"That is: intelligence ",[116,117,118],"em",{},"is"," computation.",[10,121,122],{},"If this premise holds, then any AI system ultimately belongs to the class of computable systems. Yet computability theory tells us that a large number of undecidable problems exist (e.g., the halting problem).",[10,124,125,126,129],{},"Hence, a question worth pondering emerges: if AGI is defined as an ",[14,127,128],{},"intelligence capable of solving all problems",", then it must inevitably confront undecidable problems.",[10,131,132],{},"Undecidable problems, according to the Church–Turing framework, are, by their very nature, unsolvable by any computable system.",[10,134,135],{},"Thus, if one adopts this \"omniscient and omnipotent\" definition of AGI, an obvious tension arises between it and computability theory. Of course, this does not mean that AGI is necessarily impossible; rather, it indicates the need for a more precise definition of what \"general intelligence\" actually means.",[10,137,138,145,146,149],{},[139,140,144],"a",{"href":141,"rel":142},"https:\u002F\u002Farxiv.org\u002Fabs\u002Fcs\u002F0004001",[143],"nofollow","AIXI has already revealed this contradiction",". In fact, this issue emerged long ago. Marcus Hutter's ",[14,147,148],{},"AIXI"," is one of the most well-known mathematical models of general intelligence.",[10,151,152],{},"AIXI defines intelligence as:",[25,154,155],{},[10,156,157],{},"The optimal agent that maximizes long-term expected reward across all computable environments.",[10,159,160,161,164,165,168],{},"It possesses a rigorous mathematical definition and is therefore often regarded as the theoretical upper bound of AGI. At the same time, however, AIXI itself is ",[14,162,163],{},"uncomputable",". The reason is that it relies on ",[14,166,167],{},"Solomonoff induction",", which requires summing over all possible programs — an operation that inherently involves uncomputable objects and thus cannot be realized in the real physical world.",[10,170,171],{},"In other words, we already possess a formalized definition of a \"perfect intelligence,\" yet we have proven that it cannot be computed.",[10,173,174],{},"This is not an engineering difficulty, but a theoretical limitation.",[10,176,177],{},"I believe one missing axiom. Currently, many institutions, including DeepMind, are exploring unified frameworks such as meta-learning, adaptive agents, and world models. In a sense, these efforts can all be understood as searches for an \"axiomatic system\" of AGI.",[10,179,180],{},"However, I believe one critical axiom may be missing:",[25,182,183],{},[10,184,185],{},"An intelligent agent must maintain a continuous causal embedding with its environment, rather than functioning as a detached, observer-like solver.",[10,187,188],{},"True intelligence is not a one-shot inference over a fixed dataset, but rather something that continuously forms itself through ongoing interaction with the environment, action, feedback, and revision.",[10,190,191],{},"Without this causal closed loop, even the largest language models resemble pattern-matching systems operating within a static world, rather than developing intelligence in a substantive sense.",[10,193,194],{},"Perhaps what we truly need to seek is not a universal AI. I am increasingly inclined to believe that the question truly worth investigating is not how to build an \"omnipotent\" AI, but rather:",[25,196,197],{},[10,198,199],{},"Does intelligence possess fundamental constraint principles analogous to the second law of thermodynamics or Shannon's limit?",[10,201,202],{},"If, in the future, someone were to prove that no system situated within a finite physical spacetime can realize absolute general intelligence,",[10,204,205],{},"then the importance of such work would perhaps be no less than Turing’s establishment of computation theory, nor less than the foundational frameworks built by Newton and Einstein for physics.",[10,207,208],{},"It would not herald the failure of AI.",[10,210,211],{},"On the contrary, it would reveal that what is truly worth pursuing is not infinite intelligence, but a bounded intelligence that progressively approximates the optimum within theoretical limits.",[10,213,214],{},"Perhaps, looking back from the future, people will remember today not as the year AGI emerged in 2026, but as the year when we began to rethink the foundational question: \"What is intelligence, really?\"",{"title":216,"searchDepth":217,"depth":217,"links":218},"",2,[219,220,221],{"id":38,"depth":217,"text":39},{"id":67,"depth":217,"text":68},{"id":103,"depth":217,"text":104},null,"Recently, I watched Liang Wenfeng’s talk on AGI, in which he pointed out that the ultimate goal of artificial intelligence is Artificial General Intelligence (AGI) — a form of general intelligence capable of solving a wide variety of problems in a human-like manner.",false,true,"md",{},"2026-07-30","\u002Fblog\u002F2026\u002F2026-07-30-what-agi-needs-may-not-be-a-larger-model-but-a-set-of-theories",{"title":5,"description":223},"blog\u002F2026\u002F2026-07-30-what-agi-needs-may-not-be-a-larger-model-but-a-set-of-theories","AGI research remains empirical and lacks a foundational theoretical framework. A central tension exists between computationalism and undecidable problems, as AIXI illustrates. The author proposes that continuous causal embedding with the environment may be a missing axiom, and calls for discovering fundamental constraint principles that bound any physically realizable intelligence.",[234,235],"ai","thoughts","x6rzLdP28NE8Ajsc8f-lTKfnahQIL83TKufifbWb_JI",1785431396110]