Recently, I watched Liang Wenfeng’s talk on AGI, in which he pointed out that the ultimate goal of artificial intelligence is Artificial General Intelligence (AGI) — a form of general intelligence capable of solving a wide variety of problems in a human-like manner.

Yet I increasingly feel that the current discourse surrounding AGI across the AI field remains at an empirical level.

We discuss model scale, parameter counts, reinforcement learning, agents, and multimodality, but rarely does anyone address a more fundamental question:

What exactly is AGI?

To this day, there is still no widely accepted formal definition. Most discussions describe phenomena rather than capture the essence.

Theoretical Framework

What has truly changed the world historically is not any single machine, but the theory behind it. To study mechanical computation, Turing proposed the Turing machine model. Subsequently, the Church–Turing thesis stated:

Any function that can be computed by an effective procedure can be computed by a Turing machine.

Almost all modern computers are engineering realizations of this model: a finite-state controller combined with external storage. More importantly, Turing further proved that the halting problem is undecidable. This meant that, for the first time, people realized:

Not all problems are computable.

In other words, the computer acquired, for the first time, a clearly defined theoretical upper bound. The significance of this discovery is no less than that of the second law of thermodynamics in physics.

Limitations in Theory

Today’s AGI is, in essence, at a stage comparable to the period before the invention of the computer.

Everyone is building ever-larger machines, yet the following questions remain unanswered:

• What is intelligence?
• Can intelligence be formalized?
• What are the necessary conditions for general intelligence?
• Does intelligence have theoretical limits?

If someone were to propose, in the future, a set of AGI axioms analogous to the Turing machine’s role in computation theory, its significance might far exceed that of training an even larger model.

Only after such a theory is established can we truly discuss: what can be achieved, and what can never be achieved.

Inherent Boundaries of Computationism

Virtually all mainstream AI approaches today are, at their core, built upon computationalism.

That is: intelligence is computation.

If this premise holds, then any AI system ultimately belongs to the class of computable systems. Yet computability theory tells us that a large number of undecidable problems exist (e.g., the halting problem).

Hence, a question worth pondering emerges: if AGI is defined as an intelligence capable of solving all problems, then it must inevitably confront undecidable problems.

Undecidable problems, according to the Church–Turing framework, are, by their very nature, unsolvable by any computable system.

Thus, if one adopts this "omniscient and omnipotent" definition of AGI, an obvious tension arises between it and computability theory. Of course, this does not mean that AGI is necessarily impossible; rather, it indicates the need for a more precise definition of what "general intelligence" actually means.

AIXI has already revealed this contradiction. In fact, this issue emerged long ago. Marcus Hutter's AIXI is one of the most well-known mathematical models of general intelligence.

AIXI defines intelligence as:

The optimal agent that maximizes long-term expected reward across all computable environments.

It possesses a rigorous mathematical definition and is therefore often regarded as the theoretical upper bound of AGI. At the same time, however, AIXI itself is uncomputable. The reason is that it relies on Solomonoff induction, which requires summing over all possible programs — an operation that inherently involves uncomputable objects and thus cannot be realized in the real physical world.

In other words, we already possess a formalized definition of a "perfect intelligence," yet we have proven that it cannot be computed.

This is not an engineering difficulty, but a theoretical limitation.

I believe one missing axiom. Currently, many institutions, including DeepMind, are exploring unified frameworks such as meta-learning, adaptive agents, and world models. In a sense, these efforts can all be understood as searches for an "axiomatic system" of AGI.

However, I believe one critical axiom may be missing:

An intelligent agent must maintain a continuous causal embedding with its environment, rather than functioning as a detached, observer-like solver.

True intelligence is not a one-shot inference over a fixed dataset, but rather something that continuously forms itself through ongoing interaction with the environment, action, feedback, and revision.

Without this causal closed loop, even the largest language models resemble pattern-matching systems operating within a static world, rather than developing intelligence in a substantive sense.

Perhaps what we truly need to seek is not a universal AI. I am increasingly inclined to believe that the question truly worth investigating is not how to build an "omnipotent" AI, but rather:

Does intelligence possess fundamental constraint principles analogous to the second law of thermodynamics or Shannon's limit?

If, in the future, someone were to prove that no system situated within a finite physical spacetime can realize absolute general intelligence,

then the importance of such work would perhaps be no less than Turing’s establishment of computation theory, nor less than the foundational frameworks built by Newton and Einstein for physics.

It would not herald the failure of AI.

On the contrary, it would reveal that what is truly worth pursuing is not infinite intelligence, but a bounded intelligence that progressively approximates the optimum within theoretical limits.

Perhaps, looking back from the future, people will remember today not as the year AGI emerged in 2026, but as the year when we began to rethink the foundational question: "What is intelligence, really?"