If you have followed the progress in mechanistic interpretability over the past two years, you have likely encountered a certain kind of figure: when the activation vectors of a particular layer in a large language model are projected down to two or three dimensions and plotted, the representations of concepts such as "months," "days of the week," "years," and "colors" do not scatter chaotically through space, but instead arrange themselves along an elegant curve—sometimes even a closed circle or a torus. Researchers such as Chris Olah, Josh Batson at Anthropic, and Engels et al. have all demonstrated this phenomenon in works including Not All Language Model Features Are One-Dimensionally Linear.

These discoveries are captivating, but they leave behind an awkward lacuna: we see the manifold, yet we cannot articulate precisely what relationship holds between the manifold and the "concept" it represents. Why would "year"—something that ought to be a straight line—be twisted inside the model into a curved，meandering curve through high-dimensional space? Is the circular arrangement of "colors" a coincidence, or a necessity? Can the cosine similarity between vectors actually tell us how semantically close two concepts are?

Three mathematicians—Alexander Modell, Patrick Rubin-Delanchy, and Nick Whiteley, from Imperial College London, the University of Edinburgh, and the University of Bristol, respectively—attempt, in their paper The Origins of Representation Manifolds in Large Language Models, to provide, for the first time, a "minimally viable" mathematical theory to answer these questions. This post traces the arc of their argument and offers my own reflections.

The Linear Representation Hypothesis

The field of mechanistic interpretability has long been anchored by a central conviction known as the Linear Representation Hypothesis (LRH): a model encodes human-interpretable "features"—such as "possesses fluffy ears," "mentions the Eiffel Tower," or "is in Arabic"—as a set of nearly orthogonal direction vectors in representation space. The representation of a given input is then a sparse linear combination of these direction vectors, weighted by whether and to what degree each feature is present. The Sparse Autoencoder (SAE), a widely adopted interpretability tool today, is built directly on this hypothesis: one trains an autoencoder with a sparsity penalty to approximate these "dictionary vectors."

Yet mounting evidence suggests that this "black-or-white, one-feature-one-direction" model cannot account for certain phenomena. The authors cite an extensive body of literature: digits in modular addition tasks are encoded as circles; ring-like structures appear in multilingual models; "dates" and "days of the week" exhibit distorted toroidal geometries; fractal geometries even emerge in simulated hidden Markov models. These examples share a common thread: a feature is no longer a single line, but an entire, continuous, potentially nonlinearly curved manifold.

The field has therefore proposed a generalized version of LRH—the Multi-Dimensional Linear Representation Hypothesis:

Here $v_f(x)$ is no longer a fixed direction but can vary continuously with the input $x$ within some subspace $V_f$. The standard LRH is merely the special case where $v_f(x)$ is constant and $V_f$ is one-dimensional.

What this paper sets out to address is precisely the most central and yet most ambiguous part of this generalized hypothesis: in what manifold form does feature $f$ manifest within subspace $V_f$, and what is the relationship between this manifold and the "feature" itself?

Defining "Features" as Metric Spaces

The most elegant move in the paper is to first resolve a question that sounds somewhat philosophical but is in fact critically important: what exactly is "a feature"?

The answer the authors provide is unexpectedly succinct: a feature is a metric space $(Z_f, d_f)$—a set together with a notion of "distance" defined on that set. This definition, though seemingly plain, turns out to be remarkably expressive:

• Atomic features (presence or absence of a cat): $Z_f$ is a singleton set;
• Hierarchical features (a taxonomic tree of animals): $Z_f$ is a discrete set, and $d_f$ is the tree-distance on it;
• Continuous features (color hue, day of the year, calendar year): $Z_f$ can be an interval, a circle, or a higher-dimensional Euclidean space.

This framework is more flexible than the "Euclidean space" or "hypersphere" assumptions common in learning theory, yet substantially simpler—and more tractable—than the Riemannian manifolds with group structure found in the disentanglement literature. It is a characteristically mathematician's choice: strike the right balance between expressive power and tractability.

With the definition of "feature as metric space" in hand, the authors state the first core hypothesis of the paper:

Hypothesis 1 (Continuous Correspondence Hypothesis). There exists a continuous, invertible, one-to-one correspondence between feature values $z_f(x)$ and representation directions $v_f(x)$; that is, there exists a continuous map $\phi_f: Z_f \to S^{D-1}$ (into the unit hypersphere) such that $v_f(x) = \phi_f(z_f(x))$.

Coupled with the technical premise that $Z_f$ is compact, this hypothesis immediately yields a clean corollary (Proposition 1): $\phi_f$ is a homeomorphism. In other words, the representation manifold $M_f$ (the image of $\phi_f$) is topologically "the same shape" as the original feature space $Z_f$: if $Z_f$ is an interval, $M_f$ is a curve; if $Z_f$ is a circle, $M_f$ is a loop; connected components, holes, and branch points—all these topological properties are faithfully preserved.

The significance of this step is that for the first time, it ties "what the manifold looks like" to "the structure of the concept itself" in rigorous mathematical language, rather than stopping at the intuitive description of "it looks like a circle."

Empirical Validation with Real Data

A theory, however elegant, must answer to data. The authors selected three representative case studies:

1. Colors: 3072-dimensional embeddings were generated from English color names using OpenAI's text-embedding-large-3, then reduced to three dimensions via PCA for visualization;
2. Years: the "20th-century year" feature was extracted from layer 7 of GPT-2-small using SAEs, following Engels et al. (2025);
3. Dates: embeddings were generated from prompts such as "January 1st" through "December 31st," again using OpenAI's embedding model.

The results are striking: the color embeddings arrange themselves along a ring, with the hue ordering (red → purple → blue → green → yellow → orange → red) matching the standard color wheel exactly. The token activations for years trace out a curve that winds through three-dimensional space, faintly evocative of the human intuition of a "timeline." The authors further estimated the ordinal structure along the manifold using a k-nearest-neighbor graph and computed rank correlations with the true years, obtaining a Kendall correlation coefficient of 0.97 and a Spearman correlation exceeding 0.99—an almost perfectly monotonic correspondence, providing strong support for the homeomorphism prediction.

The paper also contains a particularly revealing "gotcha" detail: in the original Engels et al. work, the representations of "days of the week" and "months," when projected onto the first two principal components, appeared as tidy circles. The authors point out, however, that once the third principal component is examined, one finds that this "circle" is in fact continuously twisting and weaving in the third dimension; the clean circle compressed into a two-dimensional plane is a visual artifact (see their Figure 2). This observation serves as a cautionary note: low-dimensional projections, while indispensable as visualization tools in interpretability research, can also mislead.

An Explanation in Terms of Computational Expressivity

At this point a natural question arises: since "year" is fundamentally a one-dimensional quantity, why doesn't the model simply encode it as a straight line segment in representation space, rather than twisting it into a curve through high dimensions?

The authors offer an answer that is both intuitive and characteristically mathematical: expressivity. If the goal were merely to "read out" $z$ itself through a linear projection (i.e., to make the identity function $\mathrm{id}(z) = z$ computable via a single linear operation), two orthogonal directions $v_0, v_1$ would suffice. But if one also wishes to make higher-order polynomials of $z$ (e.g., $z^2$) linearly readable, then more orthogonal directions $v_0, \dots, v_{p+1}$ are required, and the path traced by $\phi(z)$ must accordingly bend through a $(p+2)$-dimensional subspace.

In other words: the degree to which a manifold is "twisted" through high-dimensional space encodes, in some sense, how many distinct (nonlinear) functions over this feature can be read out directly by subsequent network layers through a simple linear projection. This provides a functionalist explanation for why the model encodes a simple one-dimensional concept as a complex high-dimensional manifold—not because the model "can't help itself," but because doing so facilitates downstream computation.

The authors then connect this perspective to the superposition hypothesis: since the representation dimension is far smaller than the number of latent features, the model has little choice but to have most features share the same representation space sparsely and near-orthogonally. Under this constraint, the fact that $\phi_f$—the direction encoding for a given feature—does indeed contain the "linearly readable" component of the identity function also goes some way toward explaining why simple linear probes are often surprisingly effective at "fishing out" a specific feature from superposed representations in practice.

Cosine Similarity

If the preceding sections constitute the scaffolding, then this section is the paper's true "hardcore" contribution and, in my view, the part most worth remembering.

The authors advance Hypothesis 2 (Cosine Similarity Reflects Distance): locally, the cosine similarity between representations is some decreasing function of the squared distance between feature values:

The only requirements are that $g_f$ be twice differentiable near 0 and satisfy $g_f'(0) < 0$; beyond these conditions, no further restrictions are imposed.

Under the joint validity of Hypotheses 1 and 2, the authors prove the paper's central result, Theorem 1: Let $\eta$ be a finite-length path in the feature space $Z_f$, and let $\gamma$ be its corresponding path on the manifold $M_f$. Then

In plain language: the length of the shortest path (geodesic) along the manifold is strictly proportional to the length of the corresponding shortest path in the feature space. That is, even without knowing the exact form of $g_f$, so long as the local cosine similarity is some smooth, decreasing function of the squared distance, the "geodesic distance" along the representation manifold—traveling along the manifold itself—recovers the true distance in concept space, exactly, up to a single uniform scaling constant.

This result addresses a hanging conjecture posed by Olah and Batson in 2024: they had written that "the idea that feature manifolds are embedded in more complex ways than their topology alone would require, possibly in order to realize a specific distance metric, may be quite deep and important." This paper, in effect, uses the rigorous language of metric geometry to crystallize that intuitive conjecture into a theorem that can be both proved and falsified.

The proof itself draws on the classical definition of "path length" in metric geometry (the supremum of polygonal-approximation sums), with the central technique being a Taylor expansion of cosine similarity near zero, followed by a series of careful triangle-inequality arguments to bound the error terms—a solid piece of analytic reasoning (see the appendix for details).

Homeomorphism Is Easy; Isometry Is Hard

Theory established, the authors return to the three empirical case studies—colors, years, and dates—to test Hypothesis 2 and Theorem 1. They design two diagnostic approaches:

1. Direct method: plot "cosine similarity vs. squared distance" as a scatter plot to see whether a locally decreasing trend emerges, using Chatterjee's correlation coefficient $\xi$ to quantify the overall degree of functional dependence;
2. Indirect method: test Theorem 1 itself—estimate geodesic distances along the manifold using a k-nearest-neighbor graph, check whether they are proportional to geodesic distances in the feature space, and use the Pearson correlation coefficient to assess linearity.

The results are intriguing. For "colors" and "dates"—both of which carry a natural periodic structure—a simple circular metric (hue angle; day of the year) yields reasonably strong support for isometry (dates achieve a Pearson correlation of 0.97). "Years," however, stumbles: if one assumes that the distance between years is simply $|x-y|$ (e.g., 1990 and 2000 are 10 units apart), the empirical data do not support this hypothesis at all. Figure 4 reveals that years closer to "the present" (the paper takes GPT-2's release year of 2019 as the reference point) are stretched further apart on the manifold—their distances are "magnified."

The authors therefore make a very elegant correction: they replace the metric space of years with Euclidean distance on $\log(2019 - \text{year})$—that is, what the model may be encoding is not "calendar year" per se, but "how long ago" on a logarithmic scale. This new metric space is topologically equivalent to the original year interval (homeomorphism is preserved, with rank correlations still near 1), but when isometry is tested under this new metric, the results flip from "unsupported" to "strongly supported" (Chatterjee coefficient 0.84, Pearson correlation 0.99).

This section is, I think, the most insightful empirical part of the entire paper: it demonstrates with clarity that "homeomorphism" and "isometry" are two entirely different levels of geometric fidelity. The former asks only "is the shape right?"; the latter asks "are the numerical distance relations right?" A manifold can be topologically perfectly consistent with the concept space you envision, yet geometrically (in the specific functional form of the distance) be entirely different. And the process of deciphering "how exactly is distance encoded" is itself an act of reconnaissance into how the model "understands" the concept of time. That GPT-2 encodes "more recent years" as further apart from one another hints, in a certain sense, that years closer to the model's training cutoff may carry denser semantic texture—news and events are more densely packed—and are therefore allotted a larger "representational space budget." This is a rather enchanting conjecture, though the paper itself does not delve deeply into a causal explanation for this pattern.

The Platonic Representation Hypothesis

While reading this paper, I found myself unable to resist drawing comparisons to another position paper, also from 2024, that provoked substantial discussion in the machine learning community—the Platonic Representation Hypothesis (PRH) proposed by Huh, Cheung, Wang, and Isola (ICML 2024). Although the two papers differ in their specific objects of study and methodology almost entirely, reading them side by side reveals that they operate at two different scales on the same overarching question, and that they are in fact complementary—arguably even two successive links in a causal chain.

The core claim of PRH is this: deep networks with different architectures, different modalities (vision, language), and different training objectives tend, as they scale up in size and task diversity, to converge their internal representations toward a shared geometric structure. The authors liken this hypothesized representation space—progressively approximated by more and more models—to the Platonic "ideal reality" (the realm of Forms) that exists independently of and transcends any particular concrete entity: the representation learned by each specific model is merely a noisy,biased projection of this "Platonic representation." Their empirical evidence includes the finding that, when processing paired image-text data, the way image models and language models measure the distances between data points becomes increasingly similar as the models grow larger; and that across an expanding array of vision models with different architectures and training regimes, the pairwise representational similarity systematically rises. They further advance an explanatory conjecture: the driving force behind this convergence is that models, in the course of learning, are compelled to approximate the common, truth statistical structure of the data-generating process ("reality" itself)—and the more diverse the tasks, the stronger this convergence pressure becomes.

If PRH is concerned with "across different models, whether the representation space as a whole is converging toward a single geometry," then the paper by Modell, Rubin-Delanchy, and Whiteley is concerned with a smaller and more specific scale: within a single model, why does the submanifold corresponding to a single feature (color, year, date) exhibit a particular geometric shape, and what is the precise mathematical relationship between this shape and the "concept" it corresponds to?

Taken together, the two papers can be read as tracing a rather coherent logical chain:

1. PRH advances a macroscopic conjecture—that the representation spaces of different models are converging toward a common geometry that reflects the "true statistical structure of the world," and that the degree of this convergence can be quantified by the "consistency in how models measure distances between data points" (the PRH paper uses precisely tools such as kernel alignment and representational similarity, which assess the consistency of pairwise-distance patterns). But PRH itself remains at the level of "existence" and "convergence trends"; it does not delve deeply into a more foundational question: how, within the representation space of a single model, does distance itself correspond to the "true" distance in concept space?
2. The present paper, The Origins of Representation Manifolds, answers precisely this more foundational, more microscopic question. Theorem 1 tells us: so long as there exists a smooth functional relationship between local cosine similarity and feature distance (Hypothesis 2), the geodesic distance along the representation manifold will recover the "true" distance in concept space, exactly, up to a single proportionality constant. This supplies, for PRH's claim that "models learn to measure distances between data points in some consistent way," an explanation at the level of geometric mechanism: the model does not simply "memorize" distances; rather, by mapping features onto a manifold of a particular shape and curvature, it implicitly encodes the distance structure through the geodesic paths on that manifold.
3. Conversely, the circular structures observed for colors and dates in this paper—and the specific finding that "colors are arranged in the hue order of the standard color wheel"—can be seen as a microscopic corroboration of PRH: hue itself is a periodic structure that exists objectively in the physical world (the continuous spectrum of visible-light wavelengths, together with the response curves of the three types of cone cells in the human retina, jointly determining the color space). If different image models and different language models, independently and without coordination, all learn that "hue is a circle," this is precisely an instance—concrete, isolatable, and testable—of the PRH phenomenon that different models converge to a shared representation reflecting the true statistical structure of the world. In other words, PRH provides the macroscopic narrative of "why convergence happens"; this paper provides the microscopic characterization of "what the geometry looks like after convergence, and how this geometry precisely encodes semantic distance."

Of course, there exists an evident tension between the two works, and it is worth acknowledging honestly. The convergence claim of PRH rests, to a large extent, on cross-model comparisons—using tools such as canonical correlation analysis and kernel alignment to compare how different models measure the pairwise-distance patterns of the same set of data points, which belongs to the statistics of "relations between representations." The Origins of Representation Manifolds, by contrast, discusses from start to finish the geometric structure within a single model, within the subspace of a single feature, and barely touches on the question of whether representations from different models are comparable to one another. That is to say, even if the theorem relating cosine similarity to geodesic distance holds for a particular model (e.g., GPT-2), it does not directly imply that another model with a completely different architecture would encode the feature "year" in the same way—with the same metric space, the same logarithmic scale. Indeed, the specific finding that "years are encoded on a logarithmic scale, with distances increasingly stretched for years closer to the reference year" carries, in itself, a considerable degree of model-specificity. That the reference point for GPT-2 is naturally set as its own release year (2019) suggests that this encoding scheme is likely strongly tied to the temporal distribution of this particular model's training corpus, rather than being a universal, "Platonic" distance encoding to which all language models converge. If one were to repeat the same experiment on a model with a different training cutoff and a different corpus distribution, would the logarithmic scale and the reference point shift accordingly? This is, in fact, a question eminently worthy of direct investigation in follow-up work—and in a certain sense, it constitutes a 天然 experimental design for subjecting PRH's convergence claim to an empirical test at the finer granularity of "the specific geometric parameters of a single feature."

In my assessment, reading these two papers together is more illuminating than reading either alone: PRH tells us that "everyone is converging toward the same direction," while this paper supplies the mathematical language for characterizing what exactly that convergence converges to, and how such convergence can be rigorously measured and falsified. If, in the future, someone wishes to genuinely test whether PRH holds at the level of specific features—for instance, whether the circular structure of hue or the logarithmic encoding of time reappear consistently across models with different architectures and training data—then the homeomorphism tests, the isometry tests (Chatterjee coefficient, Pearson correlation of KNN-based geodesic distances) proposed in The Origins of Representation Manifolds constitute an almost ready-made, standardized toolbox. This may well be this paper's most broadly reusable "by-product," beyond its own mathematical results.

Concluding Remarks

Having read the paper, I would like to offer my assessment from several angles.

First, this is a rare effort to rigorize a vague intuition, and its primary value lies in providing language and tools rather than a definitive answer. The field of mechanistic interpretability presently abounds in discoveries that amount to "looking at pictures and describing them"—"oh, this feature looks like a circle," "this one looks like a tree structure"—but lacks a unified mathematical language with which to organize these observations. Defining features as metric spaces, and cleanly separating the notions of "homeomorphism" and "isometry," is itself an important conceptual tool: it forces researchers to translate the vague intuition of "I think this manifold corresponds to that concept" into concrete, falsifiable hypotheses ("is this the right metric space?"), and provides specific statistical testing procedures (Chatterjee coefficient, geodesic distances via k-nearest-neighbor graphs). This kind of rigor is currently in relatively short supply in the field.

Second, Theorem 1 is elegant mathematics, but its "probative force" depends, to some degree, on a rather strong premise—that a smooth functional relationship exists between cosine similarity and distance. This hypothesis itself is not derived from model training dynamics or architectural design considerations, but is instead placed on the table as a "reasonable guess" and then verified empirically. In other words, the paper reads more like "if this hypothesis holds, what beautiful corollaries follow?" than "why the representation space necessarily exhibits this structure" (this is the sense in which the word "Origins" in the title is perhaps a touch wishful—the paper does not actually derive, from optimization objectives or the dynamics of gradient descent, why manifolds "emerge"; it largely operates under the premise that they already exist and proceeds to characterize what they should look like). This is not entirely a criticism, however: mechanistic interpretability as a whole is still at the stage of "observing phenomena and constructing descriptive theories," and very few works genuinely derive representation geometry from first principles. This paper at least executes the "descriptive theory" step with greater rigor than most comparable efforts.

Third, the logarithmic-scale discovery for years is the single most impressive—and most thought-provoking—empirical result in the entire paper. It hints at a methodological trap worth guarding against: the homeomorphism test (is the topology right?) has a very low bar; it is easy to get something that "looks right," but this is far from sufficient to demonstrate that we genuinely understand the model's encoding scheme. Had the authors stopped at "years are arranged along a curve whose order matches the true chronological order, with a rank correlation of 0.97, indicating that the model has learned the ordering of years," this conclusion would be rather a hole—virtually any monotonic mapping could achieve this. What carries genuine information is the non-uniform stretching of distances revealed by the isometry test, and the detailed hypothesis behind it: encoding "time since the reference point" on a logarithmic scale. This reminds us that future interpretability research of a similar kind should not stop at "does the shape look similar?"; rather, it should, as this paper does, press further to ask "is the distance metric correct?"—for only then can we excavate the implicit assumptions that the model has truly encoded.

Fourth, from an applied perspective, this paper carries direct methodological implications for steering research, but remains some distance from a truly operational tool. The paper concludes by noting that, if one could learn $\phi_f$ (the map from features to the manifold), one could in principle perform more refined representation editing that respects the intrinsic geometric structure of the concept—for example, to "shift" a date feature forward by half a year, one should travel along the manifold's geodesic rather than simply adding a vector in Euclidean space. This is a promising direction, and it echoes the authors' call for "manifold-aware SAEs." At present, however, this remains at the conceptual level: how to robustly estimate a high-dimensional, noisy manifold is still an unsolved statistical problem, as the authors themselves frankly acknowledge in the limitations section.

Fifth, a contribution that may be underappreciated is the paper's implications for text embedding services. Many practitioners working on RAG, semantic search, and recommendation systems unthinkingly use cosine similarity as a measure of "semantic proximity," rarely pausing to ask what geometric structure actually lies beneath that numerical similarity score. This paper serves as a reminder: cosine similarity can indeed faithfully reflect distances "along the intrinsic geometric paths of concept space"—but this is a local property, contingent on Hypothesis 2 holding, and the geometric shape differs entirely from one feature to another (circle, line segment, logarithmic line segment). Treating it as a universal, cross-feature "semantic distance" metric may obscure a great deal of important nonlinear structure (as the color example illustrates, where "cosine similarity at large distances manifestly deviates from an isometric relationship"). This is a caution worth lodging in the minds of engineers tuning embedding-based retrieval systems and anomaly detection pipelines.

This paper does not attempt to explain all representational-geometric phenomena, nor does it deliver a tool ready for production deployment. It simply does "a mathematician's job": for a phenomenon that has been widely observed yet lacks precise definition, erect a minimal but rigorous theoretical framework, and honestly take it to the data to test where its boundaries lie. In a field that increasingly relies on "alchemical," empirically driven observation, this kind of work—returning to first principles, willing to state its assumptions clearly, write out its proofs completely, and honestly its limitations openly—is intrinsically worth being seen by more people.

If I had to name the single deepest impression this paper left on me, it would probably be the specific finding that "years are encoded as logarithmic time-distance." Through one minimal example, it shows that behind the seemingly plain geometry of vector spaces, there may truly reside something akin to a human cognitive intuition: recent events are sharper, more finely discriminated; distant events are compressed in memory. This is perhaps the most captivating thing about mechanistic interpretability research: not proving that the model is a black-box piece of magic, but, little by little, translating what it is truly "thinking" on the inside into a language we can read.