[{"data":1,"prerenderedAt":5439},["ShallowReactive",2],{"post-\u002Fblog\u002F2026\u002F2026-08-13-xie-zuo-yu-yan-de-zhuan-bian":3},{"post":4,"nextPost":212,"prevPost":485},{"id":5,"title":6,"body":7,"description":13,"draft":198,"enableComment":199,"extension":200,"image":195,"important":198,"location":201,"meta":202,"navigation":199,"ogImage":203,"onday":204,"path":205,"seo":206,"stem":207,"summary":208,"tags":209,"__hash__":211},"blog\u002Fblog\u002F2026\u002F2026-08-13-xie-zuo-yu-yan-de-zhuan-bian.md","写作语言的转变",{"type":8,"value":9,"toc":194},"minimark",[10,14,17,20,176,179,182,185,188,191],[11,12,13],"p",{},"长期以来，我对自己的博客定位从来不是分享，而更多是给自己看的「笔记」。很多情况是：记录一个刚刚搞懂的东西 → 写下来 → 以后自己查 → 写完就结束。我的博客基本都很短，基本三分钟以内就能读完。大约在 2023 年开始，同时为了学习英语，锻炼技术写作能力，我在疯狂背单词的同时，也可以保持用英语进行技术写作。现在坚持了快两年。所以长期以来，利用英语写作，对我来说压力并不大。写作时不需要追求完整论证，只需要把自己的思路编码下来。哪怕句子稍微生硬一点，只要自己以后能看懂，就已经达成目的。",[11,15,16],{},"过去两年的英语写作之所以轻松，是因为它服务于「笔记」功能——短、线性、结论明确。这种写作不需要复杂的逻辑嵌套，不需要反复回看和修改结构，英语的线性特征反而成了一种约束，帮你把想法压缩成清晰的短句。",[11,18,19],{},"但是后来，情况发生了变化，我的博客从「给自己看的笔记」的定位，开始转移到了「表达与推演」。当写作从记录变成表达与推演，语言就从工具变成了负担。写作负荷不是恒定的，它会随着结构复杂度和语言熟练度的乘积而非线性增长。",[11,21,22,23,175],{},"文章越来越长，关于技术写作的部分，有的文章我需要插入大量的 ",[24,25,29,52],"span",{"className":26,"translate":28},[27],"katex","no",[24,30,33],{"className":31},[32],"katex-mathml",[34,35,37],"math",{"xmlns":36},"http:\u002F\u002Fwww.w3.org\u002F1998\u002FMath\u002FMathML",[38,39,40,47],"semantics",{},[41,42,43],"mrow",{},[44,45,46],"mtext",{},"LaTeX",[48,49,51],"annotation",{"encoding":50},"application\u002Fx-tex","\\LaTeX",[24,53,57],{"className":54,"ariaHidden":56},[55],"katex-html","true",[24,58,61,66],{"className":59},[60],"base",[24,62],{"className":63,"style":65},[64],"strut","height:0.8988em;vertical-align:-0.2155em;",[24,67,71,76,81,113,117],{"className":68},[69,70],"mord","text",[24,72,75],{"className":73},[69,74],"textrm","L",[24,77],{"className":78,"style":80},[79],"mspace","margin-right:-0.36em;",[24,82,85],{"className":83},[84],"vlist-t",[24,86,89],{"className":87},[88],"vlist-r",[24,90,94],{"className":91,"style":93},[92],"vlist","height:0.6833em;",[24,95,97,102],{"style":96},"top:-2.905em;",[24,98],{"className":99,"style":101},[100],"pstrut","height:2.7em;",[24,103,105],{"className":104},[69],[24,106,112],{"className":107},[69,74,108,109,110,111],"mtight","sizing","reset-size6","size3","A",[24,114],{"className":115,"style":116},[79],"margin-right:-0.15em;",[24,118,120,124,128,167,171],{"className":119},[69,70],[24,121,123],{"className":122},[69,74],"T",[24,125],{"className":126,"style":127},[79],"margin-right:-0.1667em;",[24,129,132,158],{"className":130},[84,131],"vlist-t2",[24,133,135,153],{"className":134},[88],[24,136,139],{"className":137,"style":138},[92],"height:0.4678em;",[24,140,142,146],{"style":141},"top:-2.7845em;",[24,143],{"className":144,"style":145},[100],"height:3em;",[24,147,149],{"className":148},[69],[24,150,152],{"className":151},[69,74],"E",[24,154,157],{"className":155},[156],"vlist-s","​",[24,159,161],{"className":160},[88],[24,162,165],{"className":163,"style":164},[92],"height:0.2155em;",[24,166],{},[24,168],{"className":169,"style":170},[79],"margin-right:-0.125em;",[24,172,174],{"className":173},[69,74],"X"," 公式、推导和演绎过程。此时再使用非母语进行技术写作时心智负担显著增大。短文里，一句话如果不知道怎么写，一般情况可以用简单句子绕一下，问题不大。但是到了长篇技术文章，可能连续几页都在描述一个复杂的思想。这时候需要同时控制数学符号、技术概念、论证结构、前后术语一致性、句法，还有段落之间的衔接。于是工作记忆里面同时跑着很多东西。",[11,177,178],{},"而且，长篇博客并不是一次写完的，很可能需要花上几个星期慢慢打磨，或者经常回顾我自己的博客的思想。写作的时候，我大多数时间都在思考：「这个理论到底应该怎么解释？」。但是，结果脑子里却不断出现：「这里应该用 which 还是 that？」、「这个东西应该叫 representation 还是 formulation？」、「这个词性是否合适，有没有对应的名词形式？」之类细枝末节的语法问题。久而久之，写作起来会非常累。在短文本写作里，这种差异几乎感觉不到。但当文本从几百字增长到几千、几万字以后，语言的视觉结构、信息密度、词法形态、定位效率和工作记忆负担都会开始成为写作系统的一部分。",[11,180,181],{},"另外一种原因，英文是线形文字，阅读效率天然就很低：想在长文中定位到某一块位置，必须要从每一段从头开始逐行扫过，无法像汉字那样逐块扫描。很多时候我在长文写作中，需要不断地往回头看。而英语的语法线性强，从句嵌套多了以后，读者和作者都容易迷失。因为汉字同时包含视觉图像和声音两种信息，它信息密度和视觉辨识特征，使中文文本非常适合视觉扫描。而英文单词之间存在大量空格，真正有语义重量的东西被拆成了很多视觉单元。所以当文章达到几千甚至上万字以后，回来看自己几周前写的东西，会出现一种很奇怪的体验：中文是在「看结构」，英文更容易变成「读句子」。所以，就导致语义单元和视觉单元不对齐——一个概念可能要三四个单词才能表达，视觉上要扫过更长的距离才能抓住一个完整意群。",[11,183,184],{},"其实不止是我，很多技术写作者都会遇到一个问题：语言不是中性的容器，它会反过来塑造你思考的形状。",[11,186,187],{},"笔记型写作的特点，思维已经完成。只是在编码一个已经清晰的结论，供未来的自己检索。表达与推演型写作则不同，写作本身就是思维过程。你在写的过程中推演、发现、修正。文字不是思维的镜像，而是思维的工具。当写作成为思考工具时，语言就不再只是输出端的问题，而是输入端的问题。非常需要语言来帮助你组织尚未成形的想法，来试探逻辑的边界，来连接不同的概念。",[11,189,190],{},"从此以后，我的博客功能，从以前的博客 externalized memory（外部记忆），到现在逐渐变成了 externalized thinking（外部化思考）。",[11,192,193],{},"也许从英语写作到中文写作，是在为思维本身让路。让认知资源从语言操作中解放出来，还给真正的思考。",{"title":195,"searchDepth":196,"depth":196,"links":197},"",2,[],false,true,"md","河南郑州",{},null,"2026-08-13","\u002Fblog\u002F2026\u002F2026-08-13-xie-zuo-yu-yan-de-zhuan-bian",{"title":6,"description":13},"blog\u002F2026\u002F2026-08-13-xie-zuo-yu-yan-de-zhuan-bian","长期以来，为了学习英语并增强熟练度，在技术写作时，我都会刻意使用英语来写作。但是后来我发现使用英语的阅读和思考心智负担非常大。从笔记到表达与推演，写作目标发生变化后，认知资源也需要实现对应的重新分配。",[210],"thoughts","GgbL6w_I6tbQihpjDly03_3pFg9J3kM5x2weDCk5vHw",{"id":213,"title":214,"body":215,"description":219,"draft":198,"enableComment":199,"extension":200,"image":195,"important":198,"location":201,"meta":477,"navigation":199,"ogImage":203,"onday":478,"path":479,"seo":480,"stem":481,"summary":482,"tags":483,"__hash__":484},"blog\u002Fblog\u002F2026\u002F2026-08-14-demystification-research.md","祛魅科研，每个研究生的必修课",{"type":8,"value":216,"toc":469},[217,220,223,227,230,233,236,247,250,253,256,259,262,265,268,271,274,277,281,284,287,290,297,305,308,311,314,317,320,326,329,332,335,338,343,346,349,352,355,361,364,367,374,377,380,383,386,389,392,398,401,404,407,410,413,416,419,422,425,432,436,443,448,451,454,457,460,463,466],[11,218,219],{},"从 2025 年开始入学读研，到现在已经有一个年头了，不知不觉已经研二了。在过去的一年的研究生学习中，我发现我的心态、世界观经历了一次完整的、打击性的重塑。如果说之前我的心态是中规中矩，那么现在则是满心失落。自从经历过完整的科研训练，调研课题、找 idea，做实验、处理数据、撰写论文、修改文章、等投稿、写 Rebuttal、拒稿再转投、论文改版等一系列流程完整经历过后，只剩下满心苦涩。一度怀疑，我们搞的这个所谓的「科研」的价值和意义。😇",[11,221,222],{},"因为笔者读的研究生是计算机工程类专业，对于其他理工科领域、文科领域，笔者不是很熟悉，所以以下观点体验仅适用于该专业。",[224,225,226],"h2",{"id":226},"草台班子",[11,228,229],{},"第一个对我的改变就是对学术成果的怀疑。学术圈的科研，其实并没有那么高端，可以说是草台班子。",[11,231,232],{},"在读研之前，我一直认为学术论文其实是非常高端的东西，至少是可信度很高的参考。但是读研之后，这个想法被改变了：绝大多数论文，他们的观点、数据、实验，看看就好，不必认真。",[11,234,235],{},"一篇论文的结论，往往只有作者自己的机器上成立。审稿人不会复现，编辑不会复现，引用者也不会复现。大家都默认了这个不可验证的游戏潜规则。论文并不是经过验证的知识，而是一个经过美化的故事。",[11,237,238,239,246],{},"有人专门对被 ICML 收录的 Oral 论文进行了复现，并给出了",[240,241,245],"a",{"href":242,"rel":243},"https:\u002F\u002Fx.com\u002FChenhaoTan\u002Fstatus\u002F2079969545629118737",[244],"nofollow","论文的报告信息","。结果发现，在 105 篇完整的复现论文中，只有 27 篇复现了超过 40% 的声称效果，剩余的几乎无法复现，要么效果不达标。原因包括代码无法运行、文件缺失以及模型已弃用等等。",[11,248,249],{},"复现困难确实是机器学习领域长期存在的问题。因为，CUDA 版本、显卡型号、CPU 型号、PyTorch 框架版本，可能都有大量隐蔽的细节差异，如果再组合起来，那么效果就完全无法验证。实验中的很多关键 Hack（比如特定的随机种子、数据预处理的微妙顺序、超参数的精细调节）可能只存在于作者的实验笔记或潜意识里。这些隐性经验很难通过文字传递，导致复现者像在黑暗中摸索。",[11,251,252],{},"甚至如果作者的代码里有 Bug，也很可能会造成完全错误的结论。我曾经跑过前人的基线实验，发现一篇论文，他的某个指标又非常虚高。最后才发现，作者对数据集根本没有清洗干净，也就是说，这份成绩是假的。",[11,254,255],{},"但问题是，这个论文已经被大量后人引用且作为基线了，后人想要再投稿，那么他的成绩必须要比这个假成绩更高，但是你根本比不过这假成绩，怎么办呢？你只能偷偷摸摸用点魔法手段，万不可认真，否则反而会显得像是你做错了，审稿人不接受。后来的研究者反而成了受害者。这不就是难为后人么？",[11,257,258],{},"所以，后人也只能不得不逼着也参与这种数字游戏，偷偷使用手段。结果就是，学术论文逐渐沦为一场“数字通胀”。每一篇新论文都在前人的数字泡沫上再吹一层泡沫，直到某个数据集上的准确率达到 99.9%，然后这个方向就“死掉”了——因为没人能再超越了，而大家都知道那个 99.9% 是假的，但谁都不想捅破。",[11,260,261],{},"这已经超出了学术不端的范畴，演变成了一种系统性的「囚徒困境」，很遗憾，这种困境，在学术圈里几乎无解。",[11,263,264],{},"无法复现不一定是作者故意有造假意向，而是因为深度学习本身就是一个巨大的黑盒，可验证性、可解释性非常差。很多错误，作者不一定能及时发现，审稿人也不一定能发现，于是，“只要能 work 就能发文”。当审稿人面对一篇充满“惊艳”实验数据的论文时，他既没有算力去复现，也没有理论工具去证伪其内部的混沌过程。于是，审稿的标准退化为：“只要故事逻辑自洽，且结果看起来比 SOTA（当前最优）高，我就信。”这就给出了巨大的浑水摸鱼的空间。",[11,266,267],{},"这种无法复现的现象，也会给自己带来很多的麻烦：如果你的文章需要依赖前人的工作，但是前人的工作本身就有问题，那么你的工作就几乎不可能顺利下去。你只能花大把时间来反思、调试，最后还很可能一无所获。",[11,269,270],{},"最后被浪费大把的时间和精力。这种消耗是精神凌迟。后续工作往往不得不替它买单。",[11,272,273],{},"有时候，我也甚至怀疑作者是故意在开源里缺斤少两，删减关键代码和文件或者埋入 Bug，或者论文被 Accept 后立即下架数据集，极力避免复现。😇要知道，审稿人通常只有 2-3 位，他们在几周内免费审稿，不可能复现你的实验，也无法验证你的原始数据。他们主要检查的是逻辑是否顺畅和方法是否看起来合理，而不是结论是否绝对正确。",[11,275,276],{},"学术论文它的首要目标是发表，而不是传世。为了发表，作者必须讲一个完整且自洽的故事。他们会突出最漂亮的数据，而弱化不支持的“噪音”结果。所以，他们会在讨论部分夹带私货，把故事讲得更有吸引力。它也确实是痛苦的。因为这意味着你失去了一个可以无条件信任的知识权威。",[224,278,280],{"id":279},"ai-审稿危机","AI 审稿危机",[11,282,283],{},"读研起到现在，我已经投稿了三篇文章，这点我是有亲身体会的。",[11,285,286],{},"另一种尴尬是 AI 时代的审稿危机。AI 时代的恶果就是，论文灌水越来越容易。往常，写一篇论文很可能需要一学期、大半年的时间，要辛辛苦苦做实验、分析结果、画图表、斟酌写作。但是现在不一样了，利用自动化学术 Agent 如 AutoResearch，只要输入合适的提示词和模糊的想法再交给 Agent，它两天时间就可以编出一篇像模像样的论文，自动编写程序做实验，自动画图表，一个人一个月就能编写出 10 篇论文，直接投到顶会轰炸。",[11,288,289],{},"AAAI 在 2026 年收到了五万多份取号，几乎是 2025 年的两倍。而前年 2024 年才不过九千多份，短短两年时间就增长了五倍！其实不止 AAAI，其他的计算机会议的投稿量也是几乎翻倍增长。假设每篇论文标准配备 3 名审稿人，组委会将需要管理 120,000 份独立审稿意见：",[11,291,292],{},[293,294],"img",{"alt":295,"src":296},"计算机顶会近五年投稿量。自从 2025 年后，大多数投稿量都是翻倍式增长","https:\u002F\u002Fimage-assets.dreams.plus\u002F202608111219721.jpg",[11,298,299,300,304],{},"这就会导致一个后果：劣币驱逐良币。审稿体系正在加速崩溃。就算你辛辛苦苦完成了心血，设计实验，写好文章，",[301,302,303],"strong",{},"审稿人也几乎不会认真看你的文章，而是直接交给 LLM 敷衍。"," 因为审稿人压根无法区分哪篇论文是 AI 写得，哪篇不是。更糟糕的是，学术领域是高度分化的，审稿人极有可能也不熟悉、也不理解你的研究领域。所以，他也只能同样一股脑交给 AI 审稿。",[11,306,307],{},"在这种情况下，灌水零成本，中间大量灌水的作者和敷衍的审稿人占多数。认真的审稿人和认真的作者只能被动深受其害。受害的永远只是认真搞学术的你。在这种情况下，审稿工作还能正常进行下去吗？",[11,309,310],{},"而 LLM 审稿本身就有很大的问题，AI 并不会理解你的文章，LLM 的幻觉问题众所周知。它会用非常刁钻的角度在你的文章里挑刺，哪怕是没有问题也会制造出一些问题。这是因为 LLM 在训练、SFT 偏好微调的时候，天生就倾向于给你打低分。LLM 并不真正拥有论文作者的研究上下文。它尤其容易犯一种很危险的错误：把“我没理解”转换成“作者的方法存在问题”。",[11,312,313],{},"LLM 模型学习到的统计规律是：“审稿”这个动作的语义空间，几乎完全由“批评性词汇”构成。因此，即使一篇论文毫无瑕疵，模型根据概率分布生成的“审稿风格”文本，天然就带有负面倾向。SFT 微调的时候，宽松的论文意见并不受欢迎，为了要尽可能压榨 LLM 能力，在微调的时候会特意设计出非常严苛的训练案例。这种奖惩机制直接塑造了模型的“人格”：它必须“生产批评”来证明自己的价值。哪怕没有真实缺陷，它也会启动“防御性挑刺”模式，利用模式匹配强行构造出看似合理的问题。",[11,315,316],{},"我的文章被拒稿过一次，三个 Reviewer 里两人给出的意见，明显就是 AI 复制粘贴的。😅 但是你不能抗议，你还要捏着鼻子忍着恶心，在 rebuttal 里面假模假样地感谢审稿人，再假模假样地写 rebuttal，跪求他赏赐一口饭吃。😅",[11,318,319],{},"审稿体系存在一个非常明显的权力不对称。Reviewer 可以随意评价“The novelty is insufficient.”而作者很难说：“你根本没读懂我的论文。”因为作者没有证据。另外，Reviewer 可以洋洋洒洒写出几千字的审稿意见，没有字数限制，而作者给的 Rebuttal 却严格限制在几百字以内。这也是最恶心的一点😅",[11,321,322],{},[293,323],{"alt":324,"src":325},"如图，是几个 LLM 在审稿 ArXiv 计算机学科论文时给出的平均得分（10 分制）","https:\u002F\u002Fimage-assets.dreams.plus\u002F202608111210591.png",[11,327,328],{},"试过把 21-23 年，GPT 出来以前的三大顶会里，已经发表收录的论文随机爬下来，投给 AI 审，50 篇里 30 多个 weak reject，10 多个 weak accept，剩下的全部都是 reject。一篇 accept 都没有。后 AI 时代所谓的顶会论文已经变成笑话哩。",[11,330,331],{},"审稿人因为反驳文太长而疲惫，作者因为审稿人固执而绝望。当作者知道审稿人是 AI，审稿人知道作者用了 AI 时，作者 - 审稿人 - 编辑三方之间的这场学术对话，就彻底沦为了一场荒诞剧。这就造成了非常滑稽的景象，AI 写 AI 审：",[11,333,334],{},"用 AI 写论文、写代码，再用 AI 初审，根据 AI 的意见修改，完成初稿。审稿人拿到初稿，再交给 AI 审稿，用 AI 的意见给出 review 意见，作者拿到 AI 写的意见后再交给 AI 写 rebuttal。审稿人再用 AI 根据 rebuttal 做出决定。堪称学术出版领域正在逼近的赛博朋克式奇观。",[11,336,337],{},"在三方中，编辑\u002F主席的地位最为尴尬。当所有文字意见如审稿、反驳都由 AI 生成时，编辑失去了判断学术创新性的任何抓手。他唯一能做的，就是检查流程是否走完：AI 是否提了 3 个问题？作者是否逐条回复？回复长度是否达标？只要格式合规，就可以按下接受键。学术判断的权力，在此刻已完全让渡给了硅基算法。",[11,339,340],{},[301,341,342],{},"所以，能否发顶会、顶刊，实际上已经越来越像摸彩票中奖的运气、概率问题，跟你的文章质量、工作效果已经几乎没有什么关系了。",[224,344,345],{"id":345},"手艺人",[11,347,348],{},"硕博生这个身份曾经是不少人引以为傲的根本。但是真正体验过他们的生活，也会发现他们本质上和流水线的螺丝工人相差无几，这种生活状态其实是非常压抑的，如果过这种日子，那只能用「熬」来形容。",[11,350,351],{},"硕博生本质上也是出卖高强度脑力劳动换取生存资格的劳动者。跟工地上抗水泥的农民工、顶着大太阳湿透衣服的清洁工没有本质区别。",[11,353,354],{},"对于理工科研究生，996 是常态，实验室的灯永远亮着。每天睁眼闭眼就是要面对屏幕上密密麻麻的实验数据、代码和仪器。高强度脑力劳动后，带给身心的除了疲劳还是疲劳。日子久了，精气神会被消磨掉，慢慢丧失对这个世界的好奇心和一切欲望，不想谈恋爱，不想出去旅游，哪怕是手里的游戏，日子久了玩起来也没意思。",[356,357,358],"blockquote",{},[11,359,360],{},"到周末后，只想在床上躺着，啥也不干。脑一旦被单一的高强度任务长期占据，负责发散思维、感受情绪、产生欲望的脑区就会被持续抑制。只像一个漂浮在数据海洋里的意识，拖着一具沉重而麻木的肉体，犹如冢中枯骨而已。",[11,362,363],{},"这是最致命的。当你看清你所做的研究可能只是学术游戏里的一个废棋，对外部世界毫无影响时，熬就变成了一种精神上的凌迟。不禁会问，「我做的这一切，受了那么多折磨，到底有什么意义？」",[11,365,366],{},"工人进厂时，起码能自知之明、清醒地知道这是出卖身体，用劳动换生存。但硕博生被社会、被家人、被曾经的自己赋予了天子骄子、知识精英的光环。当现实变成日复一日地跑数据、伺候仪器时，日子久了你会有这样一种感觉：自己并不是一个人，而是一个庞大机器中的一颗零件。",[11,368,369,370,373],{},"硕士生（Master）也不是大师。博士生也不博学。他的知识广度，甚至还可能不如一个高中生。",[301,371,372],{},"读研后，你的视角只会被限制在高度狭窄且专业的小领域里。"," 在自己的学术孤岛里自娱自乐。",[11,375,376],{},"中世纪的经院哲学家热衷于争论「一个针尖上能站几个天使」，而今天的学术圈，大量精力被消耗在维护主流范式上。正如前文所提到的，学术论文本身也是高度固定化的、范式的。有时候你不会感觉自己「在创造一个想法」，而是「完成一个八股文」。",[11,378,379],{},"如果要比喻硕博生这个群体的身份，它更像是一个高度程序化的手艺人、螺丝工。搞科研的流程本身就是高度固定化、流水线化的。调研、idea、实验、写作、改稿、Rebuttal。这一圈下来，恭喜你，你已经是一名合格的劳工！",[224,381,382],{"id":382},"学术圈",[11,384,385],{},"曾经对学术圈的浪漫想象，至少代表了知识的前沿、先进的生产力和思想文化，具有进步性。我其实对科研领域内的学者，高校里的教授、教师等群体，长期是存在敬仰的，认为他们或多或少都是代表了人类开拓认知知识、征服星辰大海的一批人，在心里也会敬三分。",[11,387,388],{},"现在才知道，学术圈其实是高度封闭的。因为知识本身就有很强的入门壁垒，当人类认知突破到一定边界时，工具、术语和范式的复杂度必然形成门槛，学术方向往往会走向高度分化、隔行如隔山的细碎分支，各个分支又会高度壁垒。这就造成了学术圈的高度封闭性。",[11,390,391],{},"其实，越是高度封闭的圈子，越容易产生高度固化的权力结构，行事作风越是封建化、越是讲政治。😅其实现在的学术圈其实跟欧洲中世纪的经学院教派之争、西藏喇嘛们的辩经并没有什么区别。那些专家，领域学者，头衔看得是挺唬人，但做的无非在极度封闭的圈子里，用只有内部人能懂的黑话，争论着对外部世界影响甚微的问题，而决定胜负的常常不是真理，而是资历、人脉和对经典的诠释权。",[11,393,394,397],{},[301,395,396],{},"学术圈，其实比大多数人想象的还要小。"," 因为现代学科已经进入高度分化、高度专业化的时代了。如果细分下去，全中国十四亿人里，同一个领域的研究同行很可能不超过百人，甚至十几人。在一个村落里，任何小动作，全落在这几十个低头不见抬头见的人手里。这里没法对事不对人，因为在结构上，所有的事，最终都是人的事。",[11,399,400],{},"然而，现在的学术圈是零和博弈，资源是极其有限的。这个在申请基金、文章版面、学术交流等等活动中，只要有人胜出，必然会有人落选。这也就意味着，你的小圈子里，可能到处都在「树敌」。这并不意味着本人有错，而是你的存在，本身就是威胁。",[11,402,403],{},"所以遇到同行暗中使绊子，也是常见的事。你的审稿人很可能跟你的导师有竞争或者过节，就直接轻松 Reject 你的心血。即使现在的审稿制度大多是双盲制，在一个领域只有几十上百人的圈子里，根本不存在真正的双向匿名。看研究问题、看方法、看引用的文献，审稿人闭着眼都能猜到这篇稿子出自哪个课题组。",[11,405,406],{},"对一个埋头苦干的学生来说，这是最深的打击。你相信公正，相信学术质量至上。然后，一堵由学派、人情、资源争夺构成的墙，悄无声息地挡在你面前，将你的心血轻松驳回。你甚至没有一个明确的敌人去质问抗争，你甚至不知道你的敌人是谁，又得罪了谁，只剩下无尽的无力感和被戏弄的愤怒。这种被暗算的体验会深刻腐蚀对学术共同体的信任。",[11,408,409],{},"像中世纪的领主分封土地一样，大牛导师和顶尖实验室把持着顶级期刊的版面、重大项目的经费和学阀圈子的话语权。你想在他的领地上发文章，就得遵循他的范式、引他的文章、甚至拜他的码头。学术圈的游戏规则是，正确不等于接受。投稿像一场赌博，审稿人的口味、当期版面、甚至运气，都比你那篇精心打磨的论文权重更高。",[224,411,412],{"id":412},"事业意义",[11,414,415],{},"虽然小时候有「长大要当科学家」这种理想，但是，个人认为「学术」这条路并不适合像我这样的平民子弟。没有充裕的家底和财力作为后盾，吃学术这碗饭，也是一种高风险职业。",[11,417,418],{},"我觉得那些在学术圈里搞研究的人，其实也挺可悲的。自己把大量的青春，时间，精力投入到自己的课题里，勉强能讨得经费，靠这个饭碗。因为，选择某个研究领域，在初期往往带有偶然性。但一旦投入，就成了无法回头的豪赌。赌的是这个方向在几十年内不被证伪、不被超越、不被认为是死胡同、不会没落。他用的是几十年的青春和精力做赌注，用最严谨的头脑，从事着一项本质上充满不确定性的高风险事业。",[11,420,421],{},"如果有人突然跳出来用新理论、新范式挑战他，或者推翻了他的观点课题，这无异于把他降维打击成了一块废品，弃之如敝履，这不是嘲讽，而是真实的悲剧。在高度职业化的学术圈里，一个人的身份、地位、自尊，都深深扎根于他那一亩三分地的研究课题。",[11,423,424],{},"当他的理论被推翻，在外人看来不过是一个观点被证伪。但对他而言，无异于整个学术人格被判处了死刑，在这个圈子里，会被迅速边缘化，从而判了死刑。他毕生构建的意义大厦，瞬间崩塌为一座废墟。这就是一种存在主义危机。",[11,426,427,428,431],{},"大多数普通人活下去本身就很难。因为 ",[301,429,430],{},"他们的人生，还有其他责任"," 。若压制住七情六欲，寒窗几十年，去碰学术圈，也未免太委屈了。如果一个人 25 岁读博士，30 岁左右博士毕业，然后经历博士后、非升即走、青年项目竞争，他可能在四十岁前都处于高度竞争状态。而同期进入工业界的人，可能已经积累了财富、住房和职业资本。",[224,433,435],{"id":434},"破局功利化读研","破局：功利化读研",[11,437,438,439,442],{},"在中国，虽然知识分子往往被冠以社会期待的光环，但是这也是一种负担和枷锁。请记住：我们是普通人尤其是出身平民家庭，我们首要目标是生存。在生存生计成为问题的时候，我们 ",[301,440,441],{},"没有义务背负太多期待","。",[356,444,445],{},[11,446,447],{},"沧浪之水清兮，可以濯吾缨；沧浪之水浊兮，可以濯吾足。",[11,449,450],{},"请卸下你的「学术羞耻感」。把研究生学历视为一份职业准入资格证，而非学术朝圣。学术职业是一种高风险选择，而不是所有人都必须承担的使命。",[11,452,453],{},"如何破局——功利化读研不失为一种出路。研究生必须思考：我读研的目的是为了什么？获得更好的就业门槛，暂时避开竞争激烈的就业市场，获得更多选择权。这完全是一种合理的人生规划。",[11,455,456],{},"首要的当然是毕业、混文凭，获得硕博的身份————这也是最现实、也最重要的一条。所以，在读研前期，你的一切目标是，必须以尽可能短的时间，完成最低毕业要求。大量水论文、蹭项目就足矣，不需要尽善尽美。科研嘛，也就那样，随便搞搞就行。",[11,458,459],{},"这是功利化读研真正的溢价所在。既然科研只求及格，那你必须把多出来的精力毫无愧疚地投入生存技能的构建。",[11,461,462],{},"在当前的环境下，「包装」比「做事」更重要，「数量」比「质量」更重要。不要把自己的全部人生价值绑定在学术成果上。科研如此，创业如此，艺术如此，很多长期主义事业都是如此。我不需要证明自己是英雄，我只需要把自己的人生过好。知识值得敬畏，但人的生命也值得敬畏。学术可以是人生的一部分，但不必成为人生的全部。",[11,464,465],{},"希望研究生们，不必神化科研、不必自我内耗、不必绑定学术理想，认清行业真相后，依然可以清醒活着、务实成长。",[11,467,468],{},"到最后，这种「虽千万人，吾往矣」，本身就有壮士断腕的秋风式悲凉，不是吗？😮‍💨",{"title":195,"searchDepth":196,"depth":196,"links":470},[471,472,473,474,475,476],{"id":226,"depth":196,"text":226},{"id":279,"depth":196,"text":280},{"id":345,"depth":196,"text":345},{"id":382,"depth":196,"text":382},{"id":412,"depth":196,"text":412},{"id":434,"depth":196,"text":435},{},"2026-08-14","\u002Fblog\u002F2026\u002F2026-08-14-demystification-research",{"title":214,"description":219},"blog\u002F2026\u002F2026-08-14-demystification-research","自 2025 年入学至今，一年光阴悄然而逝，我已步入研二。回望这一年的研究生生活，我的心态与世界观经历了一场彻底而沉重的重塑——曾经的从容平实，如今已被挥之不去的失落感取代。",[210],"5MCFbs1-_AYzP3xpVDJ1BYVVmL8G250ZC9iJQ5yNkBM",{"id":486,"title":487,"body":488,"description":5428,"draft":198,"enableComment":199,"extension":200,"image":195,"important":198,"location":203,"meta":5429,"navigation":199,"ogImage":203,"onday":5430,"path":5431,"seo":5432,"stem":5433,"summary":5434,"tags":5435,"__hash__":5438},"blog\u002Fblog\u002F2026\u002F2026-08-04-the-representations-in-llm.md","Representation Manifolds of LLM",{"type":8,"value":489,"toc":5418},[490,502,509,520,524,531,537,544,857,1207,1313,1317,1323,1471,1776,1782,1785,2398,3039,3046,3050,3053,3079,3082,3088,3092,3095,3695,3702,3783,3787,3790,3797,4237,4442,4652,4913,4990,4993,4996,5000,5003,5049,5120,5207,5214,5218,5225,5236,5246,5249,5273,5291,5301,5305,5308,5315,5318,5325,5398,5405,5412],[11,491,492,493,501],{},"If you have followed the progress in mechanistic interpretability over the past two years, you have likely encountered a certain kind of figure: when the activation vectors of a particular layer in a large language model are projected down to two or three dimensions and plotted, the representations of concepts such as \"months,\" \"days of the week,\" \"years,\" and \"colors\" do not scatter chaotically through space, but instead arrange themselves along an elegant curve—sometimes even a closed circle or a torus. Researchers such as Chris Olah, Josh Batson at Anthropic, and Engels et al. have all demonstrated this phenomenon in works including ",[494,495,496],"em",{},[240,497,500],{"href":498,"rel":499},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2405.14860",[244],"Not All Language Model Features Are One-Dimensionally Linear",".",[11,503,504,505,508],{},"These discoveries are captivating, but they leave behind an awkward lacuna: ",[301,506,507],{},"we see the manifold, yet we cannot articulate precisely what relationship holds between the manifold and the \"concept\" it represents."," Why would \"year\"—something that ought to be a straight line—be twisted inside the model into a curved，meandering curve through high-dimensional space? Is the circular arrangement of \"colors\" a coincidence, or a necessity? Can the cosine similarity between vectors actually tell us how semantically close two concepts are?",[11,510,511,512,519],{},"Three mathematicians—Alexander Modell, Patrick Rubin-Delanchy, and Nick Whiteley, from Imperial College London, the University of Edinburgh, and the University of Bristol, respectively—attempt, in their paper ",[494,513,514],{},[240,515,518],{"href":516,"rel":517},"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2505.18235",[244],"The Origins of Representation Manifolds in Large Language Models",", to provide, for the first time, a \"minimally viable\" mathematical theory to answer these questions. This post traces the arc of their argument and offers my own reflections.",[224,521,523],{"id":522},"the-linear-representation-hypothesis","The Linear Representation Hypothesis",[11,525,526,527,530],{},"The field of mechanistic interpretability has long been anchored by a central conviction known as the ",[301,528,529],{},"Linear Representation Hypothesis (LRH)",": a model encodes human-interpretable \"features\"—such as \"possesses fluffy ears,\" \"mentions the Eiffel Tower,\" or \"is in Arabic\"—as a set of nearly orthogonal direction vectors in representation space. The representation of a given input is then a sparse linear combination of these direction vectors, weighted by whether and to what degree each feature is present. The Sparse Autoencoder (SAE), a widely adopted interpretability tool today, is built directly on this hypothesis: one trains an autoencoder with a sparsity penalty to approximate these \"dictionary vectors.\"",[11,532,533,534],{},"Yet mounting evidence suggests that this \"black-or-white, one-feature-one-direction\" model cannot account for certain phenomena. The authors cite an extensive body of literature: digits in modular addition tasks are encoded as circles; ring-like structures appear in multilingual models; \"dates\" and \"days of the week\" exhibit distorted toroidal geometries; fractal geometries even emerge in simulated hidden Markov models. These examples share a common thread: ",[301,535,536],{},"a feature is no longer a single line, but an entire, continuous, potentially nonlinearly curved manifold.",[11,538,539,540,543],{},"The field has therefore proposed a generalized version of LRH—the ",[301,541,542],{},"Multi-Dimensional Linear Representation Hypothesis",":",[24,545,548],{"className":546,"translate":28},[547],"katex-display",[24,549,551,633],{"className":550,"translate":28},[27],[24,552,554],{"className":553},[32],[34,555,557],{"xmlns":36,"display":556},"block",[38,558,559,630],{},[41,560,561,566,571,574,577,580,603,611,613,615,617,624,626,628],{},[562,563,565],"mi",{"mathvariant":564},"normal","Ψ",[567,568,570],"mo",{"stretchy":569},"false","(",[562,572,573],{},"x",[567,575,576],{"stretchy":569},")",[567,578,579],{},"=",[581,582,583,586],"munder",{},[567,584,585],{},"∑",[41,587,588,591,594,597,599,601],{},[562,589,590],{},"f",[567,592,593],{},"∈",[562,595,596],{},"F",[567,598,570],{"stretchy":569},[562,600,573],{},[567,602,576],{"stretchy":569},[604,605,606,609],"msub",{},[562,607,608],{},"ρ",[562,610,590],{},[567,612,570],{"stretchy":569},[562,614,573],{},[567,616,576],{"stretchy":569},[604,618,619,622],{},[562,620,621],{},"v",[562,623,590],{},[567,625,570],{"stretchy":569},[562,627,573],{},[567,629,576],{"stretchy":569},[48,631,632],{"encoding":50},"\\Psi(x) = \\sum_{f \\in F(x)} \\rho_f(x) v_f(x)",[24,634,636,669],{"className":635,"ariaHidden":56},[55],[24,637,639,643,646,650,654,658,662,666],{"className":638},[60],[24,640],{"className":641,"style":642},[64],"height:1em;vertical-align:-0.25em;",[24,644,565],{"className":645},[69],[24,647,570],{"className":648},[649],"mopen",[24,651,573],{"className":652},[69,653],"mathnormal",[24,655,576],{"className":656},[657],"mclose",[24,659],{"className":660,"style":661},[79],"margin-right:0.2778em;",[24,663,579],{"className":664},[665],"mrel",[24,667],{"className":668,"style":661},[79],[24,670,672,676,749,753,797,800,803,806,848,851,854],{"className":671},[60],[24,673],{"className":674,"style":675},[64],"height:2.566em;vertical-align:-1.516em;",[24,677,681],{"className":678},[679,680],"mop","op-limits",[24,682,684,740],{"className":683},[84,131],[24,685,687,737],{"className":686},[88],[24,688,691,724],{"className":689,"style":690},[92],"height:1.05em;",[24,692,694,698],{"style":693},"top:-1.809em;margin-left:0em;",[24,695],{"className":696,"style":697},[100],"height:3.05em;",[24,699,701],{"className":700},[109,110,111,108],[24,702,704,708,711,715,718,721],{"className":703},[69,108],[24,705,590],{"className":706,"style":707},[69,653,108],"margin-right:0.1076em;",[24,709,593],{"className":710},[665,108],[24,712,596],{"className":713,"style":714},[69,653,108],"margin-right:0.1389em;",[24,716,570],{"className":717},[649,108],[24,719,573],{"className":720},[69,653,108],[24,722,576],{"className":723},[657,108],[24,725,727,730],{"style":726},"top:-3.05em;",[24,728],{"className":729,"style":697},[100],[24,731,732],{},[24,733,585],{"className":734},[679,735,736],"op-symbol","large-op",[24,738,157],{"className":739},[156],[24,741,743],{"className":742},[88],[24,744,747],{"className":745,"style":746},[92],"height:1.516em;",[24,748],{},[24,750],{"className":751,"style":752},[79],"margin-right:0.1667em;",[24,754,756,759],{"className":755},[69],[24,757,608],{"className":758},[69,653],[24,760,763],{"className":761},[762],"msupsub",[24,764,766,788],{"className":765},[84,131],[24,767,769,785],{"className":768},[88],[24,770,773],{"className":771,"style":772},[92],"height:0.3361em;",[24,774,776,779],{"style":775},"top:-2.55em;margin-left:0em;margin-right:0.05em;",[24,777],{"className":778,"style":101},[100],[24,780,782],{"className":781},[109,110,111,108],[24,783,590],{"className":784,"style":707},[69,653,108],[24,786,157],{"className":787},[156],[24,789,791],{"className":790},[88],[24,792,795],{"className":793,"style":794},[92],"height:0.2861em;",[24,796],{},[24,798,570],{"className":799},[649],[24,801,573],{"className":802},[69,653],[24,804,576],{"className":805},[657],[24,807,809,813],{"className":808},[69],[24,810,621],{"className":811,"style":812},[69,653],"margin-right:0.0359em;",[24,814,816],{"className":815},[762],[24,817,819,840],{"className":818},[84,131],[24,820,822,837],{"className":821},[88],[24,823,825],{"className":824,"style":772},[92],[24,826,828,831],{"style":827},"top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;",[24,829],{"className":830,"style":101},[100],[24,832,834],{"className":833},[109,110,111,108],[24,835,590],{"className":836,"style":707},[69,653,108],[24,838,157],{"className":839},[156],[24,841,843],{"className":842},[88],[24,844,846],{"className":845,"style":794},[92],[24,847],{},[24,849,570],{"className":850},[649],[24,852,573],{"className":853},[69,653],[24,855,576],{"className":856},[657],[11,858,859,860,946,947,976,977,1051,1052,1136,1137,1206],{},"Here ",[24,861,863,887],{"className":862,"translate":28},[27],[24,864,866],{"className":865},[32],[34,867,868],{"xmlns":36},[38,869,870,884],{},[41,871,872,878,880,882],{},[604,873,874,876],{},[562,875,621],{},[562,877,590],{},[567,879,570],{"stretchy":569},[562,881,573],{},[567,883,576],{"stretchy":569},[48,885,886],{"encoding":50},"v_f(x)",[24,888,890],{"className":889,"ariaHidden":56},[55],[24,891,893,897,937,940,943],{"className":892},[60],[24,894],{"className":895,"style":896},[64],"height:1.0361em;vertical-align:-0.2861em;",[24,898,900,903],{"className":899},[69],[24,901,621],{"className":902,"style":812},[69,653],[24,904,906],{"className":905},[762],[24,907,909,929],{"className":908},[84,131],[24,910,912,926],{"className":911},[88],[24,913,915],{"className":914,"style":772},[92],[24,916,917,920],{"style":827},[24,918],{"className":919,"style":101},[100],[24,921,923],{"className":922},[109,110,111,108],[24,924,590],{"className":925,"style":707},[69,653,108],[24,927,157],{"className":928},[156],[24,930,932],{"className":931},[88],[24,933,935],{"className":934,"style":794},[92],[24,936],{},[24,938,570],{"className":939},[649],[24,941,573],{"className":942},[69,653],[24,944,576],{"className":945},[657]," is no longer a fixed direction but can vary continuously with the input ",[24,948,950,963],{"className":949,"translate":28},[27],[24,951,953],{"className":952},[32],[34,954,955],{"xmlns":36},[38,956,957,961],{},[41,958,959],{},[562,960,573],{},[48,962,573],{"encoding":50},[24,964,966],{"className":965,"ariaHidden":56},[55],[24,967,969,973],{"className":968},[60],[24,970],{"className":971,"style":972},[64],"height:0.4306em;",[24,974,573],{"className":975},[69,653]," within some subspace ",[24,978,980,999],{"className":979,"translate":28},[27],[24,981,983],{"className":982},[32],[34,984,985],{"xmlns":36},[38,986,987,996],{},[41,988,989],{},[604,990,991,994],{},[562,992,993],{},"V",[562,995,590],{},[48,997,998],{"encoding":50},"V_f",[24,1000,1002],{"className":1001,"ariaHidden":56},[55],[24,1003,1005,1009],{"className":1004},[60],[24,1006],{"className":1007,"style":1008},[64],"height:0.9694em;vertical-align:-0.2861em;",[24,1010,1012,1016],{"className":1011},[69],[24,1013,993],{"className":1014,"style":1015},[69,653],"margin-right:0.2222em;",[24,1017,1019],{"className":1018},[762],[24,1020,1022,1043],{"className":1021},[84,131],[24,1023,1025,1040],{"className":1024},[88],[24,1026,1028],{"className":1027,"style":772},[92],[24,1029,1031,1034],{"style":1030},"top:-2.55em;margin-left:-0.2222em;margin-right:0.05em;",[24,1032],{"className":1033,"style":101},[100],[24,1035,1037],{"className":1036},[109,110,111,108],[24,1038,590],{"className":1039,"style":707},[69,653,108],[24,1041,157],{"className":1042},[156],[24,1044,1046],{"className":1045},[88],[24,1047,1049],{"className":1048,"style":794},[92],[24,1050],{},". The standard LRH is merely the special case where ",[24,1053,1055,1078],{"className":1054,"translate":28},[27],[24,1056,1058],{"className":1057},[32],[34,1059,1060],{"xmlns":36},[38,1061,1062,1076],{},[41,1063,1064,1070,1072,1074],{},[604,1065,1066,1068],{},[562,1067,621],{},[562,1069,590],{},[567,1071,570],{"stretchy":569},[562,1073,573],{},[567,1075,576],{"stretchy":569},[48,1077,886],{"encoding":50},[24,1079,1081],{"className":1080,"ariaHidden":56},[55],[24,1082,1084,1087,1127,1130,1133],{"className":1083},[60],[24,1085],{"className":1086,"style":896},[64],[24,1088,1090,1093],{"className":1089},[69],[24,1091,621],{"className":1092,"style":812},[69,653],[24,1094,1096],{"className":1095},[762],[24,1097,1099,1119],{"className":1098},[84,131],[24,1100,1102,1116],{"className":1101},[88],[24,1103,1105],{"className":1104,"style":772},[92],[24,1106,1107,1110],{"style":827},[24,1108],{"className":1109,"style":101},[100],[24,1111,1113],{"className":1112},[109,110,111,108],[24,1114,590],{"className":1115,"style":707},[69,653,108],[24,1117,157],{"className":1118},[156],[24,1120,1122],{"className":1121},[88],[24,1123,1125],{"className":1124,"style":794},[92],[24,1126],{},[24,1128,570],{"className":1129},[649],[24,1131,573],{"className":1132},[69,653],[24,1134,576],{"className":1135},[657]," is constant and ",[24,1138,1140,1157],{"className":1139,"translate":28},[27],[24,1141,1143],{"className":1142},[32],[34,1144,1145],{"xmlns":36},[38,1146,1147,1155],{},[41,1148,1149],{},[604,1150,1151,1153],{},[562,1152,993],{},[562,1154,590],{},[48,1156,998],{"encoding":50},[24,1158,1160],{"className":1159,"ariaHidden":56},[55],[24,1161,1163,1166],{"className":1162},[60],[24,1164],{"className":1165,"style":1008},[64],[24,1167,1169,1172],{"className":1168},[69],[24,1170,993],{"className":1171,"style":1015},[69,653],[24,1173,1175],{"className":1174},[762],[24,1176,1178,1198],{"className":1177},[84,131],[24,1179,1181,1195],{"className":1180},[88],[24,1182,1184],{"className":1183,"style":772},[92],[24,1185,1186,1189],{"style":1030},[24,1187],{"className":1188,"style":101},[100],[24,1190,1192],{"className":1191},[109,110,111,108],[24,1193,590],{"className":1194,"style":707},[69,653,108],[24,1196,157],{"className":1197},[156],[24,1199,1201],{"className":1200},[88],[24,1202,1204],{"className":1203,"style":794},[92],[24,1205],{}," is one-dimensional.",[11,1208,1209,1210],{},"What this paper sets out to address is precisely the most central and yet most ambiguous part of this generalized hypothesis: ",[301,1211,1212,1213,1242,1243,1312],{},"in what manifold form does feature ",[24,1214,1216,1229],{"className":1215,"translate":28},[27],[24,1217,1219],{"className":1218},[32],[34,1220,1221],{"xmlns":36},[38,1222,1223,1227],{},[41,1224,1225],{},[562,1226,590],{},[48,1228,590],{"encoding":50},[24,1230,1232],{"className":1231,"ariaHidden":56},[55],[24,1233,1235,1239],{"className":1234},[60],[24,1236],{"className":1237,"style":1238},[64],"height:0.8889em;vertical-align:-0.1944em;",[24,1240,590],{"className":1241,"style":707},[69,653]," manifest within subspace ",[24,1244,1246,1263],{"className":1245,"translate":28},[27],[24,1247,1249],{"className":1248},[32],[34,1250,1251],{"xmlns":36},[38,1252,1253,1261],{},[41,1254,1255],{},[604,1256,1257,1259],{},[562,1258,993],{},[562,1260,590],{},[48,1262,998],{"encoding":50},[24,1264,1266],{"className":1265,"ariaHidden":56},[55],[24,1267,1269,1272],{"className":1268},[60],[24,1270],{"className":1271,"style":1008},[64],[24,1273,1275,1278],{"className":1274},[69],[24,1276,993],{"className":1277,"style":1015},[69,653],[24,1279,1281],{"className":1280},[762],[24,1282,1284,1304],{"className":1283},[84,131],[24,1285,1287,1301],{"className":1286},[88],[24,1288,1290],{"className":1289,"style":772},[92],[24,1291,1292,1295],{"style":1030},[24,1293],{"className":1294,"style":101},[100],[24,1296,1298],{"className":1297},[109,110,111,108],[24,1299,590],{"className":1300,"style":707},[69,653,108],[24,1302,157],{"className":1303},[156],[24,1305,1307],{"className":1306},[88],[24,1308,1310],{"className":1309,"style":794},[92],[24,1311],{},", and what is the relationship between this manifold and the \"feature\" itself?",[224,1314,1316],{"id":1315},"defining-features-as-metric-spaces","Defining \"Features\" as Metric Spaces",[11,1318,1319,1320],{},"The most elegant move in the paper is to first resolve a question that sounds somewhat philosophical but is in fact critically important: ",[301,1321,1322],{},"what exactly is \"a feature\"?",[11,1324,1325,1326,1329,1330,1470],{},"The answer the authors provide is unexpectedly succinct: ",[301,1327,1328],{},"a feature is a metric space"," ",[24,1331,1333,1366],{"className":1332,"translate":28},[27],[24,1334,1336],{"className":1335},[32],[34,1337,1338],{"xmlns":36},[38,1339,1340,1363],{},[41,1341,1342,1344,1351,1354,1361],{},[567,1343,570],{"stretchy":569},[604,1345,1346,1349],{},[562,1347,1348],{},"Z",[562,1350,590],{},[567,1352,1353],{"separator":56},",",[604,1355,1356,1359],{},[562,1357,1358],{},"d",[562,1360,590],{},[567,1362,576],{"stretchy":569},[48,1364,1365],{"encoding":50},"(Z_f, d_f)",[24,1367,1369],{"className":1368,"ariaHidden":56},[55],[24,1370,1372,1375,1378,1420,1424,1427,1467],{"className":1371},[60],[24,1373],{"className":1374,"style":896},[64],[24,1376,570],{"className":1377},[649],[24,1379,1381,1385],{"className":1380},[69],[24,1382,1348],{"className":1383,"style":1384},[69,653],"margin-right:0.0715em;",[24,1386,1388],{"className":1387},[762],[24,1389,1391,1412],{"className":1390},[84,131],[24,1392,1394,1409],{"className":1393},[88],[24,1395,1397],{"className":1396,"style":772},[92],[24,1398,1400,1403],{"style":1399},"top:-2.55em;margin-left:-0.0715em;margin-right:0.05em;",[24,1401],{"className":1402,"style":101},[100],[24,1404,1406],{"className":1405},[109,110,111,108],[24,1407,590],{"className":1408,"style":707},[69,653,108],[24,1410,157],{"className":1411},[156],[24,1413,1415],{"className":1414},[88],[24,1416,1418],{"className":1417,"style":794},[92],[24,1419],{},[24,1421,1353],{"className":1422},[1423],"mpunct",[24,1425],{"className":1426,"style":752},[79],[24,1428,1430,1433],{"className":1429},[69],[24,1431,1358],{"className":1432},[69,653],[24,1434,1436],{"className":1435},[762],[24,1437,1439,1459],{"className":1438},[84,131],[24,1440,1442,1456],{"className":1441},[88],[24,1443,1445],{"className":1444,"style":772},[92],[24,1446,1447,1450],{"style":775},[24,1448],{"className":1449,"style":101},[100],[24,1451,1453],{"className":1452},[109,110,111,108],[24,1454,590],{"className":1455,"style":707},[69,653,108],[24,1457,157],{"className":1458},[156],[24,1460,1462],{"className":1461},[88],[24,1463,1465],{"className":1464,"style":794},[92],[24,1466],{},[24,1468,576],{"className":1469},[657],"—a set together with a notion of \"distance\" defined on that set. This definition, though seemingly plain, turns out to be remarkably expressive:",[1472,1473,1474,1552,1700],"ul",{},[1475,1476,1477,1480,1481,1551],"li",{},[301,1478,1479],{},"Atomic features"," (presence or absence of a cat): ",[24,1482,1484,1502],{"className":1483,"translate":28},[27],[24,1485,1487],{"className":1486},[32],[34,1488,1489],{"xmlns":36},[38,1490,1491,1499],{},[41,1492,1493],{},[604,1494,1495,1497],{},[562,1496,1348],{},[562,1498,590],{},[48,1500,1501],{"encoding":50},"Z_f",[24,1503,1505],{"className":1504,"ariaHidden":56},[55],[24,1506,1508,1511],{"className":1507},[60],[24,1509],{"className":1510,"style":1008},[64],[24,1512,1514,1517],{"className":1513},[69],[24,1515,1348],{"className":1516,"style":1384},[69,653],[24,1518,1520],{"className":1519},[762],[24,1521,1523,1543],{"className":1522},[84,131],[24,1524,1526,1540],{"className":1525},[88],[24,1527,1529],{"className":1528,"style":772},[92],[24,1530,1531,1534],{"style":1399},[24,1532],{"className":1533,"style":101},[100],[24,1535,1537],{"className":1536},[109,110,111,108],[24,1538,590],{"className":1539,"style":707},[69,653,108],[24,1541,157],{"className":1542},[156],[24,1544,1546],{"className":1545},[88],[24,1547,1549],{"className":1548,"style":794},[92],[24,1550],{}," is a singleton set;",[1475,1553,1554,1557,1558,1627,1628,1699],{},[301,1555,1556],{},"Hierarchical features"," (a taxonomic tree of animals): ",[24,1559,1561,1578],{"className":1560,"translate":28},[27],[24,1562,1564],{"className":1563},[32],[34,1565,1566],{"xmlns":36},[38,1567,1568,1576],{},[41,1569,1570],{},[604,1571,1572,1574],{},[562,1573,1348],{},[562,1575,590],{},[48,1577,1501],{"encoding":50},[24,1579,1581],{"className":1580,"ariaHidden":56},[55],[24,1582,1584,1587],{"className":1583},[60],[24,1585],{"className":1586,"style":1008},[64],[24,1588,1590,1593],{"className":1589},[69],[24,1591,1348],{"className":1592,"style":1384},[69,653],[24,1594,1596],{"className":1595},[762],[24,1597,1599,1619],{"className":1598},[84,131],[24,1600,1602,1616],{"className":1601},[88],[24,1603,1605],{"className":1604,"style":772},[92],[24,1606,1607,1610],{"style":1399},[24,1608],{"className":1609,"style":101},[100],[24,1611,1613],{"className":1612},[109,110,111,108],[24,1614,590],{"className":1615,"style":707},[69,653,108],[24,1617,157],{"className":1618},[156],[24,1620,1622],{"className":1621},[88],[24,1623,1625],{"className":1624,"style":794},[92],[24,1626],{}," is a discrete set, and ",[24,1629,1631,1649],{"className":1630,"translate":28},[27],[24,1632,1634],{"className":1633},[32],[34,1635,1636],{"xmlns":36},[38,1637,1638,1646],{},[41,1639,1640],{},[604,1641,1642,1644],{},[562,1643,1358],{},[562,1645,590],{},[48,1647,1648],{"encoding":50},"d_f",[24,1650,1652],{"className":1651,"ariaHidden":56},[55],[24,1653,1655,1659],{"className":1654},[60],[24,1656],{"className":1657,"style":1658},[64],"height:0.9805em;vertical-align:-0.2861em;",[24,1660,1662,1665],{"className":1661},[69],[24,1663,1358],{"className":1664},[69,653],[24,1666,1668],{"className":1667},[762],[24,1669,1671,1691],{"className":1670},[84,131],[24,1672,1674,1688],{"className":1673},[88],[24,1675,1677],{"className":1676,"style":772},[92],[24,1678,1679,1682],{"style":775},[24,1680],{"className":1681,"style":101},[100],[24,1683,1685],{"className":1684},[109,110,111,108],[24,1686,590],{"className":1687,"style":707},[69,653,108],[24,1689,157],{"className":1690},[156],[24,1692,1694],{"className":1693},[88],[24,1695,1697],{"className":1696,"style":794},[92],[24,1698],{}," is the tree-distance on it;",[1475,1701,1702,1705,1706,1775],{},[301,1703,1704],{},"Continuous features"," (color hue, day of the year, calendar year): ",[24,1707,1709,1726],{"className":1708,"translate":28},[27],[24,1710,1712],{"className":1711},[32],[34,1713,1714],{"xmlns":36},[38,1715,1716,1724],{},[41,1717,1718],{},[604,1719,1720,1722],{},[562,1721,1348],{},[562,1723,590],{},[48,1725,1501],{"encoding":50},[24,1727,1729],{"className":1728,"ariaHidden":56},[55],[24,1730,1732,1735],{"className":1731},[60],[24,1733],{"className":1734,"style":1008},[64],[24,1736,1738,1741],{"className":1737},[69],[24,1739,1348],{"className":1740,"style":1384},[69,653],[24,1742,1744],{"className":1743},[762],[24,1745,1747,1767],{"className":1746},[84,131],[24,1748,1750,1764],{"className":1749},[88],[24,1751,1753],{"className":1752,"style":772},[92],[24,1754,1755,1758],{"style":1399},[24,1756],{"className":1757,"style":101},[100],[24,1759,1761],{"className":1760},[109,110,111,108],[24,1762,590],{"className":1763,"style":707},[69,653,108],[24,1765,157],{"className":1766},[156],[24,1768,1770],{"className":1769},[88],[24,1771,1773],{"className":1772,"style":794},[92],[24,1774],{}," can be an interval, a circle, or a higher-dimensional Euclidean space.",[11,1777,1778,1779],{},"This framework is more flexible than the \"Euclidean space\" or \"hypersphere\" assumptions common in learning theory, yet substantially simpler—and more tractable—than the Riemannian manifolds with group structure found in the disentanglement literature. It is a characteristically mathematician's choice: ",[301,1780,1781],{},"strike the right balance between expressive power and tractability.",[11,1783,1784],{},"With the definition of \"feature as metric space\" in hand, the authors state the first core hypothesis of the paper:",[356,1786,1787],{},[11,1788,1789,1792,1793,1881,1882,1966,1967,2180,2181,501],{},[301,1790,1791],{},"Hypothesis 1 (Continuous Correspondence Hypothesis)."," There exists a continuous, invertible, one-to-one correspondence between feature values ",[24,1794,1796,1821],{"className":1795,"translate":28},[27],[24,1797,1799],{"className":1798},[32],[34,1800,1801],{"xmlns":36},[38,1802,1803,1818],{},[41,1804,1805,1812,1814,1816],{},[604,1806,1807,1810],{},[562,1808,1809],{},"z",[562,1811,590],{},[567,1813,570],{"stretchy":569},[562,1815,573],{},[567,1817,576],{"stretchy":569},[48,1819,1820],{"encoding":50},"z_f(x)",[24,1822,1824],{"className":1823,"ariaHidden":56},[55],[24,1825,1827,1830,1872,1875,1878],{"className":1826},[60],[24,1828],{"className":1829,"style":896},[64],[24,1831,1833,1837],{"className":1832},[69],[24,1834,1809],{"className":1835,"style":1836},[69,653],"margin-right:0.044em;",[24,1838,1840],{"className":1839},[762],[24,1841,1843,1864],{"className":1842},[84,131],[24,1844,1846,1861],{"className":1845},[88],[24,1847,1849],{"className":1848,"style":772},[92],[24,1850,1852,1855],{"style":1851},"top:-2.55em;margin-left:-0.044em;margin-right:0.05em;",[24,1853],{"className":1854,"style":101},[100],[24,1856,1858],{"className":1857},[109,110,111,108],[24,1859,590],{"className":1860,"style":707},[69,653,108],[24,1862,157],{"className":1863},[156],[24,1865,1867],{"className":1866},[88],[24,1868,1870],{"className":1869,"style":794},[92],[24,1871],{},[24,1873,570],{"className":1874},[649],[24,1876,573],{"className":1877},[69,653],[24,1879,576],{"className":1880},[657]," and representation directions ",[24,1883,1885,1908],{"className":1884,"translate":28},[27],[24,1886,1888],{"className":1887},[32],[34,1889,1890],{"xmlns":36},[38,1891,1892,1906],{},[41,1893,1894,1900,1902,1904],{},[604,1895,1896,1898],{},[562,1897,621],{},[562,1899,590],{},[567,1901,570],{"stretchy":569},[562,1903,573],{},[567,1905,576],{"stretchy":569},[48,1907,886],{"encoding":50},[24,1909,1911],{"className":1910,"ariaHidden":56},[55],[24,1912,1914,1917,1957,1960,1963],{"className":1913},[60],[24,1915],{"className":1916,"style":896},[64],[24,1918,1920,1923],{"className":1919},[69],[24,1921,621],{"className":1922,"style":812},[69,653],[24,1924,1926],{"className":1925},[762],[24,1927,1929,1949],{"className":1928},[84,131],[24,1930,1932,1946],{"className":1931},[88],[24,1933,1935],{"className":1934,"style":772},[92],[24,1936,1937,1940],{"style":827},[24,1938],{"className":1939,"style":101},[100],[24,1941,1943],{"className":1942},[109,110,111,108],[24,1944,590],{"className":1945,"style":707},[69,653,108],[24,1947,157],{"className":1948},[156],[24,1950,1952],{"className":1951},[88],[24,1953,1955],{"className":1954,"style":794},[92],[24,1956],{},[24,1958,570],{"className":1959},[649],[24,1961,573],{"className":1962},[69,653],[24,1964,576],{"className":1965},[657],"; that is, there exists a continuous map ",[24,1968,1970,2018],{"className":1969,"translate":28},[27],[24,1971,1973],{"className":1972},[32],[34,1974,1975],{"xmlns":36},[38,1976,1977,2015],{},[41,1978,1979,1986,1988,1994,1997],{},[604,1980,1981,1984],{},[562,1982,1983],{},"ϕ",[562,1985,590],{},[567,1987,543],{},[604,1989,1990,1992],{},[562,1991,1348],{},[562,1993,590],{},[567,1995,1996],{},"→",[1998,1999,2000,2003],"msup",{},[562,2001,2002],{},"S",[41,2004,2005,2008,2011],{},[562,2006,2007],{},"D",[567,2009,2010],{},"−",[2012,2013,2014],"mn",{},"1",[48,2016,2017],{"encoding":50},"\\phi_f: Z_f \\to S^{D-1}",[24,2019,2021,2076,2131],{"className":2020,"ariaHidden":56},[55],[24,2022,2024,2027,2067,2070,2073],{"className":2023},[60],[24,2025],{"className":2026,"style":1658},[64],[24,2028,2030,2033],{"className":2029},[69],[24,2031,1983],{"className":2032},[69,653],[24,2034,2036],{"className":2035},[762],[24,2037,2039,2059],{"className":2038},[84,131],[24,2040,2042,2056],{"className":2041},[88],[24,2043,2045],{"className":2044,"style":772},[92],[24,2046,2047,2050],{"style":775},[24,2048],{"className":2049,"style":101},[100],[24,2051,2053],{"className":2052},[109,110,111,108],[24,2054,590],{"className":2055,"style":707},[69,653,108],[24,2057,157],{"className":2058},[156],[24,2060,2062],{"className":2061},[88],[24,2063,2065],{"className":2064,"style":794},[92],[24,2066],{},[24,2068],{"className":2069,"style":661},[79],[24,2071,543],{"className":2072},[665],[24,2074],{"className":2075,"style":661},[79],[24,2077,2079,2082,2122,2125,2128],{"className":2078},[60],[24,2080],{"className":2081,"style":1008},[64],[24,2083,2085,2088],{"className":2084},[69],[24,2086,1348],{"className":2087,"style":1384},[69,653],[24,2089,2091],{"className":2090},[762],[24,2092,2094,2114],{"className":2093},[84,131],[24,2095,2097,2111],{"className":2096},[88],[24,2098,2100],{"className":2099,"style":772},[92],[24,2101,2102,2105],{"style":1399},[24,2103],{"className":2104,"style":101},[100],[24,2106,2108],{"className":2107},[109,110,111,108],[24,2109,590],{"className":2110,"style":707},[69,653,108],[24,2112,157],{"className":2113},[156],[24,2115,2117],{"className":2116},[88],[24,2118,2120],{"className":2119,"style":794},[92],[24,2121],{},[24,2123],{"className":2124,"style":661},[79],[24,2126,1996],{"className":2127},[665],[24,2129],{"className":2130,"style":661},[79],[24,2132,2134,2138],{"className":2133},[60],[24,2135],{"className":2136,"style":2137},[64],"height:0.8413em;",[24,2139,2141,2145],{"className":2140},[69],[24,2142,2002],{"className":2143,"style":2144},[69,653],"margin-right:0.0576em;",[24,2146,2148],{"className":2147},[762],[24,2149,2151],{"className":2150},[84],[24,2152,2154],{"className":2153},[88],[24,2155,2157],{"className":2156,"style":2137},[92],[24,2158,2160,2163],{"style":2159},"top:-3.063em;margin-right:0.05em;",[24,2161],{"className":2162,"style":101},[100],[24,2164,2166],{"className":2165},[109,110,111,108],[24,2167,2169,2173,2177],{"className":2168},[69,108],[24,2170,2007],{"className":2171,"style":2172},[69,653,108],"margin-right:0.0278em;",[24,2174,2010],{"className":2175},[2176,108],"mbin",[24,2178,2014],{"className":2179},[69,108]," (into the unit hypersphere) such that ",[24,2182,2184,2232],{"className":2183,"translate":28},[27],[24,2185,2187],{"className":2186},[32],[34,2188,2189],{"xmlns":36},[38,2190,2191,2229],{},[41,2192,2193,2199,2201,2203,2205,2207,2213,2215,2221,2223,2225,2227],{},[604,2194,2195,2197],{},[562,2196,621],{},[562,2198,590],{},[567,2200,570],{"stretchy":569},[562,2202,573],{},[567,2204,576],{"stretchy":569},[567,2206,579],{},[604,2208,2209,2211],{},[562,2210,1983],{},[562,2212,590],{},[567,2214,570],{"stretchy":569},[604,2216,2217,2219],{},[562,2218,1809],{},[562,2220,590],{},[567,2222,570],{"stretchy":569},[562,2224,573],{},[567,2226,576],{"stretchy":569},[567,2228,576],{"stretchy":569},[48,2230,2231],{"encoding":50},"v_f(x) = \\phi_f(z_f(x))",[24,2233,2235,2299],{"className":2234,"ariaHidden":56},[55],[24,2236,2238,2241,2281,2284,2287,2290,2293,2296],{"className":2237},[60],[24,2239],{"className":2240,"style":896},[64],[24,2242,2244,2247],{"className":2243},[69],[24,2245,621],{"className":2246,"style":812},[69,653],[24,2248,2250],{"className":2249},[762],[24,2251,2253,2273],{"className":2252},[84,131],[24,2254,2256,2270],{"className":2255},[88],[24,2257,2259],{"className":2258,"style":772},[92],[24,2260,2261,2264],{"style":827},[24,2262],{"className":2263,"style":101},[100],[24,2265,2267],{"className":2266},[109,110,111,108],[24,2268,590],{"className":2269,"style":707},[69,653,108],[24,2271,157],{"className":2272},[156],[24,2274,2276],{"className":2275},[88],[24,2277,2279],{"className":2278,"style":794},[92],[24,2280],{},[24,2282,570],{"className":2283},[649],[24,2285,573],{"className":2286},[69,653],[24,2288,576],{"className":2289},[657],[24,2291],{"className":2292,"style":661},[79],[24,2294,579],{"className":2295},[665],[24,2297],{"className":2298,"style":661},[79],[24,2300,2302,2305,2345,2348,2388,2391,2394],{"className":2301},[60],[24,2303],{"className":2304,"style":896},[64],[24,2306,2308,2311],{"className":2307},[69],[24,2309,1983],{"className":2310},[69,653],[24,2312,2314],{"className":2313},[762],[24,2315,2317,2337],{"className":2316},[84,131],[24,2318,2320,2334],{"className":2319},[88],[24,2321,2323],{"className":2322,"style":772},[92],[24,2324,2325,2328],{"style":775},[24,2326],{"className":2327,"style":101},[100],[24,2329,2331],{"className":2330},[109,110,111,108],[24,2332,590],{"className":2333,"style":707},[69,653,108],[24,2335,157],{"className":2336},[156],[24,2338,2340],{"className":2339},[88],[24,2341,2343],{"className":2342,"style":794},[92],[24,2344],{},[24,2346,570],{"className":2347},[649],[24,2349,2351,2354],{"className":2350},[69],[24,2352,1809],{"className":2353,"style":1836},[69,653],[24,2355,2357],{"className":2356},[762],[24,2358,2360,2380],{"className":2359},[84,131],[24,2361,2363,2377],{"className":2362},[88],[24,2364,2366],{"className":2365,"style":772},[92],[24,2367,2368,2371],{"style":1851},[24,2369],{"className":2370,"style":101},[100],[24,2372,2374],{"className":2373},[109,110,111,108],[24,2375,590],{"className":2376,"style":707},[69,653,108],[24,2378,157],{"className":2379},[156],[24,2381,2383],{"className":2382},[88],[24,2384,2386],{"className":2385,"style":794},[92],[24,2387],{},[24,2389,570],{"className":2390},[649],[24,2392,573],{"className":2393},[69,653],[24,2395,2397],{"className":2396},[657],"))",[11,2399,2400,2401,2470,2471,2544,2545,2618,2619,2688,2689,2758,2759,2828,2829,2898,2899,2968,2969,3038],{},"Coupled with the technical premise that ",[24,2402,2404,2421],{"className":2403,"translate":28},[27],[24,2405,2407],{"className":2406},[32],[34,2408,2409],{"xmlns":36},[38,2410,2411,2419],{},[41,2412,2413],{},[604,2414,2415,2417],{},[562,2416,1348],{},[562,2418,590],{},[48,2420,1501],{"encoding":50},[24,2422,2424],{"className":2423,"ariaHidden":56},[55],[24,2425,2427,2430],{"className":2426},[60],[24,2428],{"className":2429,"style":1008},[64],[24,2431,2433,2436],{"className":2432},[69],[24,2434,1348],{"className":2435,"style":1384},[69,653],[24,2437,2439],{"className":2438},[762],[24,2440,2442,2462],{"className":2441},[84,131],[24,2443,2445,2459],{"className":2444},[88],[24,2446,2448],{"className":2447,"style":772},[92],[24,2449,2450,2453],{"style":1399},[24,2451],{"className":2452,"style":101},[100],[24,2454,2456],{"className":2455},[109,110,111,108],[24,2457,590],{"className":2458,"style":707},[69,653,108],[24,2460,157],{"className":2461},[156],[24,2463,2465],{"className":2464},[88],[24,2466,2468],{"className":2467,"style":794},[92],[24,2469],{}," is compact, this hypothesis immediately yields a clean corollary (Proposition 1): ",[301,2472,2473,2543],{},[24,2474,2476,2494],{"className":2475,"translate":28},[27],[24,2477,2479],{"className":2478},[32],[34,2480,2481],{"xmlns":36},[38,2482,2483,2491],{},[41,2484,2485],{},[604,2486,2487,2489],{},[562,2488,1983],{},[562,2490,590],{},[48,2492,2493],{"encoding":50},"\\phi_f",[24,2495,2497],{"className":2496,"ariaHidden":56},[55],[24,2498,2500,2503],{"className":2499},[60],[24,2501],{"className":2502,"style":1658},[64],[24,2504,2506,2509],{"className":2505},[69],[24,2507,1983],{"className":2508},[69,653],[24,2510,2512],{"className":2511},[762],[24,2513,2515,2535],{"className":2514},[84,131],[24,2516,2518,2532],{"className":2517},[88],[24,2519,2521],{"className":2520,"style":772},[92],[24,2522,2523,2526],{"style":775},[24,2524],{"className":2525,"style":101},[100],[24,2527,2529],{"className":2528},[109,110,111,108],[24,2530,590],{"className":2531,"style":707},[69,653,108],[24,2533,157],{"className":2534},[156],[24,2536,2538],{"className":2537},[88],[24,2539,2541],{"className":2540,"style":794},[92],[24,2542],{}," is a homeomorphism."," In other words, the representation manifold ",[24,2546,2548,2567],{"className":2547,"translate":28},[27],[24,2549,2551],{"className":2550},[32],[34,2552,2553],{"xmlns":36},[38,2554,2555,2564],{},[41,2556,2557],{},[604,2558,2559,2562],{},[562,2560,2561],{},"M",[562,2563,590],{},[48,2565,2566],{"encoding":50},"M_f",[24,2568,2570],{"className":2569,"ariaHidden":56},[55],[24,2571,2573,2576],{"className":2572},[60],[24,2574],{"className":2575,"style":1008},[64],[24,2577,2579,2583],{"className":2578},[69],[24,2580,2561],{"className":2581,"style":2582},[69,653],"margin-right:0.109em;",[24,2584,2586],{"className":2585},[762],[24,2587,2589,2610],{"className":2588},[84,131],[24,2590,2592,2607],{"className":2591},[88],[24,2593,2595],{"className":2594,"style":772},[92],[24,2596,2598,2601],{"style":2597},"top:-2.55em;margin-left:-0.109em;margin-right:0.05em;",[24,2599],{"className":2600,"style":101},[100],[24,2602,2604],{"className":2603},[109,110,111,108],[24,2605,590],{"className":2606,"style":707},[69,653,108],[24,2608,157],{"className":2609},[156],[24,2611,2613],{"className":2612},[88],[24,2614,2616],{"className":2615,"style":794},[92],[24,2617],{}," (the image of ",[24,2620,2622,2639],{"className":2621,"translate":28},[27],[24,2623,2625],{"className":2624},[32],[34,2626,2627],{"xmlns":36},[38,2628,2629,2637],{},[41,2630,2631],{},[604,2632,2633,2635],{},[562,2634,1983],{},[562,2636,590],{},[48,2638,2493],{"encoding":50},[24,2640,2642],{"className":2641,"ariaHidden":56},[55],[24,2643,2645,2648],{"className":2644},[60],[24,2646],{"className":2647,"style":1658},[64],[24,2649,2651,2654],{"className":2650},[69],[24,2652,1983],{"className":2653},[69,653],[24,2655,2657],{"className":2656},[762],[24,2658,2660,2680],{"className":2659},[84,131],[24,2661,2663,2677],{"className":2662},[88],[24,2664,2666],{"className":2665,"style":772},[92],[24,2667,2668,2671],{"style":775},[24,2669],{"className":2670,"style":101},[100],[24,2672,2674],{"className":2673},[109,110,111,108],[24,2675,590],{"className":2676,"style":707},[69,653,108],[24,2678,157],{"className":2679},[156],[24,2681,2683],{"className":2682},[88],[24,2684,2686],{"className":2685,"style":794},[92],[24,2687],{},") is topologically \"the same shape\" as the original feature space ",[24,2690,2692,2709],{"className":2691,"translate":28},[27],[24,2693,2695],{"className":2694},[32],[34,2696,2697],{"xmlns":36},[38,2698,2699,2707],{},[41,2700,2701],{},[604,2702,2703,2705],{},[562,2704,1348],{},[562,2706,590],{},[48,2708,1501],{"encoding":50},[24,2710,2712],{"className":2711,"ariaHidden":56},[55],[24,2713,2715,2718],{"className":2714},[60],[24,2716],{"className":2717,"style":1008},[64],[24,2719,2721,2724],{"className":2720},[69],[24,2722,1348],{"className":2723,"style":1384},[69,653],[24,2725,2727],{"className":2726},[762],[24,2728,2730,2750],{"className":2729},[84,131],[24,2731,2733,2747],{"className":2732},[88],[24,2734,2736],{"className":2735,"style":772},[92],[24,2737,2738,2741],{"style":1399},[24,2739],{"className":2740,"style":101},[100],[24,2742,2744],{"className":2743},[109,110,111,108],[24,2745,590],{"className":2746,"style":707},[69,653,108],[24,2748,157],{"className":2749},[156],[24,2751,2753],{"className":2752},[88],[24,2754,2756],{"className":2755,"style":794},[92],[24,2757],{},": if ",[24,2760,2762,2779],{"className":2761,"translate":28},[27],[24,2763,2765],{"className":2764},[32],[34,2766,2767],{"xmlns":36},[38,2768,2769,2777],{},[41,2770,2771],{},[604,2772,2773,2775],{},[562,2774,1348],{},[562,2776,590],{},[48,2778,1501],{"encoding":50},[24,2780,2782],{"className":2781,"ariaHidden":56},[55],[24,2783,2785,2788],{"className":2784},[60],[24,2786],{"className":2787,"style":1008},[64],[24,2789,2791,2794],{"className":2790},[69],[24,2792,1348],{"className":2793,"style":1384},[69,653],[24,2795,2797],{"className":2796},[762],[24,2798,2800,2820],{"className":2799},[84,131],[24,2801,2803,2817],{"className":2802},[88],[24,2804,2806],{"className":2805,"style":772},[92],[24,2807,2808,2811],{"style":1399},[24,2809],{"className":2810,"style":101},[100],[24,2812,2814],{"className":2813},[109,110,111,108],[24,2815,590],{"className":2816,"style":707},[69,653,108],[24,2818,157],{"className":2819},[156],[24,2821,2823],{"className":2822},[88],[24,2824,2826],{"className":2825,"style":794},[92],[24,2827],{}," is an interval, ",[24,2830,2832,2849],{"className":2831,"translate":28},[27],[24,2833,2835],{"className":2834},[32],[34,2836,2837],{"xmlns":36},[38,2838,2839,2847],{},[41,2840,2841],{},[604,2842,2843,2845],{},[562,2844,2561],{},[562,2846,590],{},[48,2848,2566],{"encoding":50},[24,2850,2852],{"className":2851,"ariaHidden":56},[55],[24,2853,2855,2858],{"className":2854},[60],[24,2856],{"className":2857,"style":1008},[64],[24,2859,2861,2864],{"className":2860},[69],[24,2862,2561],{"className":2863,"style":2582},[69,653],[24,2865,2867],{"className":2866},[762],[24,2868,2870,2890],{"className":2869},[84,131],[24,2871,2873,2887],{"className":2872},[88],[24,2874,2876],{"className":2875,"style":772},[92],[24,2877,2878,2881],{"style":2597},[24,2879],{"className":2880,"style":101},[100],[24,2882,2884],{"className":2883},[109,110,111,108],[24,2885,590],{"className":2886,"style":707},[69,653,108],[24,2888,157],{"className":2889},[156],[24,2891,2893],{"className":2892},[88],[24,2894,2896],{"className":2895,"style":794},[92],[24,2897],{}," is a curve; if ",[24,2900,2902,2919],{"className":2901,"translate":28},[27],[24,2903,2905],{"className":2904},[32],[34,2906,2907],{"xmlns":36},[38,2908,2909,2917],{},[41,2910,2911],{},[604,2912,2913,2915],{},[562,2914,1348],{},[562,2916,590],{},[48,2918,1501],{"encoding":50},[24,2920,2922],{"className":2921,"ariaHidden":56},[55],[24,2923,2925,2928],{"className":2924},[60],[24,2926],{"className":2927,"style":1008},[64],[24,2929,2931,2934],{"className":2930},[69],[24,2932,1348],{"className":2933,"style":1384},[69,653],[24,2935,2937],{"className":2936},[762],[24,2938,2940,2960],{"className":2939},[84,131],[24,2941,2943,2957],{"className":2942},[88],[24,2944,2946],{"className":2945,"style":772},[92],[24,2947,2948,2951],{"style":1399},[24,2949],{"className":2950,"style":101},[100],[24,2952,2954],{"className":2953},[109,110,111,108],[24,2955,590],{"className":2956,"style":707},[69,653,108],[24,2958,157],{"className":2959},[156],[24,2961,2963],{"className":2962},[88],[24,2964,2966],{"className":2965,"style":794},[92],[24,2967],{}," is a circle, ",[24,2970,2972,2989],{"className":2971,"translate":28},[27],[24,2973,2975],{"className":2974},[32],[34,2976,2977],{"xmlns":36},[38,2978,2979,2987],{},[41,2980,2981],{},[604,2982,2983,2985],{},[562,2984,2561],{},[562,2986,590],{},[48,2988,2566],{"encoding":50},[24,2990,2992],{"className":2991,"ariaHidden":56},[55],[24,2993,2995,2998],{"className":2994},[60],[24,2996],{"className":2997,"style":1008},[64],[24,2999,3001,3004],{"className":3000},[69],[24,3002,2561],{"className":3003,"style":2582},[69,653],[24,3005,3007],{"className":3006},[762],[24,3008,3010,3030],{"className":3009},[84,131],[24,3011,3013,3027],{"className":3012},[88],[24,3014,3016],{"className":3015,"style":772},[92],[24,3017,3018,3021],{"style":2597},[24,3019],{"className":3020,"style":101},[100],[24,3022,3024],{"className":3023},[109,110,111,108],[24,3025,590],{"className":3026,"style":707},[69,653,108],[24,3028,157],{"className":3029},[156],[24,3031,3033],{"className":3032},[88],[24,3034,3036],{"className":3035,"style":794},[92],[24,3037],{}," is a loop; connected components, holes, and branch points—all these topological properties are faithfully preserved.",[11,3040,3041,3042,3045],{},"The significance of this step is that ",[301,3043,3044],{},"for the first time, it ties \"what the manifold looks like\" to \"the structure of the concept itself\" in rigorous mathematical language",", rather than stopping at the intuitive description of \"it looks like a circle.\"",[224,3047,3049],{"id":3048},"empirical-validation-with-real-data","Empirical Validation with Real Data",[11,3051,3052],{},"A theory, however elegant, must answer to data. The authors selected three representative case studies:",[3054,3055,3056,3067,3073],"ol",{},[1475,3057,3058,3061,3062,3066],{},[301,3059,3060],{},"Colors",": 3072-dimensional embeddings were generated from English color names using OpenAI's ",[3063,3064,3065],"code",{},"text-embedding-large-3",", then reduced to three dimensions via PCA for visualization;",[1475,3068,3069,3072],{},[301,3070,3071],{},"Years",": the \"20th-century year\" feature was extracted from layer 7 of GPT-2-small using SAEs, following Engels et al. (2025);",[1475,3074,3075,3078],{},[301,3076,3077],{},"Dates",": embeddings were generated from prompts such as \"January 1st\" through \"December 31st,\" again using OpenAI's embedding model.",[11,3080,3081],{},"The results are striking: the color embeddings arrange themselves along a ring, with the hue ordering (red → purple → blue → green → yellow → orange → red) matching the standard color wheel exactly. The token activations for years trace out a curve that winds through three-dimensional space, faintly evocative of the human intuition of a \"timeline.\" The authors further estimated the ordinal structure along the manifold using a k-nearest-neighbor graph and computed rank correlations with the true years, obtaining a Kendall correlation coefficient of 0.97 and a Spearman correlation exceeding 0.99—an almost perfectly monotonic correspondence, providing strong support for the homeomorphism prediction.",[11,3083,3084,3085],{},"The paper also contains a particularly revealing \"gotcha\" detail: in the original Engels et al. work, the representations of \"days of the week\" and \"months,\" when projected onto the first two principal components, appeared as tidy circles. The authors point out, however, that once the third principal component is examined, one finds that this \"circle\" is in fact continuously twisting and weaving in the third dimension; the clean circle compressed into a two-dimensional plane is a visual artifact (see their Figure 2). This observation serves as a cautionary note: ",[301,3086,3087],{},"low-dimensional projections, while indispensable as visualization tools in interpretability research, can also mislead.",[224,3089,3091],{"id":3090},"an-explanation-in-terms-of-computational-expressivity","An Explanation in Terms of Computational Expressivity",[11,3093,3094],{},"At this point a natural question arises: since \"year\" is fundamentally a one-dimensional quantity, why doesn't the model simply encode it as a straight line segment in representation space, rather than twisting it into a curve through high dimensions?",[11,3096,3097,3098,3101,3102,3130,3131,3207,3208,3336,3337,3365,3366,3427,3428,3587,3588,3632,3633,3694],{},"The authors offer an answer that is both intuitive and characteristically mathematical: ",[301,3099,3100],{},"expressivity",". If the goal were merely to \"read out\" ",[24,3103,3105,3118],{"className":3104,"translate":28},[27],[24,3106,3108],{"className":3107},[32],[34,3109,3110],{"xmlns":36},[38,3111,3112,3116],{},[41,3113,3114],{},[562,3115,1809],{},[48,3117,1809],{"encoding":50},[24,3119,3121],{"className":3120,"ariaHidden":56},[55],[24,3122,3124,3127],{"className":3123},[60],[24,3125],{"className":3126,"style":972},[64],[24,3128,1809],{"className":3129,"style":1836},[69,653]," itself through a linear projection (i.e., to make the identity function ",[24,3132,3134,3163],{"className":3133,"translate":28},[27],[24,3135,3137],{"className":3136},[32],[34,3138,3139],{"xmlns":36},[38,3140,3141,3160],{},[41,3142,3143,3150,3152,3154,3156,3158],{},[41,3144,3145,3148],{},[562,3146,3147],{"mathvariant":564},"i",[562,3149,1358],{"mathvariant":564},[567,3151,570],{"stretchy":569},[562,3153,1809],{},[567,3155,576],{"stretchy":569},[567,3157,579],{},[562,3159,1809],{},[48,3161,3162],{"encoding":50},"\\mathrm{id}(z) = z",[24,3164,3166,3198],{"className":3165,"ariaHidden":56},[55],[24,3167,3169,3172,3180,3183,3186,3189,3192,3195],{"className":3168},[60],[24,3170],{"className":3171,"style":642},[64],[24,3173,3175],{"className":3174},[69],[24,3176,3179],{"className":3177},[69,3178],"mathrm","id",[24,3181,570],{"className":3182},[649],[24,3184,1809],{"className":3185,"style":1836},[69,653],[24,3187,576],{"className":3188},[657],[24,3190],{"className":3191,"style":661},[79],[24,3193,579],{"className":3194},[665],[24,3196],{"className":3197,"style":661},[79],[24,3199,3201,3204],{"className":3200},[60],[24,3202],{"className":3203,"style":972},[64],[24,3205,1809],{"className":3206,"style":1836},[69,653]," computable via a single linear operation), two orthogonal directions ",[24,3209,3211,3238],{"className":3210,"translate":28},[27],[24,3212,3214],{"className":3213},[32],[34,3215,3216],{"xmlns":36},[38,3217,3218,3235],{},[41,3219,3220,3227,3229],{},[604,3221,3222,3224],{},[562,3223,621],{},[2012,3225,3226],{},"0",[567,3228,1353],{"separator":56},[604,3230,3231,3233],{},[562,3232,621],{},[2012,3234,2014],{},[48,3236,3237],{"encoding":50},"v_0, v_1",[24,3239,3241],{"className":3240,"ariaHidden":56},[55],[24,3242,3244,3248,3290,3293,3296],{"className":3243},[60],[24,3245],{"className":3246,"style":3247},[64],"height:0.625em;vertical-align:-0.1944em;",[24,3249,3251,3254],{"className":3250},[69],[24,3252,621],{"className":3253,"style":812},[69,653],[24,3255,3257],{"className":3256},[762],[24,3258,3260,3281],{"className":3259},[84,131],[24,3261,3263,3278],{"className":3262},[88],[24,3264,3267],{"className":3265,"style":3266},[92],"height:0.3011em;",[24,3268,3269,3272],{"style":827},[24,3270],{"className":3271,"style":101},[100],[24,3273,3275],{"className":3274},[109,110,111,108],[24,3276,3226],{"className":3277},[69,108],[24,3279,157],{"className":3280},[156],[24,3282,3284],{"className":3283},[88],[24,3285,3288],{"className":3286,"style":3287},[92],"height:0.15em;",[24,3289],{},[24,3291,1353],{"className":3292},[1423],[24,3294],{"className":3295,"style":752},[79],[24,3297,3299,3302],{"className":3298},[69],[24,3300,621],{"className":3301,"style":812},[69,653],[24,3303,3305],{"className":3304},[762],[24,3306,3308,3328],{"className":3307},[84,131],[24,3309,3311,3325],{"className":3310},[88],[24,3312,3314],{"className":3313,"style":3266},[92],[24,3315,3316,3319],{"style":827},[24,3317],{"className":3318,"style":101},[100],[24,3320,3322],{"className":3321},[109,110,111,108],[24,3323,2014],{"className":3324},[69,108],[24,3326,157],{"className":3327},[156],[24,3329,3331],{"className":3330},[88],[24,3332,3334],{"className":3333,"style":3287},[92],[24,3335],{}," would suffice. But if one also wishes to make higher-order polynomials of ",[24,3338,3340,3353],{"className":3339,"translate":28},[27],[24,3341,3343],{"className":3342},[32],[34,3344,3345],{"xmlns":36},[38,3346,3347,3351],{},[41,3348,3349],{},[562,3350,1809],{},[48,3352,1809],{"encoding":50},[24,3354,3356],{"className":3355,"ariaHidden":56},[55],[24,3357,3359,3362],{"className":3358},[60],[24,3360],{"className":3361,"style":972},[64],[24,3363,1809],{"className":3364,"style":1836},[69,653]," (e.g., ",[24,3367,3369,3388],{"className":3368,"translate":28},[27],[24,3370,3372],{"className":3371},[32],[34,3373,3374],{"xmlns":36},[38,3375,3376,3385],{},[41,3377,3378],{},[1998,3379,3380,3382],{},[562,3381,1809],{},[2012,3383,3384],{},"2",[48,3386,3387],{"encoding":50},"z^2",[24,3389,3391],{"className":3390,"ariaHidden":56},[55],[24,3392,3394,3398],{"className":3393},[60],[24,3395],{"className":3396,"style":3397},[64],"height:0.8141em;",[24,3399,3401,3404],{"className":3400},[69],[24,3402,1809],{"className":3403,"style":1836},[69,653],[24,3405,3407],{"className":3406},[762],[24,3408,3410],{"className":3409},[84],[24,3411,3413],{"className":3412},[88],[24,3414,3416],{"className":3415,"style":3397},[92],[24,3417,3418,3421],{"style":2159},[24,3419],{"className":3420,"style":101},[100],[24,3422,3424],{"className":3423},[109,110,111,108],[24,3425,3384],{"className":3426},[69,108],") linearly readable, then more orthogonal directions ",[24,3429,3431,3469],{"className":3430,"translate":28},[27],[24,3432,3434],{"className":3433},[32],[34,3435,3436],{"xmlns":36},[38,3437,3438,3466],{},[41,3439,3440,3446,3448,3451,3453],{},[604,3441,3442,3444],{},[562,3443,621],{},[2012,3445,3226],{},[567,3447,1353],{"separator":56},[567,3449,3450],{},"…",[567,3452,1353],{"separator":56},[604,3454,3455,3457],{},[562,3456,621],{},[41,3458,3459,3461,3464],{},[562,3460,11],{},[567,3462,3463],{},"+",[2012,3465,2014],{},[48,3467,3468],{"encoding":50},"v_0, \\dots, v_{p+1}",[24,3470,3472],{"className":3471,"ariaHidden":56},[55],[24,3473,3475,3479,3519,3522,3525,3529,3532,3535,3538],{"className":3474},[60],[24,3476],{"className":3477,"style":3478},[64],"height:0.7167em;vertical-align:-0.2861em;",[24,3480,3482,3485],{"className":3481},[69],[24,3483,621],{"className":3484,"style":812},[69,653],[24,3486,3488],{"className":3487},[762],[24,3489,3491,3511],{"className":3490},[84,131],[24,3492,3494,3508],{"className":3493},[88],[24,3495,3497],{"className":3496,"style":3266},[92],[24,3498,3499,3502],{"style":827},[24,3500],{"className":3501,"style":101},[100],[24,3503,3505],{"className":3504},[109,110,111,108],[24,3506,3226],{"className":3507},[69,108],[24,3509,157],{"className":3510},[156],[24,3512,3514],{"className":3513},[88],[24,3515,3517],{"className":3516,"style":3287},[92],[24,3518],{},[24,3520,1353],{"className":3521},[1423],[24,3523],{"className":3524,"style":752},[79],[24,3526,3450],{"className":3527},[3528],"minner",[24,3530],{"className":3531,"style":752},[79],[24,3533,1353],{"className":3534},[1423],[24,3536],{"className":3537,"style":752},[79],[24,3539,3541,3544],{"className":3540},[69],[24,3542,621],{"className":3543,"style":812},[69,653],[24,3545,3547],{"className":3546},[762],[24,3548,3550,3579],{"className":3549},[84,131],[24,3551,3553,3576],{"className":3552},[88],[24,3554,3556],{"className":3555,"style":3266},[92],[24,3557,3558,3561],{"style":827},[24,3559],{"className":3560,"style":101},[100],[24,3562,3564],{"className":3563},[109,110,111,108],[24,3565,3567,3570,3573],{"className":3566},[69,108],[24,3568,11],{"className":3569},[69,653,108],[24,3571,3463],{"className":3572},[2176,108],[24,3574,2014],{"className":3575},[69,108],[24,3577,157],{"className":3578},[156],[24,3580,3582],{"className":3581},[88],[24,3583,3585],{"className":3584,"style":794},[92],[24,3586],{}," are required, and the path traced by ",[24,3589,3591,3611],{"className":3590,"translate":28},[27],[24,3592,3594],{"className":3593},[32],[34,3595,3596],{"xmlns":36},[38,3597,3598,3608],{},[41,3599,3600,3602,3604,3606],{},[562,3601,1983],{},[567,3603,570],{"stretchy":569},[562,3605,1809],{},[567,3607,576],{"stretchy":569},[48,3609,3610],{"encoding":50},"\\phi(z)",[24,3612,3614],{"className":3613,"ariaHidden":56},[55],[24,3615,3617,3620,3623,3626,3629],{"className":3616},[60],[24,3618],{"className":3619,"style":642},[64],[24,3621,1983],{"className":3622},[69,653],[24,3624,570],{"className":3625},[649],[24,3627,1809],{"className":3628,"style":1836},[69,653],[24,3630,576],{"className":3631},[657]," must accordingly bend through a ",[24,3634,3636,3658],{"className":3635,"translate":28},[27],[24,3637,3639],{"className":3638},[32],[34,3640,3641],{"xmlns":36},[38,3642,3643,3655],{},[41,3644,3645,3647,3649,3651,3653],{},[567,3646,570],{"stretchy":569},[562,3648,11],{},[567,3650,3463],{},[2012,3652,3384],{},[567,3654,576],{"stretchy":569},[48,3656,3657],{"encoding":50},"(p+2)",[24,3659,3661,3682],{"className":3660,"ariaHidden":56},[55],[24,3662,3664,3667,3670,3673,3676,3679],{"className":3663},[60],[24,3665],{"className":3666,"style":642},[64],[24,3668,570],{"className":3669},[649],[24,3671,11],{"className":3672},[69,653],[24,3674],{"className":3675,"style":1015},[79],[24,3677,3463],{"className":3678},[2176],[24,3680],{"className":3681,"style":1015},[79],[24,3683,3685,3688,3691],{"className":3684},[60],[24,3686],{"className":3687,"style":642},[64],[24,3689,3384],{"className":3690},[69],[24,3692,576],{"className":3693},[657],"-dimensional subspace.",[11,3696,3697,3698,3701],{},"In other words: ",[301,3699,3700],{},"the degree to which a manifold is \"twisted\" through high-dimensional space encodes, in some sense, how many distinct (nonlinear) functions over this feature can be read out directly by subsequent network layers through a simple linear projection."," This provides a functionalist explanation for why the model encodes a simple one-dimensional concept as a complex high-dimensional manifold—not because the model \"can't help itself,\" but because doing so facilitates downstream computation.",[11,3703,3704,3705,3708,3709,3778,3779,3782],{},"The authors then connect this perspective to the ",[301,3706,3707],{},"superposition hypothesis",": since the representation dimension is far smaller than the number of latent features, the model has little choice but to have most features share the same representation space sparsely and near-orthogonally. Under this constraint, the fact that ",[24,3710,3712,3729],{"className":3711,"translate":28},[27],[24,3713,3715],{"className":3714},[32],[34,3716,3717],{"xmlns":36},[38,3718,3719,3727],{},[41,3720,3721],{},[604,3722,3723,3725],{},[562,3724,1983],{},[562,3726,590],{},[48,3728,2493],{"encoding":50},[24,3730,3732],{"className":3731,"ariaHidden":56},[55],[24,3733,3735,3738],{"className":3734},[60],[24,3736],{"className":3737,"style":1658},[64],[24,3739,3741,3744],{"className":3740},[69],[24,3742,1983],{"className":3743},[69,653],[24,3745,3747],{"className":3746},[762],[24,3748,3750,3770],{"className":3749},[84,131],[24,3751,3753,3767],{"className":3752},[88],[24,3754,3756],{"className":3755,"style":772},[92],[24,3757,3758,3761],{"style":775},[24,3759],{"className":3760,"style":101},[100],[24,3762,3764],{"className":3763},[109,110,111,108],[24,3765,590],{"className":3766,"style":707},[69,653,108],[24,3768,157],{"className":3769},[156],[24,3771,3773],{"className":3772},[88],[24,3774,3776],{"className":3775,"style":794},[92],[24,3777],{},"—the direction encoding for a given feature—does indeed contain the \"linearly readable\" component of the identity function also goes some way toward explaining why simple ",[301,3780,3781],{},"linear probes"," are often surprisingly effective at \"fishing out\" a specific feature from superposed representations in practice.",[224,3784,3786],{"id":3785},"cosine-similarity","Cosine Similarity",[11,3788,3789],{},"If the preceding sections constitute the scaffolding, then this section is the paper's true \"hardcore\" contribution and, in my view, the part most worth remembering.",[11,3791,3792,3793,3796],{},"The authors advance ",[301,3794,3795],{},"Hypothesis 2 (Cosine Similarity Reflects Distance)",": locally, the cosine similarity between representations is some decreasing function of the squared distance between feature values:",[24,3798,3800],{"className":3799,"translate":28},[547],[24,3801,3803,3906],{"className":3802,"translate":28},[27],[24,3804,3806],{"className":3805},[32],[34,3807,3808],{"xmlns":36,"display":556},[38,3809,3810,3903],{},[41,3811,3812,3830,3832,3838,3840,3842,3844,3846,3852,3854,3862,3864,3866,3868,3875,3877,3883,3885,3887,3889,3895,3901],{},[41,3813,3814,3817,3820,3823,3825,3827],{},[562,3815,3816],{"mathvariant":564},"C",[562,3818,3819],{"mathvariant":564},"o",[562,3821,3822],{"mathvariant":564},"s",[562,3824,2002],{"mathvariant":564},[562,3826,3147],{"mathvariant":564},[562,3828,3829],{"mathvariant":564},"m",[567,3831,570],{"stretchy":569},[604,3833,3834,3836],{},[562,3835,1983],{},[562,3837,590],{},[567,3839,570],{"stretchy":569},[562,3841,1809],{},[567,3843,576],{"stretchy":569},[567,3845,1353],{"separator":56},[604,3847,3848,3850],{},[562,3849,1983],{},[562,3851,590],{},[567,3853,570],{"stretchy":569},[1998,3855,3856,3858],{},[562,3857,1809],{},[567,3859,3861],{"mathvariant":564,"lspace":3860,"rspace":3860},"0em","′",[567,3863,576],{"stretchy":569},[567,3865,576],{"stretchy":569},[567,3867,579],{},[604,3869,3870,3873],{},[562,3871,3872],{},"g",[562,3874,590],{},[567,3876,570],{"stretchy":569},[604,3878,3879,3881],{},[562,3880,1358],{},[562,3882,590],{},[567,3884,570],{"stretchy":569},[562,3886,1809],{},[567,3888,1353],{"separator":56},[1998,3890,3891,3893],{},[562,3892,1809],{},[567,3894,3861],{"mathvariant":564,"lspace":3860,"rspace":3860},[1998,3896,3897,3899],{},[567,3898,576],{"stretchy":569},[2012,3900,3384],{},[567,3902,576],{"stretchy":569},[48,3904,3905],{"encoding":50},"\\mathrm{CosSim}(\\phi_f(z), \\phi_f(z')) = g_f(d_f(z, z')^2)",[24,3907,3909,4070],{"className":3908,"ariaHidden":56},[55],[24,3910,3912,3916,3923,3926,3966,3969,3972,3975,3978,3981,4021,4024,4058,4061,4064,4067],{"className":3911},[60],[24,3913],{"className":3914,"style":3915},[64],"height:1.088em;vertical-align:-0.2861em;",[24,3917,3919],{"className":3918},[69],[24,3920,3922],{"className":3921},[69,3178],"CosSim",[24,3924,570],{"className":3925},[649],[24,3927,3929,3932],{"className":3928},[69],[24,3930,1983],{"className":3931},[69,653],[24,3933,3935],{"className":3934},[762],[24,3936,3938,3958],{"className":3937},[84,131],[24,3939,3941,3955],{"className":3940},[88],[24,3942,3944],{"className":3943,"style":772},[92],[24,3945,3946,3949],{"style":775},[24,3947],{"className":3948,"style":101},[100],[24,3950,3952],{"className":3951},[109,110,111,108],[24,3953,590],{"className":3954,"style":707},[69,653,108],[24,3956,157],{"className":3957},[156],[24,3959,3961],{"className":3960},[88],[24,3962,3964],{"className":3963,"style":794},[92],[24,3965],{},[24,3967,570],{"className":3968},[649],[24,3970,1809],{"className":3971,"style":1836},[69,653],[24,3973,576],{"className":3974},[657],[24,3976,1353],{"className":3977},[1423],[24,3979],{"className":3980,"style":752},[79],[24,3982,3984,3987],{"className":3983},[69],[24,3985,1983],{"className":3986},[69,653],[24,3988,3990],{"className":3989},[762],[24,3991,3993,4013],{"className":3992},[84,131],[24,3994,3996,4010],{"className":3995},[88],[24,3997,3999],{"className":3998,"style":772},[92],[24,4000,4001,4004],{"style":775},[24,4002],{"className":4003,"style":101},[100],[24,4005,4007],{"className":4006},[109,110,111,108],[24,4008,590],{"className":4009,"style":707},[69,653,108],[24,4011,157],{"className":4012},[156],[24,4014,4016],{"className":4015},[88],[24,4017,4019],{"className":4018,"style":794},[92],[24,4020],{},[24,4022,570],{"className":4023},[649],[24,4025,4027,4030],{"className":4026},[69],[24,4028,1809],{"className":4029,"style":1836},[69,653],[24,4031,4033],{"className":4032},[762],[24,4034,4036],{"className":4035},[84],[24,4037,4039],{"className":4038},[88],[24,4040,4043],{"className":4041,"style":4042},[92],"height:0.8019em;",[24,4044,4046,4049],{"style":4045},"top:-3.113em;margin-right:0.05em;",[24,4047],{"className":4048,"style":101},[100],[24,4050,4052],{"className":4051},[109,110,111,108],[24,4053,4055],{"className":4054},[69,108],[24,4056,3861],{"className":4057},[69,108],[24,4059,2397],{"className":4060},[657],[24,4062],{"className":4063,"style":661},[79],[24,4065,579],{"className":4066},[665],[24,4068],{"className":4069,"style":661},[79],[24,4071,4073,4077,4117,4120,4160,4163,4166,4169,4172,4204,4234],{"className":4072},[60],[24,4074],{"className":4075,"style":4076},[64],"height:1.1502em;vertical-align:-0.2861em;",[24,4078,4080,4083],{"className":4079},[69],[24,4081,3872],{"className":4082,"style":812},[69,653],[24,4084,4086],{"className":4085},[762],[24,4087,4089,4109],{"className":4088},[84,131],[24,4090,4092,4106],{"className":4091},[88],[24,4093,4095],{"className":4094,"style":772},[92],[24,4096,4097,4100],{"style":827},[24,4098],{"className":4099,"style":101},[100],[24,4101,4103],{"className":4102},[109,110,111,108],[24,4104,590],{"className":4105,"style":707},[69,653,108],[24,4107,157],{"className":4108},[156],[24,4110,4112],{"className":4111},[88],[24,4113,4115],{"className":4114,"style":794},[92],[24,4116],{},[24,4118,570],{"className":4119},[649],[24,4121,4123,4126],{"className":4122},[69],[24,4124,1358],{"className":4125},[69,653],[24,4127,4129],{"className":4128},[762],[24,4130,4132,4152],{"className":4131},[84,131],[24,4133,4135,4149],{"className":4134},[88],[24,4136,4138],{"className":4137,"style":772},[92],[24,4139,4140,4143],{"style":775},[24,4141],{"className":4142,"style":101},[100],[24,4144,4146],{"className":4145},[109,110,111,108],[24,4147,590],{"className":4148,"style":707},[69,653,108],[24,4150,157],{"className":4151},[156],[24,4153,4155],{"className":4154},[88],[24,4156,4158],{"className":4157,"style":794},[92],[24,4159],{},[24,4161,570],{"className":4162},[649],[24,4164,1809],{"className":4165,"style":1836},[69,653],[24,4167,1353],{"className":4168},[1423],[24,4170],{"className":4171,"style":752},[79],[24,4173,4175,4178],{"className":4174},[69],[24,4176,1809],{"className":4177,"style":1836},[69,653],[24,4179,4181],{"className":4180},[762],[24,4182,4184],{"className":4183},[84],[24,4185,4187],{"className":4186},[88],[24,4188,4190],{"className":4189,"style":4042},[92],[24,4191,4192,4195],{"style":4045},[24,4193],{"className":4194,"style":101},[100],[24,4196,4198],{"className":4197},[109,110,111,108],[24,4199,4201],{"className":4200},[69,108],[24,4202,3861],{"className":4203},[69,108],[24,4205,4207,4210],{"className":4206},[657],[24,4208,576],{"className":4209},[657],[24,4211,4213],{"className":4212},[762],[24,4214,4216],{"className":4215},[84],[24,4217,4219],{"className":4218},[88],[24,4220,4223],{"className":4221,"style":4222},[92],"height:0.8641em;",[24,4224,4225,4228],{"style":4045},[24,4226],{"className":4227,"style":101},[100],[24,4229,4231],{"className":4230},[109,110,111,108],[24,4232,3384],{"className":4233},[69,108],[24,4235,576],{"className":4236},[657],[11,4238,4239,4240,4310,4311,4441],{},"The only requirements are that ",[24,4241,4243,4261],{"className":4242,"translate":28},[27],[24,4244,4246],{"className":4245},[32],[34,4247,4248],{"xmlns":36},[38,4249,4250,4258],{},[41,4251,4252],{},[604,4253,4254,4256],{},[562,4255,3872],{},[562,4257,590],{},[48,4259,4260],{"encoding":50},"g_f",[24,4262,4264],{"className":4263,"ariaHidden":56},[55],[24,4265,4267,4270],{"className":4266},[60],[24,4268],{"className":4269,"style":3478},[64],[24,4271,4273,4276],{"className":4272},[69],[24,4274,3872],{"className":4275,"style":812},[69,653],[24,4277,4279],{"className":4278},[762],[24,4280,4282,4302],{"className":4281},[84,131],[24,4283,4285,4299],{"className":4284},[88],[24,4286,4288],{"className":4287,"style":772},[92],[24,4289,4290,4293],{"style":827},[24,4291],{"className":4292,"style":101},[100],[24,4294,4296],{"className":4295},[109,110,111,108],[24,4297,590],{"className":4298,"style":707},[69,653,108],[24,4300,157],{"className":4301},[156],[24,4303,4305],{"className":4304},[88],[24,4306,4308],{"className":4307,"style":794},[92],[24,4309],{}," be twice differentiable near 0 and satisfy ",[24,4312,4314,4346],{"className":4313,"translate":28},[27],[24,4315,4317],{"className":4316},[32],[34,4318,4319],{"xmlns":36},[38,4320,4321,4343],{},[41,4322,4323,4332,4334,4336,4338,4341],{},[4324,4325,4326,4328,4330],"msubsup",{},[562,4327,3872],{},[562,4329,590],{},[567,4331,3861],{"mathvariant":564,"lspace":3860,"rspace":3860},[567,4333,570],{"stretchy":569},[2012,4335,3226],{},[567,4337,576],{"stretchy":569},[567,4339,4340],{},"\u003C",[2012,4342,3226],{},[48,4344,4345],{"encoding":50},"g_f'(0) \u003C 0",[24,4347,4349,4431],{"className":4348,"ariaHidden":56},[55],[24,4350,4352,4356,4413,4416,4419,4422,4425,4428],{"className":4351},[60],[24,4353],{"className":4354,"style":4355},[64],"height:1.1711em;vertical-align:-0.4192em;",[24,4357,4359,4362],{"className":4358},[69],[24,4360,3872],{"className":4361,"style":812},[69,653],[24,4363,4365],{"className":4364},[762],[24,4366,4368,4404],{"className":4367},[84,131],[24,4369,4371,4401],{"className":4370},[88],[24,4372,4375,4387],{"className":4373,"style":4374},[92],"height:0.7519em;",[24,4376,4378,4381],{"style":4377},"top:-2.4169em;margin-left:-0.0359em;margin-right:0.05em;",[24,4379],{"className":4380,"style":101},[100],[24,4382,4384],{"className":4383},[109,110,111,108],[24,4385,590],{"className":4386,"style":707},[69,653,108],[24,4388,4389,4392],{"style":2159},[24,4390],{"className":4391,"style":101},[100],[24,4393,4395],{"className":4394},[109,110,111,108],[24,4396,4398],{"className":4397},[69,108],[24,4399,3861],{"className":4400},[69,108],[24,4402,157],{"className":4403},[156],[24,4405,4407],{"className":4406},[88],[24,4408,4411],{"className":4409,"style":4410},[92],"height:0.4192em;",[24,4412],{},[24,4414,570],{"className":4415},[649],[24,4417,3226],{"className":4418},[69],[24,4420,576],{"className":4421},[657],[24,4423],{"className":4424,"style":661},[79],[24,4426,4340],{"className":4427},[665],[24,4429],{"className":4430,"style":661},[79],[24,4432,4434,4438],{"className":4433},[60],[24,4435],{"className":4436,"style":4437},[64],"height:0.6444em;",[24,4439,3226],{"className":4440},[69],"; beyond these conditions, no further restrictions are imposed.",[11,4443,4444,4445,4448,4449,4479,4480,4549,4550,4581,4582,4651],{},"Under the joint validity of Hypotheses 1 and 2, the authors prove the paper's central result, ",[301,4446,4447],{},"Theorem 1",": Let ",[24,4450,4452,4467],{"className":4451,"translate":28},[27],[24,4453,4455],{"className":4454},[32],[34,4456,4457],{"xmlns":36},[38,4458,4459,4464],{},[41,4460,4461],{},[562,4462,4463],{},"η",[48,4465,4466],{"encoding":50},"\\eta",[24,4468,4470],{"className":4469,"ariaHidden":56},[55],[24,4471,4473,4476],{"className":4472},[60],[24,4474],{"className":4475,"style":3247},[64],[24,4477,4463],{"className":4478,"style":812},[69,653]," be a finite-length path in the feature space ",[24,4481,4483,4500],{"className":4482,"translate":28},[27],[24,4484,4486],{"className":4485},[32],[34,4487,4488],{"xmlns":36},[38,4489,4490,4498],{},[41,4491,4492],{},[604,4493,4494,4496],{},[562,4495,1348],{},[562,4497,590],{},[48,4499,1501],{"encoding":50},[24,4501,4503],{"className":4502,"ariaHidden":56},[55],[24,4504,4506,4509],{"className":4505},[60],[24,4507],{"className":4508,"style":1008},[64],[24,4510,4512,4515],{"className":4511},[69],[24,4513,1348],{"className":4514,"style":1384},[69,653],[24,4516,4518],{"className":4517},[762],[24,4519,4521,4541],{"className":4520},[84,131],[24,4522,4524,4538],{"className":4523},[88],[24,4525,4527],{"className":4526,"style":772},[92],[24,4528,4529,4532],{"style":1399},[24,4530],{"className":4531,"style":101},[100],[24,4533,4535],{"className":4534},[109,110,111,108],[24,4536,590],{"className":4537,"style":707},[69,653,108],[24,4539,157],{"className":4540},[156],[24,4542,4544],{"className":4543},[88],[24,4545,4547],{"className":4546,"style":794},[92],[24,4548],{},", and let ",[24,4551,4553,4568],{"className":4552,"translate":28},[27],[24,4554,4556],{"className":4555},[32],[34,4557,4558],{"xmlns":36},[38,4559,4560,4565],{},[41,4561,4562],{},[562,4563,4564],{},"γ",[48,4566,4567],{"encoding":50},"\\gamma",[24,4569,4571],{"className":4570,"ariaHidden":56},[55],[24,4572,4574,4577],{"className":4573},[60],[24,4575],{"className":4576,"style":3247},[64],[24,4578,4564],{"className":4579,"style":4580},[69,653],"margin-right:0.0556em;"," be its corresponding path on the manifold ",[24,4583,4585,4602],{"className":4584,"translate":28},[27],[24,4586,4588],{"className":4587},[32],[34,4589,4590],{"xmlns":36},[38,4591,4592,4600],{},[41,4593,4594],{},[604,4595,4596,4598],{},[562,4597,2561],{},[562,4599,590],{},[48,4601,2566],{"encoding":50},[24,4603,4605],{"className":4604,"ariaHidden":56},[55],[24,4606,4608,4611],{"className":4607},[60],[24,4609],{"className":4610,"style":1008},[64],[24,4612,4614,4617],{"className":4613},[69],[24,4615,2561],{"className":4616,"style":2582},[69,653],[24,4618,4620],{"className":4619},[762],[24,4621,4623,4643],{"className":4622},[84,131],[24,4624,4626,4640],{"className":4625},[88],[24,4627,4629],{"className":4628,"style":772},[92],[24,4630,4631,4634],{"style":2597},[24,4632],{"className":4633,"style":101},[100],[24,4635,4637],{"className":4636},[109,110,111,108],[24,4638,590],{"className":4639,"style":707},[69,653,108],[24,4641,157],{"className":4642},[156],[24,4644,4646],{"className":4645},[88],[24,4647,4649],{"className":4648,"style":794},[92],[24,4650],{},". Then",[24,4653,4655],{"className":4654,"translate":28},[547],[24,4656,4658,4714],{"className":4657,"translate":28},[27],[24,4659,4661],{"className":4660},[32],[34,4662,4663],{"xmlns":36,"display":556},[38,4664,4665,4711],{},[41,4666,4667,4669,4671,4673,4675,4677,4700,4703,4705,4707,4709],{},[562,4668,75],{},[567,4670,570],{"stretchy":569},[562,4672,4564],{},[567,4674,576],{"stretchy":569},[567,4676,579],{},[4678,4679,4680],"msqrt",{},[41,4681,4682,4684,4686,4694,4696,4698],{},[567,4683,2010],{},[2012,4685,3384],{},[4324,4687,4688,4690,4692],{},[562,4689,3872],{},[562,4691,590],{},[567,4693,3861],{"mathvariant":564,"lspace":3860,"rspace":3860},[567,4695,570],{"stretchy":569},[2012,4697,3226],{},[567,4699,576],{"stretchy":569},[567,4701,4702],{},"⋅",[562,4704,75],{},[567,4706,570],{"stretchy":569},[562,4708,4463],{},[567,4710,576],{"stretchy":569},[48,4712,4713],{"encoding":50},"L(\\gamma) = \\sqrt{-2g_f'(0)} \\cdot L(\\eta)",[24,4715,4717,4744,4895],{"className":4716,"ariaHidden":56},[55],[24,4718,4720,4723,4726,4729,4732,4735,4738,4741],{"className":4719},[60],[24,4721],{"className":4722,"style":642},[64],[24,4724,75],{"className":4725},[69,653],[24,4727,570],{"className":4728},[649],[24,4730,4564],{"className":4731,"style":4580},[69,653],[24,4733,576],{"className":4734},[657],[24,4736],{"className":4737,"style":661},[79],[24,4739,579],{"className":4740},[665],[24,4742],{"className":4743,"style":661},[79],[24,4745,4747,4751,4886,4889,4892],{"className":4746},[60],[24,4748],{"className":4749,"style":4750},[64],"height:1.84em;vertical-align:-0.6498em;",[24,4752,4755],{"className":4753},[69,4754],"sqrt",[24,4756,4758,4877],{"className":4757},[84,131],[24,4759,4761,4874],{"className":4760},[88],[24,4762,4765,4851],{"className":4763,"style":4764},[92],"height:1.1902em;",[24,4766,4770,4774],{"className":4767,"style":4769},[4768],"svg-align","top:-3.8em;",[24,4771],{"className":4772,"style":4773},[100],"height:3.8em;",[24,4775,4778,4781,4784,4842,4845,4848],{"className":4776,"style":4777},[69],"padding-left:1em;",[24,4779,2010],{"className":4780},[69],[24,4782,3384],{"className":4783},[69],[24,4785,4787,4790],{"className":4786},[69],[24,4788,3872],{"className":4789,"style":812},[69,653],[24,4791,4793],{"className":4792},[762],[24,4794,4796,4833],{"className":4795},[84,131],[24,4797,4799,4830],{"className":4798},[88],[24,4800,4803,4815],{"className":4801,"style":4802},[92],"height:0.7337em;",[24,4804,4806,4809],{"style":4805},"top:-2.3987em;margin-left:-0.0359em;margin-right:0.05em;",[24,4807],{"className":4808,"style":101},[100],[24,4810,4812],{"className":4811},[109,110,111,108],[24,4813,590],{"className":4814,"style":707},[69,653,108],[24,4816,4818,4821],{"style":4817},"top:-3.0448em;margin-right:0.05em;",[24,4819],{"className":4820,"style":101},[100],[24,4822,4824],{"className":4823},[109,110,111,108],[24,4825,4827],{"className":4826},[69,108],[24,4828,3861],{"className":4829},[69,108],[24,4831,157],{"className":4832},[156],[24,4834,4836],{"className":4835},[88],[24,4837,4840],{"className":4838,"style":4839},[92],"height:0.4374em;",[24,4841],{},[24,4843,570],{"className":4844},[649],[24,4846,3226],{"className":4847},[69],[24,4849,576],{"className":4850},[657],[24,4852,4854,4857],{"style":4853},"top:-3.1502em;",[24,4855],{"className":4856,"style":4773},[100],[24,4858,4862],{"className":4859,"style":4861},[4860],"hide-tail","min-width:1.02em;height:1.88em;",[4863,4864,4870],"svg",{"xmlns":4865,"width":4866,"height":4867,"viewBox":4868,"preserveAspectRatio":4869},"http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","400em","1.88em","0 0 400000 1944","xMinYMin slice",[4871,4872],"path",{"d":4873},"M983 90\nl0 -0\nc4,-6.7,10,-10,18,-10 H400000v40\nH1013.1s-83.4,268,-264.1,840c-180.7,572,-277,876.3,-289,913c-4.7,4.7,-12.7,7,-24,7\ns-12,0,-12,0c-1.3,-3.3,-3.7,-11.7,-7,-25c-35.3,-125.3,-106.7,-373.3,-214,-744\nc-10,12,-21,25,-33,39s-32,39,-32,39c-6,-5.3,-15,-14,-27,-26s25,-30,25,-30\nc26.7,-32.7,52,-63,76,-91s52,-60,52,-60s208,722,208,722\nc56,-175.3,126.3,-397.3,211,-666c84.7,-268.7,153.8,-488.2,207.5,-658.5\nc53.7,-170.3,84.5,-266.8,92.5,-289.5z\nM1001 80h400000v40h-400000z",[24,4875,157],{"className":4876},[156],[24,4878,4880],{"className":4879},[88],[24,4881,4884],{"className":4882,"style":4883},[92],"height:0.6498em;",[24,4885],{},[24,4887],{"className":4888,"style":1015},[79],[24,4890,4702],{"className":4891},[2176],[24,4893],{"className":4894,"style":1015},[79],[24,4896,4898,4901,4904,4907,4910],{"className":4897},[60],[24,4899],{"className":4900,"style":642},[64],[24,4902,75],{"className":4903},[69,653],[24,4905,570],{"className":4906},[649],[24,4908,4463],{"className":4909,"style":812},[69,653],[24,4911,576],{"className":4912},[657],[11,4914,4915,4916,4919,4920,4989],{},"In plain language: ",[301,4917,4918],{},"the length of the shortest path (geodesic) along the manifold is strictly proportional to the length of the corresponding shortest path in the feature space."," That is, even without knowing the exact form of ",[24,4921,4923,4940],{"className":4922,"translate":28},[27],[24,4924,4926],{"className":4925},[32],[34,4927,4928],{"xmlns":36},[38,4929,4930,4938],{},[41,4931,4932],{},[604,4933,4934,4936],{},[562,4935,3872],{},[562,4937,590],{},[48,4939,4260],{"encoding":50},[24,4941,4943],{"className":4942,"ariaHidden":56},[55],[24,4944,4946,4949],{"className":4945},[60],[24,4947],{"className":4948,"style":3478},[64],[24,4950,4952,4955],{"className":4951},[69],[24,4953,3872],{"className":4954,"style":812},[69,653],[24,4956,4958],{"className":4957},[762],[24,4959,4961,4981],{"className":4960},[84,131],[24,4962,4964,4978],{"className":4963},[88],[24,4965,4967],{"className":4966,"style":772},[92],[24,4968,4969,4972],{"style":827},[24,4970],{"className":4971,"style":101},[100],[24,4973,4975],{"className":4974},[109,110,111,108],[24,4976,590],{"className":4977,"style":707},[69,653,108],[24,4979,157],{"className":4980},[156],[24,4982,4984],{"className":4983},[88],[24,4985,4987],{"className":4986,"style":794},[92],[24,4988],{},", so long as the local cosine similarity is some smooth, decreasing function of the squared distance, the \"geodesic distance\" along the representation manifold—traveling along the manifold itself—recovers the true distance in concept space, exactly, up to a single uniform scaling constant.",[11,4991,4992],{},"This result addresses a hanging conjecture posed by Olah and Batson in 2024: they had written that \"the idea that feature manifolds are embedded in more complex ways than their topology alone would require, possibly in order to realize a specific distance metric, may be quite deep and important.\" This paper, in effect, uses the rigorous language of metric geometry to crystallize that intuitive conjecture into a theorem that can be both proved and falsified.",[11,4994,4995],{},"The proof itself draws on the classical definition of \"path length\" in metric geometry (the supremum of polygonal-approximation sums), with the central technique being a Taylor expansion of cosine similarity near zero, followed by a series of careful triangle-inequality arguments to bound the error terms—a solid piece of analytic reasoning (see the appendix for details).",[224,4997,4999],{"id":4998},"homeomorphism-is-easy-isometry-is-hard","Homeomorphism Is Easy; Isometry Is Hard",[11,5001,5002],{},"Theory established, the authors return to the three empirical case studies—colors, years, and dates—to test Hypothesis 2 and Theorem 1. They design two diagnostic approaches:",[3054,5004,5005,5043],{},[1475,5006,5007,5010,5011,5042],{},[301,5008,5009],{},"Direct method",": plot \"cosine similarity vs. squared distance\" as a scatter plot to see whether a locally decreasing trend emerges, using Chatterjee's correlation coefficient ",[24,5012,5014,5029],{"className":5013,"translate":28},[27],[24,5015,5017],{"className":5016},[32],[34,5018,5019],{"xmlns":36},[38,5020,5021,5026],{},[41,5022,5023],{},[562,5024,5025],{},"ξ",[48,5027,5028],{"encoding":50},"\\xi",[24,5030,5032],{"className":5031,"ariaHidden":56},[55],[24,5033,5035,5038],{"className":5034},[60],[24,5036],{"className":5037,"style":1238},[64],[24,5039,5025],{"className":5040,"style":5041},[69,653],"margin-right:0.046em;"," to quantify the overall degree of functional dependence;",[1475,5044,5045,5048],{},[301,5046,5047],{},"Indirect method",": test Theorem 1 itself—estimate geodesic distances along the manifold using a k-nearest-neighbor graph, check whether they are proportional to geodesic distances in the feature space, and use the Pearson correlation coefficient to assess linearity.",[11,5050,5051,5052,5055,5056,5119],{},"The results are intriguing. For \"colors\" and \"dates\"—both of which carry a natural periodic structure—a simple circular metric (hue angle; day of the year) yields reasonably strong support for isometry (dates achieve a Pearson correlation of 0.97). ",[301,5053,5054],{},"\"Years,\" however, stumbles:"," if one assumes that the distance between years is simply ",[24,5057,5059,5083],{"className":5058,"translate":28},[27],[24,5060,5062],{"className":5061},[32],[34,5063,5064],{"xmlns":36},[38,5065,5066,5080],{},[41,5067,5068,5071,5073,5075,5078],{},[562,5069,5070],{"mathvariant":564},"∣",[562,5072,573],{},[567,5074,2010],{},[562,5076,5077],{},"y",[562,5079,5070],{"mathvariant":564},[48,5081,5082],{"encoding":50},"|x-y|",[24,5084,5086,5107],{"className":5085,"ariaHidden":56},[55],[24,5087,5089,5092,5095,5098,5101,5104],{"className":5088},[60],[24,5090],{"className":5091,"style":642},[64],[24,5093,5070],{"className":5094},[69],[24,5096,573],{"className":5097},[69,653],[24,5099],{"className":5100,"style":1015},[79],[24,5102,2010],{"className":5103},[2176],[24,5105],{"className":5106,"style":1015},[79],[24,5108,5110,5113,5116],{"className":5109},[60],[24,5111],{"className":5112,"style":642},[64],[24,5114,5077],{"className":5115,"style":812},[69,653],[24,5117,5070],{"className":5118},[69]," (e.g., 1990 and 2000 are 10 units apart), the empirical data do not support this hypothesis at all. Figure 4 reveals that years closer to \"the present\" (the paper takes GPT-2's release year of 2019 as the reference point) are stretched further apart on the manifold—their distances are \"magnified.\"",[11,5121,5122,5123,5202,5203,5206],{},"The authors therefore make a very elegant correction: they replace the metric space of years with Euclidean distance on ",[24,5124,5126,5156],{"className":5125,"translate":28},[27],[24,5127,5129],{"className":5128},[32],[34,5130,5131],{"xmlns":36},[38,5132,5133,5153],{},[41,5134,5135,5138,5141,5143,5146,5148,5151],{},[562,5136,5137],{},"log",[567,5139,5140],{},"⁡",[567,5142,570],{"stretchy":569},[2012,5144,5145],{},"2019",[567,5147,2010],{},[44,5149,5150],{},"year",[567,5152,576],{"stretchy":569},[48,5154,5155],{"encoding":50},"\\log(2019 - \\text{year})",[24,5157,5159,5187],{"className":5158,"ariaHidden":56},[55],[24,5160,5162,5165,5172,5175,5178,5181,5184],{"className":5161},[60],[24,5163],{"className":5164,"style":642},[64],[24,5166,5168,5169],{"className":5167},[679],"lo",[24,5170,3872],{"style":5171},"margin-right:0.0139em;",[24,5173,570],{"className":5174},[649],[24,5176,5145],{"className":5177},[69],[24,5179],{"className":5180,"style":1015},[79],[24,5182,2010],{"className":5183},[2176],[24,5185],{"className":5186,"style":1015},[79],[24,5188,5190,5193,5199],{"className":5189},[60],[24,5191],{"className":5192,"style":642},[64],[24,5194,5196],{"className":5195},[69,70],[24,5197,5150],{"className":5198},[69],[24,5200,576],{"className":5201},[657],"—that is, what the model may be encoding is not \"calendar year\" per se, but ",[301,5204,5205],{},"\"how long ago\" on a logarithmic scale",". This new metric space is topologically equivalent to the original year interval (homeomorphism is preserved, with rank correlations still near 1), but when isometry is tested under this new metric, the results flip from \"unsupported\" to \"strongly supported\" (Chatterjee coefficient 0.84, Pearson correlation 0.99).",[11,5208,5209,5210,5213],{},"This section is, I think, the most insightful empirical part of the entire paper: ",[301,5211,5212],{},"it demonstrates with clarity that \"homeomorphism\" and \"isometry\" are two entirely different levels of geometric fidelity."," The former asks only \"is the shape right?\"; the latter asks \"are the numerical distance relations right?\" A manifold can be topologically perfectly consistent with the concept space you envision, yet geometrically (in the specific functional form of the distance) be entirely different. And the process of deciphering \"how exactly is distance encoded\" is itself an act of reconnaissance into how the model \"understands\" the concept of time. That GPT-2 encodes \"more recent years\" as further apart from one another hints, in a certain sense, that years closer to the model's training cutoff may carry denser semantic texture—news and events are more densely packed—and are therefore allotted a larger \"representational space budget.\" This is a rather enchanting conjecture, though the paper itself does not delve deeply into a causal explanation for this pattern.",[224,5215,5217],{"id":5216},"the-platonic-representation-hypothesis","The Platonic Representation Hypothesis",[11,5219,5220,5221,5224],{},"While reading this paper, I found myself unable to resist drawing comparisons to another position paper, also from 2024, that provoked substantial discussion in the machine learning community—the ",[301,5222,5223],{},"Platonic Representation Hypothesis (PRH)"," proposed by Huh, Cheung, Wang, and Isola (ICML 2024). Although the two papers differ in their specific objects of study and methodology almost entirely, reading them side by side reveals that they operate at two different scales on the same overarching question, and that they are in fact complementary—arguably even two successive links in a causal chain.",[11,5226,5227,5228,5231,5232,5235],{},"The core claim of PRH is this: deep networks with different architectures, different modalities (vision, language), and different training objectives tend, as they scale up in size and task diversity, to ",[301,5229,5230],{},"converge"," their internal representations toward a shared geometric structure. The authors liken this hypothesized representation space—progressively approximated by more and more models—to the Platonic \"ideal reality\" (the realm of Forms) that exists independently of and transcends any particular concrete entity: the representation learned by each specific model is merely a noisy,biased projection of this \"Platonic representation.\" Their empirical evidence includes the finding that, when processing paired image-text data, the way image models and language models measure the distances between data points becomes increasingly similar as the models grow larger; and that across an expanding array of vision models with different architectures and training regimes, the pairwise representational similarity systematically rises. They further advance an explanatory conjecture: ",[301,5233,5234],{},"the driving force behind this convergence is that models, in the course of learning, are compelled to approximate the common, truth statistical structure of the data-generating process (\"reality\" itself)","—and the more diverse the tasks, the stronger this convergence pressure becomes.",[11,5237,5238,5239,5242,5243],{},"If PRH is concerned with \"",[301,5240,5241],{},"across different models",", whether the representation space as a whole is converging toward a single geometry,\" then the paper by Modell, Rubin-Delanchy, and Whiteley is concerned with a smaller and more specific scale: ",[301,5244,5245],{},"within a single model, why does the submanifold corresponding to a single feature (color, year, date) exhibit a particular geometric shape, and what is the precise mathematical relationship between this shape and the \"concept\" it corresponds to?",[11,5247,5248],{},"Taken together, the two papers can be read as tracing a rather coherent logical chain:",[3054,5250,5251,5257,5267],{},[1475,5252,5253,5254],{},"PRH advances a macroscopic conjecture—that the representation spaces of different models are converging toward a common geometry that reflects the \"true statistical structure of the world,\" and that the degree of this convergence can be quantified by the \"consistency in how models measure distances between data points\" (the PRH paper uses precisely tools such as kernel alignment and representational similarity, which assess the consistency of pairwise-distance patterns). But PRH itself remains at the level of \"existence\" and \"convergence trends\"; it does not delve deeply into a more foundational question: ",[301,5255,5256],{},"how, within the representation space of a single model, does distance itself correspond to the \"true\" distance in concept space?",[1475,5258,5259,5260,5263,5264],{},"The present paper, ",[494,5261,5262],{},"The Origins of Representation Manifolds",", answers precisely this more foundational, more microscopic question. Theorem 1 tells us: so long as there exists a smooth functional relationship between local cosine similarity and feature distance (Hypothesis 2), the geodesic distance along the representation manifold will recover the \"true\" distance in concept space, exactly, up to a single proportionality constant. This supplies, for PRH's claim that \"models learn to measure distances between data points in some consistent way,\" an explanation at the level of geometric mechanism: ",[301,5265,5266],{},"the model does not simply \"memorize\" distances; rather, by mapping features onto a manifold of a particular shape and curvature, it implicitly encodes the distance structure through the geodesic paths on that manifold.",[1475,5268,5269,5270],{},"Conversely, the circular structures observed for colors and dates in this paper—and the specific finding that \"colors are arranged in the hue order of the standard color wheel\"—can be seen as a microscopic corroboration of PRH: hue itself is a periodic structure that exists objectively in the physical world (the continuous spectrum of visible-light wavelengths, together with the response curves of the three types of cone cells in the human retina, jointly determining the color space). If different image models and different language models, independently and without coordination, all learn that \"hue is a circle,\" this is precisely an instance—concrete, isolatable, and testable—of the PRH phenomenon that different models converge to a shared representation reflecting the true statistical structure of the world. In other words, ",[301,5271,5272],{},"PRH provides the macroscopic narrative of \"why convergence happens\"; this paper provides the microscopic characterization of \"what the geometry looks like after convergence, and how this geometry precisely encodes semantic distance.\"",[11,5274,5275,5276,5279,5280,5282,5283,5286,5287,5290],{},"Of course, there exists an evident tension between the two works, and it is worth acknowledging honestly. The convergence claim of PRH rests, to a large extent, on ",[301,5277,5278],{},"cross-model comparisons","—using tools such as canonical correlation analysis and kernel alignment to compare how different models measure the pairwise-distance patterns of the same set of data points, which belongs to the statistics of \"relations between representations.\" ",[494,5281,5262],{},", by contrast, discusses from start to finish the geometric structure ",[301,5284,5285],{},"within a single model",", within the subspace of a single feature, and barely touches on the question of whether representations from different models are comparable to one another. That is to say, even if the theorem relating cosine similarity to geodesic distance holds for a particular model (e.g., GPT-2), it does not directly imply that another model with a completely different architecture would encode the feature \"year\" in the same way—with the same metric space, the same logarithmic scale. Indeed, the specific finding that \"years are encoded on a logarithmic scale, with distances increasingly stretched for years closer to the reference year\" carries, in itself, a considerable degree of ",[301,5288,5289],{},"model-specificity",". That the reference point for GPT-2 is naturally set as its own release year (2019) suggests that this encoding scheme is likely strongly tied to the temporal distribution of this particular model's training corpus, rather than being a universal, \"Platonic\" distance encoding to which all language models converge. If one were to repeat the same experiment on a model with a different training cutoff and a different corpus distribution, would the logarithmic scale and the reference point shift accordingly? This is, in fact, a question eminently worthy of direct investigation in follow-up work—and in a certain sense, it constitutes a 天然 experimental design for subjecting PRH's convergence claim to an empirical test at the finer granularity of \"the specific geometric parameters of a single feature.\"",[11,5292,5293,5294,5297,5298,5300],{},"In my assessment, reading these two papers together is more illuminating than reading either alone: ",[301,5295,5296],{},"PRH tells us that \"everyone is converging toward the same direction,\" while this paper supplies the mathematical language for characterizing what exactly that convergence converges to, and how such convergence can be rigorously measured and falsified."," If, in the future, someone wishes to genuinely test whether PRH holds at the level of specific features—for instance, whether the circular structure of hue or the logarithmic encoding of time reappear consistently across models with different architectures and training data—then the homeomorphism tests, the isometry tests (Chatterjee coefficient, Pearson correlation of KNN-based geodesic distances) proposed in ",[494,5299,5262],{}," constitute an almost ready-made, standardized toolbox. This may well be this paper's most broadly reusable \"by-product,\" beyond its own mathematical results.",[224,5302,5304],{"id":5303},"concluding-remarks","Concluding Remarks",[11,5306,5307],{},"Having read the paper, I would like to offer my assessment from several angles.",[11,5309,5310,5311,5314],{},"First, this is a rare effort to ",[301,5312,5313],{},"rigorize a vague intuition",", and its primary value lies in providing language and tools rather than a definitive answer. The field of mechanistic interpretability presently abounds in discoveries that amount to \"looking at pictures and describing them\"—\"oh, this feature looks like a circle,\" \"this one looks like a tree structure\"—but lacks a unified mathematical language with which to organize these observations. Defining features as metric spaces, and cleanly separating the notions of \"homeomorphism\" and \"isometry,\" is itself an important conceptual tool: it forces researchers to translate the vague intuition of \"I think this manifold corresponds to that concept\" into concrete, falsifiable hypotheses (\"is this the right metric space?\"), and provides specific statistical testing procedures (Chatterjee coefficient, geodesic distances via k-nearest-neighbor graphs). This kind of rigor is currently in relatively short supply in the field.",[11,5316,5317],{},"Second, Theorem 1 is elegant mathematics, but its \"probative force\" depends, to some degree, on a rather strong premise—that a smooth functional relationship exists between cosine similarity and distance. This hypothesis itself is not derived from model training dynamics or architectural design considerations, but is instead placed on the table as a \"reasonable guess\" and then verified empirically. In other words, the paper reads more like \"if this hypothesis holds, what beautiful corollaries follow?\" than \"why the representation space necessarily exhibits this structure\" (this is the sense in which the word \"Origins\" in the title is perhaps a touch wishful—the paper does not actually derive, from optimization objectives or the dynamics of gradient descent, why manifolds \"emerge\"; it largely operates under the premise that they already exist and proceeds to characterize what they should look like). This is not entirely a criticism, however: mechanistic interpretability as a whole is still at the stage of \"observing phenomena and constructing descriptive theories,\" and very few works genuinely derive representation geometry from first principles. This paper at least executes the \"descriptive theory\" step with greater rigor than most comparable efforts.",[11,5319,5320,5321,5324],{},"Third, the logarithmic-scale discovery for years is the single most impressive—and most thought-provoking—empirical result in the entire paper. ",[301,5322,5323],{},"It hints at a methodological trap worth guarding against: the homeomorphism test (is the topology right?) has a very low bar; it is easy to get something that \"looks right,\" but this is far from sufficient to demonstrate that we genuinely understand the model's encoding scheme."," Had the authors stopped at \"years are arranged along a curve whose order matches the true chronological order, with a rank correlation of 0.97, indicating that the model has learned the ordering of years,\" this conclusion would be rather a hole—virtually any monotonic mapping could achieve this. What carries genuine information is the non-uniform stretching of distances revealed by the isometry test, and the detailed hypothesis behind it: encoding \"time since the reference point\" on a logarithmic scale. This reminds us that future interpretability research of a similar kind should not stop at \"does the shape look similar?\"; rather, it should, as this paper does, press further to ask \"is the distance metric correct?\"—for only then can we excavate the implicit assumptions that the model has truly encoded.",[11,5326,5327,5328,5397],{},"Fourth, from an applied perspective, this paper carries direct methodological implications for steering research, but remains some distance from a truly operational tool. The paper concludes by noting that, if one could learn ",[24,5329,5331,5348],{"className":5330,"translate":28},[27],[24,5332,5334],{"className":5333},[32],[34,5335,5336],{"xmlns":36},[38,5337,5338,5346],{},[41,5339,5340],{},[604,5341,5342,5344],{},[562,5343,1983],{},[562,5345,590],{},[48,5347,2493],{"encoding":50},[24,5349,5351],{"className":5350,"ariaHidden":56},[55],[24,5352,5354,5357],{"className":5353},[60],[24,5355],{"className":5356,"style":1658},[64],[24,5358,5360,5363],{"className":5359},[69],[24,5361,1983],{"className":5362},[69,653],[24,5364,5366],{"className":5365},[762],[24,5367,5369,5389],{"className":5368},[84,131],[24,5370,5372,5386],{"className":5371},[88],[24,5373,5375],{"className":5374,"style":772},[92],[24,5376,5377,5380],{"style":775},[24,5378],{"className":5379,"style":101},[100],[24,5381,5383],{"className":5382},[109,110,111,108],[24,5384,590],{"className":5385,"style":707},[69,653,108],[24,5387,157],{"className":5388},[156],[24,5390,5392],{"className":5391},[88],[24,5393,5395],{"className":5394,"style":794},[92],[24,5396],{}," (the map from features to the manifold), one could in principle perform more refined representation editing that respects the intrinsic geometric structure of the concept—for example, to \"shift\" a date feature forward by half a year, one should travel along the manifold's geodesic rather than simply adding a vector in Euclidean space. This is a promising direction, and it echoes the authors' call for \"manifold-aware SAEs.\" At present, however, this remains at the conceptual level: how to robustly estimate a high-dimensional, noisy manifold is still an unsolved statistical problem, as the authors themselves frankly acknowledge in the limitations section.",[11,5399,5400,5401,5404],{},"Fifth, a contribution that may be underappreciated is the paper's implications for text embedding services. Many practitioners working on RAG, semantic search, and recommendation systems unthinkingly use cosine similarity as a measure of \"semantic proximity,\" rarely pausing to ask what geometric structure actually lies beneath that numerical similarity score. This paper serves as a reminder: ",[301,5402,5403],{},"cosine similarity can indeed faithfully reflect distances \"along the intrinsic geometric paths of concept space\"—but this is a local property, contingent on Hypothesis 2 holding, and the geometric shape differs entirely from one feature to another (circle, line segment, logarithmic line segment)."," Treating it as a universal, cross-feature \"semantic distance\" metric may obscure a great deal of important nonlinear structure (as the color example illustrates, where \"cosine similarity at large distances manifestly deviates from an isometric relationship\"). This is a caution worth lodging in the minds of engineers tuning embedding-based retrieval systems and anomaly detection pipelines.",[11,5406,5407,5408,5411],{},"This paper does not attempt to explain all representational-geometric phenomena, nor does it deliver a tool ready for production deployment. It simply does \"a mathematician's job\": ",[301,5409,5410],{},"for a phenomenon that has been widely observed yet lacks precise definition, erect a minimal but rigorous theoretical framework, and honestly take it to the data to test where its boundaries lie."," In a field that increasingly relies on \"alchemical,\" empirically driven observation, this kind of work—returning to first principles, willing to state its assumptions clearly, write out its proofs completely, and honestly its limitations openly—is intrinsically worth being seen by more people.",[11,5413,5414,5415],{},"If I had to name the single deepest impression this paper left on me, it would probably be the specific finding that \"years are encoded as logarithmic time-distance.\" Through one minimal example, it shows that behind the seemingly plain geometry of vector spaces, there may truly reside something akin to a human cognitive intuition: recent events are sharper, more finely discriminated; distant events are compressed in memory. This is perhaps the most captivating thing about mechanistic interpretability research: ",[301,5416,5417],{},"not proving that the model is a black-box piece of magic, but, little by little, translating what it is truly \"thinking\" on the inside into a language we can read.",{"title":195,"searchDepth":196,"depth":196,"links":5419},[5420,5421,5422,5423,5424,5425,5426,5427],{"id":522,"depth":196,"text":523},{"id":1315,"depth":196,"text":1316},{"id":3048,"depth":196,"text":3049},{"id":3090,"depth":196,"text":3091},{"id":3785,"depth":196,"text":3786},{"id":4998,"depth":196,"text":4999},{"id":5216,"depth":196,"text":5217},{"id":5303,"depth":196,"text":5304},"If you have followed the progress in mechanistic interpretability over the past two years, you have likely encountered a certain kind of figure: when the activation vectors of a particular layer in a large language model are projected down to two or three dimensions and plotted, the representations of concepts such as \"months,\" \"days of the week,\" \"years,\" and \"colors\" do not scatter chaotically through space, but instead arrange themselves along an elegant curve—sometimes even a closed circle or a torus. Researchers such as Chris Olah, Josh Batson at Anthropic, and Engels et al. have all demonstrated this phenomenon in works including Not All Language Model Features Are One-Dimensionally Linear.",{},"2026-08-04","\u002Fblog\u002F2026\u002F2026-08-04-the-representations-in-llm",{"title":487,"description":5428},"blog\u002F2026\u002F2026-08-04-the-representations-in-llm","Formalizes LLM representation manifolds by defining features as metric spaces, proving cosine similarity encodes geodesic feature distance, then validating homeomorphism and isometry empirically on colors, dates, and years.",[5436,5437],"machine-learning","ai","XOOXSccYekvL-iDyMBWQV5yyBGYpJ28WpGhxDSsSo00",1786880520433]