[{"data":1,"prerenderedAt":10286},["ShallowReactive",2],{"post-\u002Fblog\u002F2026\u002F2026-06-30-about-the-emergency":3},{"post":4,"nextPost":1203,"prevPost":1524},{"id":5,"title":6,"body":7,"description":1169,"draft":1188,"enableComment":1189,"extension":1190,"image":1169,"important":1188,"location":1191,"meta":1192,"navigation":1189,"ogImage":1191,"onday":1193,"path":1194,"seo":1195,"stem":1196,"summary":1197,"tags":1198,"__hash__":1202},"blog\u002Fblog\u002F2026\u002F2026-06-30-about-the-emergency.md","Is Emergence a Mirage?",{"type":8,"value":9,"toc":1168},"minimark",[10,15,30,37,41,324,502,664,672,676,683,688,891,895,1001,1005,1016,1020,1031,1042,1049,1064,1068,1071,1075,1086,1090,1111,1115,1122,1126,1129,1139,1145,1151,1154,1158,1161],[11,12,14],"h2",{"id":13},"the-emergence","The Emergence",[16,17,18,19,29],"p",{},"The claim that \"emergence\" is a pseudo-concept — that apparent qualitative transitions in complex systems are artifacts of nonlinear evaluation metrics applied to linear or smoothly varying underlying processes — has gained considerable traction since Schaeffer, Miranda, and Koyejo's 2023 paper, ",[20,21,22],"em",{},[23,24,28],"a",{"href":25,"rel":26},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.15004",[27],"nofollow","Are Emergent Abilities of Large Language Models a Mirage?"," The argument is seductive in its simplicity: choose a discontinuous metric, and you will manufacture a discontinuity where none exists in the underlying data-generating process. This essay examines the strength and the limits of that argument, situates it within the broader literature on phase transitions in physical and biological systems, and asks a more specific question: what does contemporary machine learning theory — scaling laws, loss landscape geometry, grokking, and mechanistic interpretability — actually tell us about whether emergence in trained neural networks is measurement artifact, genuine dynamical phase transition, or something that resists this binary altogether?",[16,31,32,33,36],{},"My conclusion, argued in detail below, is that the \"mirage\" critique is correct as a ",[20,34,35],{},"local"," methodological claim about a specific class of benchmark-reported emergent abilities, but it does not generalize to a categorical dismissal of emergence as a scientific concept. Machine learning research over the past three years has independently accumulated evidence — from training-loss phase transitions, from grokking dynamics, and from circuit-level interpretability — that some discontinuities in neural network behavior are properties of the optimization dynamics themselves, invariant to the choice of readout metric. Distinguishing the two cases requires a specific methodological discipline that this essay tries to make explicit.",[11,38,40],{"id":39},"the-mirage-argument-formalized","The Mirage Argument, Formalized",[16,42,43,44,91,92,121,122,153,154,319,320,323],{},"Schaeffer et al.'s core technical claim rests on a simple probabilistic decomposition. Consider a multi-step task requiring ",[45,46,50,72],"span",{"className":47,"translate":49},[48],"katex","no",[45,51,54],{"className":52},[53],"katex-mathml",[55,56,58],"math",{"xmlns":57},"http:\u002F\u002Fwww.w3.org\u002F1998\u002FMath\u002FMathML",[59,60,61,68],"semantics",{},[62,63,64],"mrow",{},[65,66,67],"mi",{},"n",[69,70,67],"annotation",{"encoding":71},"application\u002Fx-tex",[45,73,77],{"className":74,"ariaHidden":76},[75],"katex-html","true",[45,78,81,86],{"className":79},[80],"base",[45,82],{"className":83,"style":85},[84],"strut","height:0.4306em;",[45,87,67],{"className":88},[89,90],"mord","mathnormal"," independent subcomponents to each be correct — for instance, a multi-digit arithmetic problem, or a chain-of-thought derivation with several intermediate steps. If the model's per-step accuracy ",[45,93,95,108],{"className":94,"translate":49},[48],[45,96,98],{"className":97},[53],[55,99,100],{"xmlns":57},[59,101,102,106],{},[62,103,104],{},[65,105,16],{},[69,107,16],{"encoding":71},[45,109,111],{"className":110,"ariaHidden":76},[75],[45,112,114,118],{"className":113},[80],[45,115],{"className":116,"style":117},[84],"height:0.625em;vertical-align:-0.1944em;",[45,119,16],{"className":120},[89,90]," improves smoothly and continuously with parameter count ",[45,123,125,139],{"className":124,"translate":49},[48],[45,126,128],{"className":127},[53],[55,129,130],{"xmlns":57},[59,131,132,137],{},[62,133,134],{},[65,135,136],{},"N",[69,138,136],{"encoding":71},[45,140,142],{"className":141,"ariaHidden":76},[75],[45,143,145,149],{"className":144},[80],[45,146],{"className":147,"style":148},[84],"height:0.6833em;",[45,150,136],{"className":151,"style":152},[89,90],"margin-right:0.109em;"," (say, following a power law derived from standard scaling-law theory, ",[45,155,157,205],{"className":156,"translate":49},[48],[45,158,160],{"className":159},[53],[55,161,162],{"xmlns":57},[59,163,164,202],{},[62,165,166,168,173,175,178,181,185,188,190],{},[65,167,16],{},[169,170,172],"mo",{"stretchy":171},"false","(",[65,174,136],{},[169,176,177],{"stretchy":171},")",[169,179,180],{},"≈",[182,183,184],"mn",{},"1",[169,186,187],{},"−",[65,189,23],{},[191,192,193,195],"msup",{},[65,194,136],{},[62,196,197,199],{},[169,198,187],{},[65,200,201],{},"α",[69,203,204],{"encoding":71},"p(N) \\approx 1 - aN^{-\\alpha}",[45,206,208,241,262],{"className":207,"ariaHidden":76},[75],[45,209,211,215,218,222,225,229,234,238],{"className":210},[80],[45,212],{"className":213,"style":214},[84],"height:1em;vertical-align:-0.25em;",[45,216,16],{"className":217},[89,90],[45,219,172],{"className":220},[221],"mopen",[45,223,136],{"className":224,"style":152},[89,90],[45,226,177],{"className":227},[228],"mclose",[45,230],{"className":231,"style":233},[232],"mspace","margin-right:0.2778em;",[45,235,180],{"className":236},[237],"mrel",[45,239],{"className":240,"style":233},[232],[45,242,244,248,251,255,259],{"className":243},[80],[45,245],{"className":246,"style":247},[84],"height:0.7278em;vertical-align:-0.0833em;",[45,249,184],{"className":250},[89],[45,252],{"className":253,"style":254},[232],"margin-right:0.2222em;",[45,256,187],{"className":257},[258],"mbin",[45,260],{"className":261,"style":254},[232],[45,263,265,269,272],{"className":264},[80],[45,266],{"className":267,"style":268},[84],"height:0.7713em;",[45,270,23],{"className":271},[89,90],[45,273,275,278],{"className":274},[89],[45,276,136],{"className":277,"style":152},[89,90],[45,279,282],{"className":280},[281],"msupsub",[45,283,286],{"className":284},[285],"vlist-t",[45,287,290],{"className":288},[289],"vlist-r",[45,291,294],{"className":292,"style":268},[293],"vlist",[45,295,297,302],{"style":296},"top:-3.063em;margin-right:0.05em;",[45,298],{"className":299,"style":301},[300],"pstrut","height:2.7em;",[45,303,309],{"className":304},[305,306,307,308],"sizing","reset-size6","size3","mtight",[45,310,312,315],{"className":311},[89,308],[45,313,187],{"className":314},[89,308],[45,316,201],{"className":317,"style":318},[89,90,308],"margin-right:0.0037em;","), then the task-level accuracy under an ",[20,321,322],{},"exact-match"," metric is:",[45,325,328],{"className":326,"translate":49},[327],"katex-display",[45,329,331,375],{"className":330,"translate":49},[48],[45,332,334],{"className":333},[53],[55,335,337],{"xmlns":57,"display":336},"block",[59,338,339,372],{},[62,340,341,351,353,355,357,360,362,364,366],{},[342,343,344,347],"msub",{},[65,345,346],{},"P",[348,349,350],"mtext",{},"task",[169,352,172],{"stretchy":171},[65,354,136],{},[169,356,177],{"stretchy":171},[169,358,359],{},"=",[65,361,16],{},[169,363,172],{"stretchy":171},[65,365,136],{},[191,367,368,370],{},[169,369,177],{"stretchy":171},[65,371,67],{},[69,373,374],{"encoding":71},"P_{\\text{task}}(N) = p(N)^n",[45,376,378,456],{"className":377,"ariaHidden":76},[75],[45,379,381,384,438,441,444,447,450,453],{"className":380},[80],[45,382],{"className":383,"style":214},[84],[45,385,387,391],{"className":386},[89],[45,388,346],{"className":389,"style":390},[89,90],"margin-right:0.1389em;",[45,392,394],{"className":393},[281],[45,395,398,429],{"className":396},[285,397],"vlist-t2",[45,399,401,424],{"className":400},[289],[45,402,405],{"className":403,"style":404},[293],"height:0.3361em;",[45,406,408,411],{"style":407},"top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;",[45,409],{"className":410,"style":301},[300],[45,412,414],{"className":413},[305,306,307,308],[45,415,417],{"className":416},[89,308],[45,418,421],{"className":419},[89,420,308],"text",[45,422,350],{"className":423},[89,308],[45,425,428],{"className":426},[427],"vlist-s","​",[45,430,432],{"className":431},[289],[45,433,436],{"className":434,"style":435},[293],"height:0.15em;",[45,437],{},[45,439,172],{"className":440},[221],[45,442,136],{"className":443,"style":152},[89,90],[45,445,177],{"className":446},[228],[45,448],{"className":449,"style":233},[232],[45,451,359],{"className":452},[237],[45,454],{"className":455,"style":233},[232],[45,457,459,462,465,468,471],{"className":458},[80],[45,460],{"className":461,"style":214},[84],[45,463,16],{"className":464},[89,90],[45,466,172],{"className":467},[221],[45,469,136],{"className":470,"style":152},[89,90],[45,472,474,477],{"className":473},[228],[45,475,177],{"className":476},[228],[45,478,480],{"className":479},[281],[45,481,483],{"className":482},[285],[45,484,486],{"className":485},[289],[45,487,490],{"className":488,"style":489},[293],"height:0.7144em;",[45,491,493,496],{"style":492},"top:-3.113em;margin-right:0.05em;",[45,494],{"className":495,"style":301},[300],[45,497,499],{"className":498},[305,306,307,308],[45,500,67],{"className":501},[89,90,308],[16,503,504,505,533,534,611,612,656,657,660,661,663],{},"Because this is a power of a quantity approaching 1, small absolute increases in ",[45,506,508,521],{"className":507,"translate":49},[48],[45,509,511],{"className":510},[53],[55,512,513],{"xmlns":57},[59,514,515,519],{},[62,516,517],{},[65,518,16],{},[69,520,16],{"encoding":71},[45,522,524],{"className":523,"ariaHidden":76},[75],[45,525,527,530],{"className":526},[80],[45,528],{"className":529,"style":117},[84],[45,531,16],{"className":532},[89,90]," near the ceiling produce disproportionately large relative increases in ",[45,535,537,555],{"className":536,"translate":49},[48],[45,538,540],{"className":539},[53],[55,541,542],{"xmlns":57},[59,543,544,552],{},[62,545,546],{},[342,547,548,550],{},[65,549,346],{},[348,551,350],{},[69,553,554],{"encoding":71},"P_{\\text{task}}",[45,556,558],{"className":557,"ariaHidden":76},[75],[45,559,561,565],{"className":560},[80],[45,562],{"className":563,"style":564},[84],"height:0.8333em;vertical-align:-0.15em;",[45,566,568,571],{"className":567},[89],[45,569,346],{"className":570,"style":390},[89,90],[45,572,574],{"className":573},[281],[45,575,577,603],{"className":576},[285,397],[45,578,580,600],{"className":579},[289],[45,581,583],{"className":582,"style":404},[293],[45,584,585,588],{"style":407},[45,586],{"className":587,"style":301},[300],[45,589,591],{"className":590},[305,306,307,308],[45,592,594],{"className":593},[89,308],[45,595,597],{"className":596},[89,420,308],[45,598,350],{"className":599},[89,308],[45,601,428],{"className":602},[427],[45,604,606],{"className":605},[289],[45,607,609],{"className":608,"style":435},[293],[45,610],{},". The result is a curve that looks flat-then-sharp — the signature \"emergent\" S-curve reported across dozens of BIG-Bench tasks — even though ",[45,613,615,635],{"className":614,"translate":49},[48],[45,616,618],{"className":617},[53],[55,619,620],{"xmlns":57},[59,621,622,632],{},[62,623,624,626,628,630],{},[65,625,16],{},[169,627,172],{"stretchy":171},[65,629,136],{},[169,631,177],{"stretchy":171},[69,633,634],{"encoding":71},"p(N)",[45,636,638],{"className":637,"ariaHidden":76},[75],[45,639,641,644,647,650,653],{"className":640},[80],[45,642],{"className":643,"style":214},[84],[45,645,16],{"className":646},[89,90],[45,648,172],{"className":649},[221],[45,651,136],{"className":652,"style":152},[89,90],[45,654,177],{"className":655},[228]," itself is perfectly smooth. Critically, the authors show that switching the metric to something continuous (token-level log-likelihood, Brier score, or partial credit) restores smoothness for the ",[20,658,659],{},"same"," models on the ",[20,662,659],{}," tasks. This is a strong result precisely because it is a controlled comparison: same system, same data, different readout function, different qualitative conclusion.",[16,665,666,667,671],{},"This generalizes a point long known in statistics and psychophysics: a step function of a continuous variable is trivial to construct by thresholding, and thresholding-induced discontinuities tell you about the threshold, not about the variable. The same logic explains apparent \"aha moments\" in insight-problem-solving research, where subjective, binary self-report obscures continuous, sub-threshold information accumulation later revealed by eye-tracking and other continuous instruments. It explains apparent ecological \"collapses\" that dissolve into gradual decline once sampling frequency is increased. The common structural flaw across these cases is ",[668,669,670],"strong",{},"conflating the discreteness of the observation with the discreteness of the process",".",[11,673,675],{"id":674},"where-the-argument-overreaches","Where the Argument Overreaches",[16,677,678,679,682],{},"The inferential move from \"some reported emergent abilities are metric artifacts\" to \"emergence is a pseudo-concept\" commits what I will call an ",[668,680,681],{},"elimination-by-counterexample fallacy",": a specific, well-controlled falsification of a subset of claims is treated as a falsification of the entire category. Three considerations limit the scope of the mirage argument.",[684,685,687],"h3",{"id":686},"compositional-task-structure-is-a-real-property-of-the-world-not-just-of-the-metric","Compositional task structure is a real property of the world, not just of the metric",[16,689,690,691,719,720,878,879,882,883,886,887,890],{},"If a downstream application genuinely requires all ",[45,692,694,707],{"className":693,"translate":49},[48],[45,695,697],{"className":696},[53],[55,698,699],{"xmlns":57},[59,700,701,705],{},[62,702,703],{},[65,704,67],{},[69,706,67],{"encoding":71},[45,708,710],{"className":709,"ariaHidden":76},[75],[45,711,713,716],{"className":712},[80],[45,714],{"className":715,"style":85},[84],[45,717,67],{"className":718},[89,90]," steps to be correct — a compiled program must have zero syntax errors; a proof must have no invalid steps — then ",[45,721,723,760],{"className":722,"translate":49},[48],[45,724,726],{"className":725},[53],[55,727,728],{"xmlns":57},[59,729,730,758],{},[62,731,732,738,740,742,744,746,748,750,752],{},[342,733,734,736],{},[65,735,346],{},[348,737,350],{},[169,739,172],{"stretchy":171},[65,741,136],{},[169,743,177],{"stretchy":171},[169,745,359],{},[65,747,16],{},[169,749,172],{"stretchy":171},[65,751,136],{},[191,753,754,756],{},[169,755,177],{"stretchy":171},[65,757,67],{},[69,759,374],{"encoding":71},[45,761,763,833],{"className":762,"ariaHidden":76},[75],[45,764,766,769,815,818,821,824,827,830],{"className":765},[80],[45,767],{"className":768,"style":214},[84],[45,770,772,775],{"className":771},[89],[45,773,346],{"className":774,"style":390},[89,90],[45,776,778],{"className":777},[281],[45,779,781,807],{"className":780},[285,397],[45,782,784,804],{"className":783},[289],[45,785,787],{"className":786,"style":404},[293],[45,788,789,792],{"style":407},[45,790],{"className":791,"style":301},[300],[45,793,795],{"className":794},[305,306,307,308],[45,796,798],{"className":797},[89,308],[45,799,801],{"className":800},[89,420,308],[45,802,350],{"className":803},[89,308],[45,805,428],{"className":806},[427],[45,808,810],{"className":809},[289],[45,811,813],{"className":812,"style":435},[293],[45,814],{},[45,816,172],{"className":817},[221],[45,819,136],{"className":820,"style":152},[89,90],[45,822,177],{"className":823},[228],[45,825],{"className":826,"style":233},[232],[45,828,359],{"className":829},[237],[45,831],{"className":832,"style":233},[232],[45,834,836,839,842,845,848],{"className":835},[80],[45,837],{"className":838,"style":214},[84],[45,840,16],{"className":841},[89,90],[45,843,172],{"className":844},[221],[45,846,136],{"className":847,"style":152},[89,90],[45,849,851,854],{"className":850},[228],[45,852,177],{"className":853},[228],[45,855,857],{"className":856},[281],[45,858,860],{"className":859},[285],[45,861,863],{"className":862},[289],[45,864,867],{"className":865,"style":866},[293],"height:0.6644em;",[45,868,869,872],{"style":296},[45,870],{"className":871,"style":301},[300],[45,873,875],{"className":874},[305,306,307,308],[45,876,67],{"className":877},[89,90,308]," is not a measurement artifact but a ",[20,880,881],{},"correct description of user-relevant success probability",". The mechanism is fully explicable and unmysterious (it is elementary probability, not a hidden phase transition), but the phenomenon itself — a sharp increase in the fraction of usable outputs over a narrow capability range — is real and consequential. The mirage critique correctly de-mystifies the ",[20,884,885],{},"mechanism","; it does not eliminate the ",[20,888,889],{},"phenomenon",". This is an important distinction that gets collapsed in the popular restatement of the argument: \"unexplained\" and \"illusory\" are not synonyms.",[684,892,894],{"id":893},"genuine-phase-transitions-exist-elsewhere-in-complex-systems-with-metric-independent-signatures","Genuine phase transitions exist elsewhere in complex systems, with metric-independent signatures",[16,896,897,898,930,931,930,962,993,994,997,998,1000],{},"Statistical physics offers a rigorous counter-example class. The Curie point in ferromagnetic systems is a true second-order phase transition: renormalization-group theory predicts critical exponents (",[45,899,901,916],{"className":900,"translate":49},[48],[45,902,904],{"className":903},[53],[55,905,906],{"xmlns":57},[59,907,908,913],{},[62,909,910],{},[65,911,912],{},"β",[69,914,915],{"encoding":71},"\\beta",[45,917,919],{"className":918,"ariaHidden":76},[75],[45,920,922,926],{"className":921},[80],[45,923],{"className":924,"style":925},[84],"height:0.8889em;vertical-align:-0.1944em;",[45,927,912],{"className":928,"style":929},[89,90],"margin-right:0.0528em;",", ",[45,932,934,949],{"className":933,"translate":49},[48],[45,935,937],{"className":936},[53],[55,938,939],{"xmlns":57},[59,940,941,946],{},[62,942,943],{},[65,944,945],{},"γ",[69,947,948],{"encoding":71},"\\gamma",[45,950,952],{"className":951,"ariaHidden":76},[75],[45,953,955,958],{"className":954},[80],[45,956],{"className":957,"style":117},[84],[45,959,945],{"className":960,"style":961},[89,90],"margin-right:0.0556em;",[45,963,965,980],{"className":964,"translate":49},[48],[45,966,968],{"className":967},[53],[55,969,970],{"xmlns":57},[59,971,972,977],{},[62,973,974],{},[65,975,976],{},"ν",[69,978,979],{"encoding":71},"\\nu",[45,981,983],{"className":982,"ariaHidden":76},[75],[45,984,986,989],{"className":985},[80],[45,987],{"className":988,"style":85},[84],[45,990,976],{"className":991,"style":992},[89,90],"margin-right:0.0637em;",") that are confirmed across ",[20,995,996],{},"independent"," observables — magnetization, susceptibility, correlation length, specific heat — all of which diverge or vanish at the ",[20,999,659],{}," critical temperature regardless of which one you choose to measure. Percolation transitions in random graphs have analogous universality: connectivity properties transition sharply as edge density crosses a threshold, provable from first principles in probability theory, independent of any particular observational readout. These are cases where the invariance test that damns the LLM \"emergent abilities\" — does the transition survive a change of metric? — is passed with flying colors. A categorical claim that \"emergence is always a metric artifact\" is empirically false the moment one leaves the specific domain (LLM benchmark evaluation) in which the mirage paper operates.",[684,1002,1004],{"id":1003},"emergence-denotes-a-family-of-distinct-phenomena-only-some-of-which-are-at-stake","\"Emergence\" denotes a family of distinct phenomena, only some of which are at stake",[16,1006,1007,1008,1011,1012,1015],{},"The philosophy-of-science literature (Anderson's ",[20,1009,1010],{},"\"More Is Different,\""," 1972, being the canonical reference point) distinguishes at minimum: (a) epistemic emergence — properties that are difficult to predict from micro-level rules even though they are in principle derivable; (b) ontological\u002Fweak emergence — properties that are novel at the macro scale but reducible in principle via simulation; and (c) strong emergence — properties claimed to be irreducible even in principle, a much more contested notion. The mirage argument, properly scoped, is an argument about a specific ",[20,1013,1014],{},"epistemic"," illusion — apparent unpredictability induced by metric choice — and says essentially nothing about whether weak emergence exists in neural networks. Conflating these senses is part of why the debate generates more heat than light.",[684,1017,1019],{"id":1018},"the-mirror-image-case-when-a-linear-metric-conceals-genuine-nonlinear-structure","The mirror-image case: when a linear metric conceals genuine nonlinear structure",[16,1021,1022,1023,1026,1027,1030],{},"The preceding discussion has treated metric choice as a source of ",[20,1024,1025],{},"illusory"," discontinuity — the LLM case, the insight-problem-solving case, the coarse ecological census case. All three share a common direction of error: a nonlinear or thresholded readout function manufactures apparent sharpness out of smooth underlying reality. But the historical development of the Richter scale for earthquake magnitude illustrates the ",[20,1028,1029],{},"opposite"," direction of error, and is instructive precisely because it does not fit the mirage template.",[16,1032,1033,1034,1037,1038,1041],{},"Seismologists initially quantified earthquake severity using raw ground-motion amplitude, a linear instrumental reading directly proportional to the physical displacement recorded on a seismograph. This metric turned out to be poorly matched to the construct researchers actually cared about — destructive potential — because seismic energy release scales not linearly but multiplicatively with amplitude: a tenfold increase in amplitude corresponds to roughly a thirty-two-fold increase in radiated energy, and destructive capacity scales with energy, not displacement. Under the original linear amplitude metric, the true structure of the phenomenon — a Gutenberg–Richter power-law distribution of energy release, with catastrophic events differing from minor tremors by many orders of magnitude — was compressed into a narrow, visually undramatic numerical range that systematically obscured how categorically different a magnitude-8 event is from a magnitude-5 event. Charles Richter's solution, in 1935, was to define magnitude as the base-10 logarithm of amplitude (with a distance correction). This ",[20,1035,1036],{},"introduced"," nonlinearity into the metric deliberately, so that equal increments on the new scale would correspond to equal ",[20,1039,1040],{},"ratios"," of underlying energy release — turning a multiplicative, difficult-to-communicate reality into an additive, interpretable one.",[16,1043,1044,1045,1048],{},"The earthquake case is the structural mirror image of the LLM mirage: there, a nonlinear metric (exact-match accuracy under compositional task structure) manufactured a false discontinuity out of an underlying reality that was in fact smooth. Here, a linear metric (raw amplitude) concealed a real, well-characterized multiplicative structure, and the corrective move was to adopt a ",[20,1046,1047],{},"logarithmic"," — i.e., nonlinear — metric specifically because the underlying physical process (elastic energy release, governed by fault rupture area and stress drop) is itself exponential in the quantity being measured. Critically, this was not an arbitrary choice: the logarithmic transform was adopted because independent physical theory (the relationship between seismic moment, rupture dimensions, and radiated energy) predicted the multiplicative relationship in advance, and the new scale was validated against it — precisely the kind of \"independent theoretical prediction\" criterion that will be formalized as Test 2 in the diagnostic framework below.",[16,1050,1051,1052,1055,1056,1059,1060,1063],{},"Read together, the LLM and earthquake cases triangulate the deeper methodological principle at stake in this entire debate: ",[668,1053,1054],{},"no metric is neutral with respect to the qualitative shape it imposes on a phenomenon, and the direction of possible distortion runs both ways."," A nonlinear\u002Fthresholded metric can manufacture illusory sharpness from smooth reality (LLM benchmarks); a linear metric can just as easily conceal genuine, theoretically-grounded nonlinear structure (pre-Richter amplitude scales). \"Prefer continuous metrics\" is therefore not, by itself, the correct methodological lesson to extract from the mirage argument — the correct lesson is that the ",[20,1057,1058],{},"functional form of the metric must be matched to the functional form of the true relationship between the raw measured quantity and the construct of scientific interest",", and that this matching must be justified independently (by theory, or by validation against an external criterion such as destructiveness), rather than assumed by default in either direction. Choosing a continuous metric merely because it is continuous, without asking whether ",[20,1061,1062],{},"that particular"," continuous transformation tracks the construct of interest, would be as unprincipled as the exact-match errors the mirage paper criticizes.",[11,1065,1067],{"id":1066},"what-machine-learning-theory-adds-beyond-benchmark-curves","What Machine Learning Theory Adds: Beyond Benchmark Curves",[16,1069,1070],{},"The most interesting recent evidence bearing on this question does not come from benchmark accuracy curves at all, but from the internal training dynamics of neural networks — a domain the original mirage paper does not address, since it studies only inference-time scaling behavior across model sizes, not the trajectory of a single model over training time.",[684,1072,1074],{"id":1073},"grokking-as-a-genuine-dynamical-transition","Grokking as a genuine dynamical transition",[16,1076,1077,1078,1081,1082,1085],{},"Power, Burns, Edwards, Babuschkin, and Misra (2022) documented \"grokking\": small algorithmic-task networks (e.g., trained on modular arithmetic) that memorize the training set almost immediately, sit at near-zero test accuracy for an extended plateau, and then transition sharply to near-perfect generalization — often orders of magnitude later in training than when training loss first converged. This is not a benchmark-metric artifact in the Schaeffer sense: it is observed in test-set accuracy across many random seeds, holds under continuous-metric readouts such as test loss, and has since been given a mechanistic explanation via the competition between a memorizing circuit and a generalizing circuit inside the same network, with the generalizing circuit eventually winning out under weight decay pressure (Nanda et al., 2023, \"Progress Measures for Grokking via Mechanistic Interpretability\"). Crucially, Nanda et al. showed that ",[20,1079,1080],{},"continuous, mechanistic progress measures"," — quantities derived from the Fourier structure of the learned embeddings, tracking the gradual formation of trigonometric representations used to implement modular addition — rise smoothly and predictably ",[20,1083,1084],{},"during"," the apparently flat plateau in test accuracy. This is the mirror image of the LLM benchmark story: here, a discrete-looking transition in a coarse metric (accuracy) is underlain by continuous progress in a well-chosen finer-grained metric, but that finer-grained progress genuinely culminates in a real, mechanistically characterized restructuring of the network's internal computation — a circuit-level phase transition, not merely a statistical readout effect. Both the coarse and the fine pictures can be true simultaneously; the interesting question is what work the fine-grained progress measure is doing, and grokking research supplies an unusually clear answer.",[684,1087,1089],{"id":1088},"induction-heads-and-training-loss-phase-transitions","Induction heads and training-loss phase transitions",[16,1091,1092,1093,1096,1097,1100,1101,1103,1104,1107,1108,1110],{},"Olsson et al. (2022, \"In-context Learning and Induction Heads,\" Anthropic) reported a phenomenon with an even stronger claim to being a genuine phase transition: a sharp bump in the training-loss curve of transformer language models, occurring at a specific, consistent point early in training, that coincides — across model scales and architectures — with the formation of \"induction heads,\" attention circuits that implement a simple copying algorithm ([A]",[45,1094,1095],{},"B"," ... ",[45,1098,1099],{},"A"," → predict ",[45,1102,1095],{},") which is later repurposed for more general in-context learning. This transition is visible in the training loss itself — a continuous quantity, optimized directly, with no thresholding or exact-match discretization involved — which directly defeats the mirage explanation, since the mirage mechanism requires an artificially discretized readout to manufacture the appearance of sharpness. Moreover, the timing of the loss bump correlates causally (via ablation studies) with the emergence of the induction circuit, giving a mechanistic account of ",[20,1105,1106],{},"why"," the transition occurs, analogous to how renormalization-group theory explains ",[20,1109,1106],{}," the Curie point produces sharp behavior in magnetization. This is about as close as deep learning currently has to physics-style emergence: a metric-invariant, mechanistically explained, reproducible discontinuity in a dynamical system.",[684,1112,1114],{"id":1113},"scaling-laws-loss-landscape-geometry-and-the-compression-of-variance-near-threshold","Scaling laws, loss landscape geometry, and the compression of variance near threshold",[16,1116,1117,1118,1121],{},"There is a complementary, more skeptical strand of ML theory worth foregrounding, because it partially supports the mirage camp from a different angle. Standard neural scaling laws (Kaplan et al., 2020; Hoffmann et al., 2022) show that ",[20,1119,1120],{},"loss itself"," — the most theoretically well-motivated continuous metric available — scales as a smooth power law in parameters, data, and compute, with no discontinuities across many orders of magnitude. If loss is smooth and downstream task accuracy is not, the most parsimonious explanation, absent evidence of a discrete internal restructuring, is indeed that the task-level discontinuity is a downstream artifact of the mapping from loss to task performance (which is where compositional structure, thresholding, and metric choice enter) rather than a discontinuity in the model's underlying competence. This is precisely Schaeffer et al.'s point, and where it is unaddressed by a training-time mechanistic story (as it is for most reported \"emergent\" benchmark abilities, in contrast to induction heads or grokking), the mirage explanation should be treated as the default, more parsimonious hypothesis under an Occam's-razor-style prior.",[11,1123,1125],{"id":1124},"the-nature","The Nature",[16,1127,1128],{},"Bringing these threads together, I propose three operational tests for distinguishing genuine dynamical emergence from metric-induced illusion in any complex system, machine learning included.",[16,1130,1131,1134,1135,1138],{},[668,1132,1133],{},"Metric invariance, correctly directed."," Does the transition persist under a change to a metric whose functional form is independently justified as tracking the construct of interest, rather than merely being continuous? As the earthquake-magnitude case shows, \"continuous\" is not synonymous with \"correct\" — a linear metric can just as easily obscure a genuine multiplicative structure as a thresholded metric can manufacture a false one. The relevant question is not \"is the readout continuous?\" but \"does the readout's functional form match the functional form of the true generative relationship?\" Passing this properly-posed test (as with induction-head loss bumps, ferromagnetic critical exponents measured via multiple independent observables, or logarithmic earthquake magnitude validated against seismic energy theory) is strong evidence for a genuine underlying structure. Failing it (as with most BIG-Bench \"emergent abilities\" under log-likelihood readouts) is strong evidence for an artifact of the ",[20,1136,1137],{},"original"," metric choice.",[16,1140,1141,1144],{},[668,1142,1143],{},"Independent theoretical prediction versus post hoc curve-fitting."," Was the location or existence of the transition predicted by an independent theoretical framework (renormalization group theory for critical phenomena; circuit-formation theory for induction heads), or was it identified only retrospectively by eyeballing a curve that happens to bend? A priori prediction is a much stronger epistemic warrant than post hoc pattern recognition, which is highly susceptible to confirmation bias and multiple-comparisons problems across the dozens of tasks examined in benchmark suites.",[16,1146,1147,1150],{},[668,1148,1149],{},"Locus of the discontinuity: generative mechanism versus readout function."," Is the discontinuity located in the process that generates the system's internal state (a circuit forming, an order parameter crossing a critical value, a competing-circuit dynamic resolving) or in the function that maps that internal state onto an external report (an exact-match scorer, a binary \"solved\u002Funsolved\" judgment, a coarse sampling interval)? Discontinuities of the first kind are candidates for genuine emergence; discontinuities of the second kind should be treated with the presumption of artifact until proven otherwise.",[16,1152,1153],{},"Applying this framework: induction-head formation and grokking's circuit competition pass all three tests and merit description as genuine — if modest-scale — phase transitions in learning dynamics. Most benchmark-reported \"emergent abilities\" of large language models fail Test 1 outright, as Schaeffer et al. demonstrate directly, and typically also fail Test 2, since they are identified by scanning many tasks for discontinuous-looking curves rather than predicted from an independent theory of capability acquisition.",[11,1155,1157],{"id":1156},"conclusion","Conclusion",[16,1159,1160],{},"The earthquake-magnitude case is a useful corrective to place alongside the machine learning evidence discussed above, because it prevents the diagnostic framework from collapsing into a simple heuristic — \"prefer smooth metrics, distrust thresholded ones.\" That heuristic would have been the wrong lesson to draw from Schaeffer et al., since it would have counseled seismologists to reject the (nonlinear, logarithmic) Richter scale in favor of the (linear, continuous) amplitude reading it replaced — precisely backwards, given that the logarithmic scale is the one that tracks physical reality. The general principle is less about linearity per se than about whether a metric's functional form has been independently derived from, or validated against, a theory of the underlying generative process, rather than adopted by convention or convenience.",[16,1162,1163,1164,1167],{},"The mirage critique is best understood not as a demonstration that emergence is illusory in general, but as a demonstration that a ",[20,1165,1166],{},"specific and popular"," evidentiary practice — reporting sharp jumps in exact-match accuracy across model scale as evidence of qualitatively new capabilities — is statistically confounded and should be retired or substantially qualified. This is a genuine and valuable correction to a literature that had, in places, drifted toward treating benchmark curve shapes as direct windows onto model cognition. But machine learning theory itself, through mechanistic interpretability and training-dynamics research largely orthogonal to the benchmark-scaling literature that motivated the mirage paper, has independently supplied examples — grokking's circuit competition, induction-head formation and its associated training-loss transition — that satisfy much stricter, metric-invariant, mechanistically grounded criteria for genuine dynamical phase transitions. The lesson is not that emergence is real or unreal as a blanket matter, but that the word has been used to cover both categories indiscriminately, and that the field now has, for the first time, the empirical and theoretical tools to tell them apart on a case-by-case basis. That discipline — metric invariance, independent prediction, and attention to the locus of discontinuity — is the appropriate successor to both uncritical emergence-talk and its blanket dismissal.",{"title":1169,"searchDepth":1170,"depth":1170,"links":1171},"",2,[1172,1173,1174,1181,1186,1187],{"id":13,"depth":1170,"text":14},{"id":39,"depth":1170,"text":40},{"id":674,"depth":1170,"text":675,"children":1175},[1176,1178,1179,1180],{"id":686,"depth":1177,"text":687},3,{"id":893,"depth":1177,"text":894},{"id":1003,"depth":1177,"text":1004},{"id":1018,"depth":1177,"text":1019},{"id":1066,"depth":1170,"text":1067,"children":1182},[1183,1184,1185],{"id":1073,"depth":1177,"text":1074},{"id":1088,"depth":1177,"text":1089},{"id":1113,"depth":1177,"text":1114},{"id":1124,"depth":1170,"text":1125},{"id":1156,"depth":1170,"text":1157},false,true,"md",null,{},"2026-06-30","\u002Fblog\u002F2026\u002F2026-06-30-about-the-emergency",{"title":6,"description":1169},"blog\u002F2026\u002F2026-06-30-about-the-emergency","Schaeffer et al. show LLM \"emergent abilities\" often reflect discontinuous metrics masking smooth capability growth. But grokking, induction heads, and Curie-point-style transitions prove genuine phase transitions exist.",[1199,1200,1201],"ai","machine-learning","thoughts","5IerezAtZz2T7xeHq3z3hPhwlxYNqIofzYgLFbtjo8Y",{"id":1204,"title":1205,"body":1206,"description":1169,"draft":1188,"enableComment":1188,"extension":1190,"image":1169,"important":1188,"location":1191,"meta":1516,"navigation":1189,"ogImage":1191,"onday":1517,"path":1518,"seo":1519,"stem":1520,"summary":1521,"tags":1522,"__hash__":1523},"blog\u002Fblog\u002F2026\u002F2026-07-26-the-messianic-narrative-of-anthropic.md","The Messianic Narrative of Anthropic",{"type":8,"value":1207,"toc":1509},[1208,1212,1215,1218,1223,1226,1232,1235,1240,1247,1251,1258,1267,1272,1279,1282,1285,1288,1310,1313,1330,1333,1337,1342,1352,1355,1362,1365,1368,1372,1375,1385,1396,1401,1408,1411,1417,1433,1440,1443,1450,1456,1459,1461,1464,1478,1481,1484,1490,1493,1496,1499,1504],[11,1209,1211],{"id":1210},"judaism-and-the-messiah","Judaism and the Messiah",[16,1213,1214],{},"Within the theological framework of Judaism, the figure of the Messiah represents a systematic redemptive expectation gradually crystallized by the ancient Jewish people through prolonged suffering, exile, and doctrinal reflection. Its core impetus derives from a structural predicament: following the successive failures of monarchical authority and the Temple, the nation required a force transcending empirical reality to sustain identity and hope. After losing sovereign kingship, Temple sovereignty, and political freedom, a narrative capable of indefinitely deferred verification — yet perpetually furnishing meaning for action and ultimate hope — became an existential necessity. The Messiah constitutes the personified condensation of this narrative, bearing the weight of every unmet worldly aspiration: national independence, social justice, bodily resurrection, perpetual peace among nations, and the direct rule of the Divine.",[16,1216,1217],{},"Successive eras and communities have projected disparate contents into this \"formula\": the oppressed inserted armed liberation; the devout inserted Temple restoration; the afflicted inserted eschatological recompense; the guilt-ridden inserted vicarious atonement. Christianity elected the genetic strands of the \"Suffering Servant\" and the \"Heavenly Son of Man\" to define Jesus; mainstream Judaism, by contrast, ultimately rejected Jesus and preserved the yet-to-arrive, certain-to-come, thoroughly earthly Davidic Messiah.",[16,1219,1220],{},[668,1221,1222],{},"Such is the genesis of the Messianic archetype within Judaism.",[16,1224,1225],{},"From the perspectives of comparative mythology and political psychology, this structure approximates what may be termed an archetypal narrative in human culture:",[1227,1228,1229],"blockquote",{},[16,1230,1231],{},"The world descends into crisis;\nThe masses remain unaware;\nA minority attains true knowledge;\nFrom this minority a prophet emerges;\nThe prophet suffers doubt, incomprehension, and attack;\nThe prophet, having endured tribulation, ultimately acquires authority or influence;\nSalvation is delivered to the multitude, or catastrophe averted.",[16,1233,1234],{},"Viewed through this lens, the Messiah transcends the boundaries of a merely religious concept; it also constitutes one of the most ancient and influential archetypes in the human narrative repertoire:",[1227,1236,1237],{},[16,1238,1239],{},"When the world descends into crisis, a chosen figure emerges, bearing a mission surpassing ordinary human capacity, ultimately delivering redemption and renewal.",[16,1241,1242],{},[1243,1244],"img",{"alt":1245,"src":1246},"Anthropic refused to sign an initiative recognizing open-weight AI models, deliberately maintaining distance and \"independence\" from other AI companies","https:\u002F\u002Fimage-assets.dreams.plus\u002F202607310100531.png",[11,1248,1250],{"id":1249},"the-biographical-background-of-the-amodei-siblings","The Biographical Background of the Amodei Siblings",[16,1252,1253,1254,1257],{},"The very name of the company is revealing in this regard: \"Anthropic,\" etymologically derived from the Greek ",[20,1255,1256],{},"anthrōpos"," — \"human being\" — signals their self-conception as a technology enterprise oriented toward human-centeredness and the destiny and welfare of all humankind. This, precisely, is the role they have long labored to inhabit.",[16,1259,1260,1261,1266],{},"It is by no means coincidental that the Amodei siblings' mother was a devout Jewish adherent. Their mother, ",[23,1262,1265],{"href":1263,"rel":1264},"https:\u002F\u002Fgrokipedia.com\u002Fpage\u002FElena_Engel#ref-19",[27],"Elena Engel",", is a Jewish-American library project manager; their father, Riccardo Amodei, was an Italian leather artisan who suffered chronic illness and passed away when Dario was still young. The present author submits that there are reasonable grounds to suspect a correlation between Dario's preoccupation with the Messianic narrative and the circumstances of his familial upbringing. In Jewish-American households, domestic instruction frequently occupies a formative role.",[1227,1268,1269],{},[16,1270,1271],{},"As a mother, Engel played a key role in fostering her children's intellectual curiosity and ethical outlook, instilling a strong sense of right and wrong as well as responsibility toward improving the world.",[16,1273,1274,1275,1278],{},"This educational paradigm, which internalizes the repair of the world (",[20,1276,1277],{},"Tikkun Olam",", a core ethical concept in Judaism) as an individual vocation, is in its essence a secularized Messianic spirit: the conviction that human action, rather than pure divine intervention, can propel the world toward perfection. The premature death of the father and the absence of a stable masculine pillar in the developmental environment may have further intensified Dario's psychological need for a grand, determinate, and ultimate explanatory framework.",[16,1280,1281],{},"The Messianic narrative supplies precisely such a structure: it promises a definitive inflection point from chaos to order, from suffering to redemption. His technological optimism — and indeed his faith in the salvific potential of artificial general intelligence — may thus be understood not as a deduction from pure rationality, but as the projection, onto the technological age, of an ethical-redemptive complex molded by family.",[16,1283,1284],{},"Why, in the era of AGI, would a technology company so spontaneously adopt discursive structures that historically belonged to the domains of religion, philosophy of history, and eschatology?",[16,1286,1287],{},"The present author contends that traditional religion and eschatology principally address three questions:",[1289,1290,1291,1298,1304],"ul",{},[1292,1293,1294,1297],"li",{},[668,1295,1296],{},"Explaining everything",": Why is the world as it is? What are the root causes of suffering, chaos, and injustice?",[1292,1299,1300,1303],{},[668,1301,1302],{},"Promising salvation",": What constitutes the ultimate, perfect future state? (e.g., the Millennium, Heaven, the earthly Kingdom of God)",[1292,1305,1306,1309],{},[668,1307,1308],{},"Indicating the path",": Through what means (faith, spiritual practice, revolution) can that future be attained?",[16,1311,1312],{},"With the emergence of AGI, these three dimensions find precise correspondence:",[1289,1314,1315,1320,1325],{},[1292,1316,1317,1319],{},[668,1318,1296],{},": The root of all human afflictions — disease, poverty, conflict, even mortality itself — lies in \"insufficient intelligence.\" Extant human cognition and conventional computation cannot process information of adequate complexity.",[1292,1321,1322,1324],{},[668,1323,1302],{},": AGI or ASI (artificial superintelligence) will constitute an omniscient, omnibenevolent, quasi-omnipotent \"godlike entity,\" resolving energy crises, climate change, and disease, and even enabling \"mind uploading\" and \"digital immortality.\"",[1292,1326,1327,1329],{},[668,1328,1308],{},": The technological trajectory — computational scaling, algorithmic breakthroughs, alignment research.",[16,1331,1332],{},"To be clear, we cannot, on this basis, assert that Anthropic's value system derives directly from familial background, nor can we ascertain whether it bears any relationship to Israel; nor can we simply reduce its AI safety philosophy to personal developmental history. Nevertheless, this formative milieu may provide a useful interpretive lens for understanding why Dario has long gravitated toward apocalyptic and safety-centric narratives.",[11,1334,1336],{"id":1335},"the-narrative-of-capital-apocalyptic-marketing","The Narrative of Capital: Apocalyptic Marketing",[1227,1338,1339],{},[16,1340,1341],{},"AI is perilous; only we can save AI.",[16,1343,1344,1345,1348,1349,671],{},"In the modern commercial system, the dynamic behavior of markets is not determined solely by fundamental variables but is, to a substantial degree, governed by two mutually interwoven psychological variables: ",[20,1346,1347],{},"expectation"," and ",[20,1350,1351],{},"sentiment",[16,1353,1354],{},"Expectation constitutes the rational anchor of capital allocation behavior — it is grounded in deductions concerning future cash flows, technological trajectories, or policy environments, forming a logical framework subject to periodic verification. Sentiment, by contrast, functions as the nonlinear accelerator of expectation realization: driven by collective psychological states such as greed and fear, it amplifies or distorts information, thereby engendering systematic deviations of price from intrinsic value.",[16,1356,1357,1358,1361],{},"It is worth noting that capital is, in its essence, ",[20,1359,1360],{},"narrative-preferring",": it seeks out and rewards stories that simultaneously exhibit simplicity, grand vision, and verifiable milestones. Only such narratives can efficiently aggregate consensus within an environment of information asymmetry, converting dispersed individual beliefs into coherent asset-pricing behavior.",[16,1363,1364],{},"Put differently, commercial competition has, to a considerable extent, evolved into a contest of competence in managing expectations and steering sentiment. The former demands that enterprises continually honor their narrative commitments through clear milestones; the latter demands prudent calibration in response to excessive fluctuations in market psychology.",[16,1366,1367],{},"Whether a commercial narrative can sustain capital attraction, then, is determined not by its fantastical appeal, but by whether it can, across the dual dimensions of expectation and sentiment, both ignite the imagination and withstand the scrutiny of periodic factual verification.",[11,1369,1371],{"id":1370},"malefactors-and-prophets","Malefactors and Prophets",[16,1373,1374],{},"Against the backdrop of intensifying Sino-American antagonism, Anthropic has exploited a Cold War stereotype deeply embedded in modern Western discourse: the portrayal of China as an authoritarian, malevolent, despotic, and formidable power — a threat not merely to its own populace but to the entire democratic world. Anthropic, cast in the role of the minority \"prophet\" who has discerned the danger ahead of others, must take action; the \"democratic camp\" must retain AI supremacy and assume the mantle of guardianship. The company has thus fashioned itself as a democratic, progressive, safety-oriented AI enterprise.",[16,1376,1377,1378,671],{},"In recent times, Anthropic has not spared its criticism of Chinese large language models, alleging that they have performed distillation upon its Claude series of models. This is documented in the February 2026 report ",[20,1379,1380],{},[23,1381,1384],{"href":1382,"rel":1383},"https:\u002F\u002Fwww.anthropic.com\u002Fnews\u002Fdetecting-and-preventing-distillation-attacks",[27],"Detecting and Preventing Distillation Attacks",[16,1386,1387,1388,1395],{},"Implicit within this construction is a metaphor: a regime of the \"authoritarian and malevolent\" variety, such as China, allegedly lacks vigilance and rationality regarding AI, and will abuse or recklessly permit AI technologies to inflict harm. Following President Trump's visit to China in May 2026, Anthropic published a report titled ",[20,1389,1390],{},[23,1391,1394],{"href":1392,"rel":1393},"https:\u002F\u002Fwww.anthropic.com\u002Fresearch\u002F2028-ai-leadership",[27],"2028: Two Scenarios for Global AI Leadership",", in which it stated:",[1227,1397,1398],{},[16,1399,1400],{},"If the frontier is set by regimes that treat AI as an instrument of repression, military advantage over democracies, and domestic control, the transition is less likely to go well, for those regimes' own citizens or anyone else.\nHistorically, the reach of authoritarian rule has been limited by its dependence on human enforcers to carry out surveillance and repression. Powerful AI systems may remove that dependency, enabling automated repression on a far greater scale. For that reason, the prospect of the CCP leading in AI is among the greatest threats to a successful transition.",[16,1402,1403,1404,1407],{},"From this logic, the Chinese Communist Party is no longer merely a state regime but is recast as a ",[20,1405,1406],{},"civilizational risk",". This, in fact, already exceeds the traditional narrative of interstate competition, and it is here that Anthropic's approach becomes most peculiar. For ordinary interstate competition is typically framed around American interests versus Chinese interests. Anthropic's narrative, however, operates on an elevated discursive plane: the interests of the free world versus the risk to humanity's shared future.",[16,1409,1410],{},"Herein lies an \"eschatological escalation.\" In the preceding century, the Cold War narrative held that a Soviet victory would threaten the \"free world.\" In the AI era, the narrative has mutated into: the triumph of the wrong power → the endangerment of the future of human civilization.",[16,1412,1413,1416],{},[668,1414,1415],{},"AI has been invested with a status approaching that of a savior — or an apocalypse."," If AGI truly is what its proponents believe it to be — surpassing all humans in intelligence, commanding scientific research, commanding the economy, commanding military affairs — then whoever first possesses AGI no longer merely possesses a technology, but rather the capacity to define the civilization of the future. AI safety, national security, and civilizational security are thereby progressively collapsed into a single question.",[16,1418,1419,1420,930,1423,930,1426,930,1429,1432],{},"AI is, moreover, naturally suited to bear this narrative, insofar as it simultaneously possesses four amplifiers: immense unknowns, world-transforming potential, apocalyptic risk scenarios, and soteriological technological vision. One consequently observes, throughout the Anthropic community and Dario's blog, an enthusiasm for discussing AI in language approaching the theological: ",[20,1421,1422],{},"Superintelligence",[20,1424,1425],{},"Alignment",[20,1427,1428],{},"the Control Problem",[20,1430,1431],{},"Existential Risk",". While these concepts carry technical significance, they are, at the cultural level, readily mapped onto the traditional eschatological framework.",[16,1434,1435,1436,1439],{},"In religious allegory, the chain of ",[20,1437,1438],{},"crisis → enemy → guardian → redemptive solution"," constitutes an ancient and highly efficient mode of propagation. From this vantage point, AI companies may not merely be discussing technical problems; they may also be participating in a larger political and religious narrative.",[16,1441,1442],{},"Viewed accordingly, the competition among AI companies is not only a contest of model capabilities but also a struggle for discursive authority: who is qualified to define the risks of the future, who is qualified to play the guardian, and who is qualified to furnish the path to redemption.",[16,1444,1445,1446,1449],{},"However, the present author submits that what warrants the greatest vigilance here is not conspiracy but ",[20,1447,1448],{},"sacralization",". The critical point is not whether Anthropic deliberately exploits Cold War stereotypes. What merits closer attention is that when a single organization simultaneously holds the authority to define risk, to interpret technology, and to interpret morality, it readily becomes invested with a certain \"sacredness.\" And the catastrophic mass movements of history have, more often than not, been carried out precisely within the framework of such \"sacralizing narratives.\"",[16,1451,1452,1455],{},[20,1453,1454],{},"We know what the risk is; we know what the future is; we know what the correct path is."," This structure is a frequent guest in the histories of religious organizations, revolutionary parties, and state apparatuses. But for it to appear within a major technology company, elevated to the status of core values, is comparatively rare.",[16,1457,1458],{},"Historical experience instructs us that any force claiming to have simultaneously grasped \"the future,\" \"the truth,\" and \"moral legitimacy\" ought to be subjected to sustained scrutiny and oversight.",[11,1460,1157],{"id":1156},[16,1462,1463],{},"If one could still debate whether Anthropic's earlier posturing toward China and the U.S. Department of Defense amounted to \"brand theater,\" the present author maintains that the releases of Mythos and Fable represent the eve of Anthropic's narrative bankruptcy.",[16,1465,1466,1467,1470,1471,1474,1475,671],{},"From the foregoing analysis, it becomes apparent that the predominantly negative online reception of Anthropic may, in fact, serve its purposes rather well. A degree of negative appraisal is not necessarily a loss; it may instead reinforce the company's distinctive positioning. Anthropic is quite content to style itself as a \"prophet\" or \"Messiah\"-type figure. It is not merely selling a model; it is also selling an ",[20,1468,1469],{},"identity"," — a unique symbolic marker within the capital market. Anthropic's conduct has, throughout, been reiterating a single refrain: ",[20,1472,1473],{},"we are not like the other AI companies."," When a technology becomes sufficiently consequential, what companies compete on is no longer solely the product but rather the ",[20,1476,1477],{},"civilizational role",[16,1479,1480],{},"More intriguingly, this touches upon an economic question: in the AI era, \"safety\" is itself being transformed into a form of scarce brand equity.",[16,1482,1483],{},"If everyone can train models, then \"who is fastest,\" \"who is cheapest,\" and \"who is safest\" all become dimensions of competition — and Anthropic manifestly seeks to occupy the third. In its imagined division of civilizational labor within the capital market, OpenAI is the innovator, Google is the technology titan, Meta is the open-source champion, and Anthropic is the guardian — all facing a common adversary: \"malevolent China.\" And once the discourse enters the register of \"who is qualified to define the risks of the future and who is qualified to represent the interests of humanity,\" the debate naturally acquires theological and political-theological inflections, ceasing to be a merely engineering matter.",[16,1485,1486,1487,671],{},"Whether or not one endorses this image, it indisputably possesses strong market differentiation, and the narrative is undeniably compelling. It must, at this juncture, be conceded that Anthropic is indeed adept at ",[20,1488,1489],{},"storytelling",[16,1491,1492],{},"Nevertheless, the present author retains reservations regarding the story Anthropic tells, and the sustainability of this narrative remains open to question. For under a liberal market regime, in no other domain, in no other historical period, in no other market, has there ever been a precedent of capital willingly accepting compromise with \"safety\" as the selling point. Capital is inherently profit-seeking; where the returns are sufficiently high, manufacturing and selling the rope with which one is to be hanged is by no means an uncommon occurrence. The so-called \"guardian narrative\" is perhaps more a matter of brand positioning than a principle for which one is genuinely prepared to bear costs. From the standpoint of political economy, what ultimately determines the strength of an organization's convictions is seldom what it writes in its blog posts, but rather what it sacrifices when interests and principles come into conflict.",[16,1494,1495],{},"The present author holds that, from a Chinese perspective, there is no need whatsoever to engage with Anthropic's theatrics. If one strips away Anthropic's narrative, examining it purely from the perspective of corporate behavior, it is, regardless of the narrative framework it deploys, fundamentally a commercial enterprise. Commerce is inseparable from customers, financing, and IPO listings. It cannot wholly detach itself from commercial interests.",[16,1497,1498],{},"From the perspective of American strategic circles, the core anxiety is not China's ethnic character but rather the prospect that a political system divergent from the liberal democratic tradition might acquire the capacity to define the technical rules of the future. If China were to lead in the AGI competition, the future world order might be shaped according to Chinese institutional preferences. What is truly feared here is not ethnic identity but institutional competition.",[16,1500,1501],{},[668,1502,1503],{},"In brief: for the United States and the West, heirs to over a century of liberal tradition, the prospect of a political regime at variance with the liberal-democratic value system acquiring the capacity to define the technical rules of the future is entirely unacceptable. It is precisely this anxiety that furnishes Anthropic with the operational space to fish in troubled waters.",[16,1505,1506],{},[20,1507,1508],{},"Fin.",{"title":1169,"searchDepth":1170,"depth":1170,"links":1510},[1511,1512,1513,1514,1515],{"id":1210,"depth":1170,"text":1211},{"id":1249,"depth":1170,"text":1250},{"id":1335,"depth":1170,"text":1336},{"id":1370,"depth":1170,"text":1371},{"id":1156,"depth":1170,"text":1157},{},"2026-07-26","\u002Fblog\u002F2026\u002F2026-07-26-the-messianic-narrative-of-anthropic",{"title":1205,"description":1169},"blog\u002F2026\u002F2026-07-26-the-messianic-narrative-of-anthropic","The theatrics of Anthropic, led by Dario Amodei, exhibit the quintessential characteristics of the Judaic Messianic narrative, whose structure typically proceeds as follows: the world confronts an immense crisis of which the majority remains unaware; only a select few prophets perceive the danger; these prophets must be entrusted with authority; and they shall deliver salvation to the world.",[1201],"ic4nefLNk9H9Gv4XmHzEhaM5MJQrZtlLV1DYbwKHs0c",{"id":1525,"title":1526,"body":1527,"description":1169,"draft":1188,"enableComment":1189,"extension":1190,"image":1169,"important":1188,"location":1191,"meta":10276,"navigation":1189,"ogImage":1191,"onday":10277,"path":10278,"seo":10279,"stem":10280,"summary":10281,"tags":10282,"__hash__":10285},"blog\u002Fblog\u002F2026\u002F2026-06-08-explanation-of-neural-network-from-maximum-likelihood-estimation.md","Explanation of Neural Network From Maximum Likelihood Estimation",{"type":8,"value":1528,"toc":10271},[1529,1533,1536,1539,1544,1576,1579,1635,1638,1806,1841,1897,2114,2196,2252,2255,2361,2364,2368,2371,2617,2650,2657,2752,2758,3058,3168,3175,3425,3428,3700,3732,3735,4064,4067,4070,4184,4190,4303,4307,4483,4651,4654,4876,4879,5187,5190,5554,5557,5923,5955,6398,6401,6741,6777,7166,7371,7462,7465,7565,7600,8011,8014,8182,8185,8800,8803,9329,9361,9364,9762,9765,10223,10255,10258,10265],[11,1530,1532],{"id":1531},"coin-tossing","Coin Tossing",[16,1534,1535],{},"MLE (Maximum Likelihood Estimation) is one of the core ideas in statistics.",[16,1537,1538],{},"One-sentence summary:",[1227,1540,1541],{},[16,1542,1543],{},"MLE selects, from all possible model parameters, the one that \"makes the observed data most likely.\"",[16,1545,1546,1547,1575],{},"Below, you find a damaged coin with uneven texture. So you don't know if it is fair. That is, when you toss this coin, you don't know the probability of heads or tails. You want to know: what is the probability ",[45,1548,1550,1563],{"className":1549,"translate":49},[48],[45,1551,1553],{"className":1552},[53],[55,1554,1555],{"xmlns":57},[59,1556,1557,1561],{},[62,1558,1559],{},[65,1560,16],{},[69,1562,16],{"encoding":71},[45,1564,1566],{"className":1565,"ariaHidden":76},[75],[45,1567,1569,1572],{"className":1568},[80],[45,1570],{"className":1571,"style":117},[84],[45,1573,16],{"className":1574},[89,90]," of getting heads.",[16,1577,1578],{},"Thus, you decide to toss it 10 times consecutively and obtain the following result: heads 8 times, tails 2 times.",[16,1580,1581,1582,671],{},"Suppose this were a fair coin. Then we set the prior hypothesis ",[45,1583,1585,1604],{"className":1584,"translate":49},[48],[45,1586,1588],{"className":1587},[53],[55,1589,1590],{"xmlns":57},[59,1591,1592,1601],{},[62,1593,1594,1596,1598],{},[65,1595,16],{},[169,1597,359],{},[182,1599,1600],{},"0.5",[69,1602,1603],{"encoding":71},"p=0.5",[45,1605,1607,1625],{"className":1606,"ariaHidden":76},[75],[45,1608,1610,1613,1616,1619,1622],{"className":1609},[80],[45,1611],{"className":1612,"style":117},[84],[45,1614,16],{"className":1615},[89,90],[45,1617],{"className":1618,"style":233},[232],[45,1620,359],{"className":1621},[237],[45,1623],{"className":1624,"style":233},[232],[45,1626,1628,1632],{"className":1627},[80],[45,1629],{"className":1630,"style":1631},[84],"height:0.6444em;",[45,1633,1600],{"className":1634},[89],[16,1636,1637],{},"Under this prior hypothesis, the probability of obtaining the above result is:",[45,1639,1641],{"className":1640,"translate":49},[327],[45,1642,1644,1689],{"className":1643,"translate":49},[48],[45,1645,1647],{"className":1646},[53],[55,1648,1649],{"xmlns":57,"display":336},[59,1650,1651,1686],{},[62,1652,1653,1655,1657,1660,1664,1666,1668,1670,1672,1674,1681,1683],{},[65,1654,346],{},[169,1656,172],{"stretchy":171},[65,1658,1659],{},"D",[65,1661,1663],{"mathvariant":1662},"normal","∣",[65,1665,16],{},[169,1667,359],{},[182,1669,1600],{},[169,1671,177],{"stretchy":171},[169,1673,359],{},[191,1675,1676,1678],{},[182,1677,1600],{},[182,1679,1680],{},"10",[169,1682,180],{},[182,1684,1685],{},"0.000976",[69,1687,1688],{"encoding":71},"P(D|p=0.5)\n=\n0.5^{10}\n\\approx 0.000976",[45,1690,1692,1723,1744,1797],{"className":1691,"ariaHidden":76},[75],[45,1693,1695,1698,1701,1704,1708,1711,1714,1717,1720],{"className":1694},[80],[45,1696],{"className":1697,"style":214},[84],[45,1699,346],{"className":1700,"style":390},[89,90],[45,1702,172],{"className":1703},[221],[45,1705,1659],{"className":1706,"style":1707},[89,90],"margin-right:0.0278em;",[45,1709,1663],{"className":1710},[89],[45,1712,16],{"className":1713},[89,90],[45,1715],{"className":1716,"style":233},[232],[45,1718,359],{"className":1719},[237],[45,1721],{"className":1722,"style":233},[232],[45,1724,1726,1729,1732,1735,1738,1741],{"className":1725},[80],[45,1727],{"className":1728,"style":214},[84],[45,1730,1600],{"className":1731},[89],[45,1733,177],{"className":1734},[228],[45,1736],{"className":1737,"style":233},[232],[45,1739,359],{"className":1740},[237],[45,1742],{"className":1743,"style":233},[232],[45,1745,1747,1751,1755,1788,1791,1794],{"className":1746},[80],[45,1748],{"className":1749,"style":1750},[84],"height:0.8641em;",[45,1752,1754],{"className":1753},[89],"0.",[45,1756,1758,1762],{"className":1757},[89],[45,1759,1761],{"className":1760},[89],"5",[45,1763,1765],{"className":1764},[281],[45,1766,1768],{"className":1767},[285],[45,1769,1771],{"className":1770},[289],[45,1772,1774],{"className":1773,"style":1750},[293],[45,1775,1776,1779],{"style":492},[45,1777],{"className":1778,"style":301},[300],[45,1780,1782],{"className":1781},[305,306,307,308],[45,1783,1785],{"className":1784},[89,308],[45,1786,1680],{"className":1787},[89,308],[45,1789],{"className":1790,"style":233},[232],[45,1792,180],{"className":1793},[237],[45,1795],{"className":1796,"style":233},[232],[45,1798,1800,1803],{"className":1799},[80],[45,1801],{"className":1802,"style":1631},[84],[45,1804,1685],{"className":1805},[89],[16,1807,1808,1809,1837,1838],{},"That is, if this were a fair coin, then the probability of getting 8 heads and 2 tails in 10 tosses is about ",[45,1810,1812,1825],{"className":1811,"translate":49},[48],[45,1813,1815],{"className":1814},[53],[55,1816,1817],{"xmlns":57},[59,1818,1819,1823],{},[62,1820,1821],{},[182,1822,1685],{},[69,1824,1685],{"encoding":71},[45,1826,1828],{"className":1827,"ariaHidden":76},[75],[45,1829,1831,1834],{"className":1830},[80],[45,1832],{"className":1833,"style":1631},[84],[45,1835,1685],{"className":1836},[89]," — ",[668,1839,1840],{},"clearly, this number is considered impossible in probability and statistics. So you can clearly feel that this coin is not fair.",[16,1842,1843,1844,1896],{},"Now we propose another hypothesis: suppose the probability of heads is ",[45,1845,1847,1866],{"className":1846,"translate":49},[48],[45,1848,1850],{"className":1849},[53],[55,1851,1852],{"xmlns":57},[59,1853,1854,1863],{},[62,1855,1856,1858,1860],{},[65,1857,16],{},[169,1859,359],{},[182,1861,1862],{},"0.8",[69,1864,1865],{"encoding":71},"p=0.8",[45,1867,1869,1887],{"className":1868,"ariaHidden":76},[75],[45,1870,1872,1875,1878,1881,1884],{"className":1871},[80],[45,1873],{"className":1874,"style":117},[84],[45,1876,16],{"className":1877},[89,90],[45,1879],{"className":1880,"style":233},[232],[45,1882,359],{"className":1883},[237],[45,1885],{"className":1886,"style":233},[232],[45,1888,1890,1893],{"className":1889},[80],[45,1891],{"className":1892,"style":1631},[84],[45,1894,1862],{"className":1895},[89],". Then the probability of the above result is:",[45,1898,1900],{"className":1899,"translate":49},[327],[45,1901,1903,1956],{"className":1902,"translate":49},[48],[45,1904,1906],{"className":1905},[53],[55,1907,1908],{"xmlns":57,"display":336},[59,1909,1910,1953],{},[62,1911,1912,1914,1916,1918,1920,1922,1924,1926,1928,1930,1937,1940,1948,1950],{},[65,1913,346],{},[169,1915,172],{"stretchy":171},[65,1917,1659],{},[65,1919,1663],{"mathvariant":1662},[65,1921,16],{},[169,1923,359],{},[182,1925,1862],{},[169,1927,177],{"stretchy":171},[169,1929,359],{},[191,1931,1932,1934],{},[182,1933,1862],{},[182,1935,1936],{},"8",[169,1938,1939],{},"×",[191,1941,1942,1945],{},[182,1943,1944],{},"0.2",[182,1946,1947],{},"2",[169,1949,180],{},[182,1951,1952],{},"0.0067",[69,1954,1955],{"encoding":71},"P(D|p=0.8)\n=\n0.8^8\n\\times\n0.2^2 \\approx 0.0067",[45,1957,1959,1989,2010,2058,2105],{"className":1958,"ariaHidden":76},[75],[45,1960,1962,1965,1968,1971,1974,1977,1980,1983,1986],{"className":1961},[80],[45,1963],{"className":1964,"style":214},[84],[45,1966,346],{"className":1967,"style":390},[89,90],[45,1969,172],{"className":1970},[221],[45,1972,1659],{"className":1973,"style":1707},[89,90],[45,1975,1663],{"className":1976},[89],[45,1978,16],{"className":1979},[89,90],[45,1981],{"className":1982,"style":233},[232],[45,1984,359],{"className":1985},[237],[45,1987],{"className":1988,"style":233},[232],[45,1990,1992,1995,1998,2001,2004,2007],{"className":1991},[80],[45,1993],{"className":1994,"style":214},[84],[45,1996,1862],{"className":1997},[89],[45,1999,177],{"className":2000},[228],[45,2002],{"className":2003,"style":233},[232],[45,2005,359],{"className":2006},[237],[45,2008],{"className":2009,"style":233},[232],[45,2011,2013,2017,2020,2049,2052,2055],{"className":2012},[80],[45,2014],{"className":2015,"style":2016},[84],"height:0.9474em;vertical-align:-0.0833em;",[45,2018,1754],{"className":2019},[89],[45,2021,2023,2026],{"className":2022},[89],[45,2024,1936],{"className":2025},[89],[45,2027,2029],{"className":2028},[281],[45,2030,2032],{"className":2031},[285],[45,2033,2035],{"className":2034},[289],[45,2036,2038],{"className":2037,"style":1750},[293],[45,2039,2040,2043],{"style":492},[45,2041],{"className":2042,"style":301},[300],[45,2044,2046],{"className":2045},[305,306,307,308],[45,2047,1936],{"className":2048},[89,308],[45,2050],{"className":2051,"style":254},[232],[45,2053,1939],{"className":2054},[258],[45,2056],{"className":2057,"style":254},[232],[45,2059,2061,2064,2067,2096,2099,2102],{"className":2060},[80],[45,2062],{"className":2063,"style":1750},[84],[45,2065,1754],{"className":2066},[89],[45,2068,2070,2073],{"className":2069},[89],[45,2071,1947],{"className":2072},[89],[45,2074,2076],{"className":2075},[281],[45,2077,2079],{"className":2078},[285],[45,2080,2082],{"className":2081},[289],[45,2083,2085],{"className":2084,"style":1750},[293],[45,2086,2087,2090],{"style":492},[45,2088],{"className":2089,"style":301},[300],[45,2091,2093],{"className":2092},[305,306,307,308],[45,2094,1947],{"className":2095},[89,308],[45,2097],{"className":2098,"style":233},[232],[45,2100,180],{"className":2101},[237],[45,2103],{"className":2104,"style":233},[232],[45,2106,2108,2111],{"className":2107},[80],[45,2109],{"className":2110,"style":1631},[84],[45,2112,1952],{"className":2113},[89],[16,2115,2116,2117,2167,2168,671],{},"We also find that when ",[45,2118,2120,2137],{"className":2119,"translate":49},[48],[45,2121,2123],{"className":2122},[53],[55,2124,2125],{"xmlns":57},[59,2126,2127,2135],{},[62,2128,2129,2131,2133],{},[65,2130,16],{},[169,2132,359],{},[182,2134,1862],{},[69,2136,1865],{"encoding":71},[45,2138,2140,2158],{"className":2139,"ariaHidden":76},[75],[45,2141,2143,2146,2149,2152,2155],{"className":2142},[80],[45,2144],{"className":2145,"style":117},[84],[45,2147,16],{"className":2148},[89,90],[45,2150],{"className":2151,"style":233},[232],[45,2153,359],{"className":2154},[237],[45,2156],{"className":2157,"style":233},[232],[45,2159,2161,2164],{"className":2160},[80],[45,2162],{"className":2163,"style":1631},[84],[45,2165,1862],{"className":2166},[89],", the probability of the above result is maximized, achieving the maximum value ",[45,2169,2171,2184],{"className":2170,"translate":49},[48],[45,2172,2174],{"className":2173},[53],[55,2175,2176],{"xmlns":57},[59,2177,2178,2182],{},[62,2179,2180],{},[182,2181,1952],{},[69,2183,1952],{"encoding":71},[45,2185,2187],{"className":2186,"ariaHidden":76},[75],[45,2188,2190,2193],{"className":2189},[80],[45,2191],{"className":2192,"style":1631},[84],[45,2194,1952],{"className":2195},[89],[45,2197,2199],{"className":2198,"translate":49},[327],[45,2200,2202,2221],{"className":2201,"translate":49},[48],[45,2203,2205],{"className":2204},[53],[55,2206,2207],{"xmlns":57,"display":336},[59,2208,2209,2218],{},[62,2210,2211,2213,2216],{},[182,2212,1952],{},[169,2214,2215],{},">",[182,2217,1685],{},[69,2219,2220],{"encoding":71},"0.0067\n>\n0.000976",[45,2222,2224,2243],{"className":2223,"ariaHidden":76},[75],[45,2225,2227,2231,2234,2237,2240],{"className":2226},[80],[45,2228],{"className":2229,"style":2230},[84],"height:0.6835em;vertical-align:-0.0391em;",[45,2232,1952],{"className":2233},[89],[45,2235],{"className":2236,"style":233},[232],[45,2238,2215],{"className":2239},[237],[45,2241],{"className":2242,"style":233},[232],[45,2244,2246,2249],{"className":2245},[80],[45,2247],{"className":2248,"style":1631},[84],[45,2250,1685],{"className":2251},[89],[16,2253,2254],{},"This shows that if the probability of heads is 0.8, then the likelihood of observing this data is greatest.",[16,2256,2257,2258],{},"Thus: ",[45,2259,2261,2285],{"className":2260,"translate":49},[48],[45,2262,2264],{"className":2263},[53],[55,2265,2266],{"xmlns":57},[59,2267,2268,2282],{},[62,2269,2270,2278,2280],{},[2271,2272,2273,2275],"mover",{"accent":76},[65,2274,16],{},[169,2276,2277],{},"^",[169,2279,359],{},[182,2281,1862],{},[69,2283,2284],{"encoding":71},"\\hat p=0.8",[45,2286,2288,2352],{"className":2287,"ariaHidden":76},[75],[45,2289,2291,2294,2343,2346,2349],{"className":2290},[80],[45,2292],{"className":2293,"style":925},[84],[45,2295,2298],{"className":2296},[89,2297],"accent",[45,2299,2301,2334],{"className":2300},[285,397],[45,2302,2304,2331],{"className":2303},[289],[45,2305,2308,2318],{"className":2306,"style":2307},[293],"height:0.6944em;",[45,2309,2311,2315],{"style":2310},"top:-3em;",[45,2312],{"className":2313,"style":2314},[300],"height:3em;",[45,2316,16],{"className":2317},[89,90],[45,2319,2320,2323],{"style":2310},[45,2321],{"className":2322,"style":2314},[300],[45,2324,2328],{"className":2325,"style":2327},[2326],"accent-body","left:-0.1667em;",[45,2329,2277],{"className":2330},[89],[45,2332,428],{"className":2333},[427],[45,2335,2337],{"className":2336},[289],[45,2338,2341],{"className":2339,"style":2340},[293],"height:0.1944em;",[45,2342],{},[45,2344],{"className":2345,"style":233},[232],[45,2347,359],{"className":2348},[237],[45,2350],{"className":2351,"style":233},[232],[45,2353,2355,2358],{"className":2354},[80],[45,2356],{"className":2357,"style":1631},[84],[45,2359,1862],{"className":2360},[89],[16,2362,2363],{},"This is maximum likelihood estimation.",[11,2365,2367],{"id":2366},"likelihood-function","Likelihood Function",[16,2369,2370],{},"Given data:",[45,2372,2374],{"className":2373,"translate":49},[327],[45,2375,2377,2431],{"className":2376,"translate":49},[48],[45,2378,2380],{"className":2379},[53],[55,2381,2382],{"xmlns":57,"display":336},[59,2383,2384,2428],{},[62,2385,2386,2388,2390,2393,2400,2403,2409,2411,2414,2417,2419,2425],{},[65,2387,1659],{},[169,2389,359],{},[169,2391,2392],{"stretchy":171},"{",[342,2394,2395,2398],{},[65,2396,2397],{},"x",[182,2399,184],{},[169,2401,2402],{"separator":76},",",[342,2404,2405,2407],{},[65,2406,2397],{},[182,2408,1947],{},[169,2410,2402],{"separator":76},[169,2412,2413],{},"⋯",[348,2415,2416],{}," ",[169,2418,2402],{"separator":76},[342,2420,2421,2423],{},[65,2422,2397],{},[65,2424,67],{},[169,2426,2427],{"stretchy":171},"}",[69,2429,2430],{"encoding":71},"D = \\{x_1, x_2, \\cdots, x_n\\}",[45,2432,2434,2452],{"className":2433,"ariaHidden":76},[75],[45,2435,2437,2440,2443,2446,2449],{"className":2436},[80],[45,2438],{"className":2439,"style":148},[84],[45,2441,1659],{"className":2442,"style":1707},[89,90],[45,2444],{"className":2445,"style":233},[232],[45,2447,359],{"className":2448},[237],[45,2450],{"className":2451,"style":233},[232],[45,2453,2455,2458,2461,2503,2507,2511,2551,2554,2557,2561,2564,2567,2570,2573,2614],{"className":2454},[80],[45,2456],{"className":2457,"style":214},[84],[45,2459,2392],{"className":2460},[221],[45,2462,2464,2467],{"className":2463},[89],[45,2465,2397],{"className":2466},[89,90],[45,2468,2470],{"className":2469},[281],[45,2471,2473,2495],{"className":2472},[285,397],[45,2474,2476,2492],{"className":2475},[289],[45,2477,2480],{"className":2478,"style":2479},[293],"height:0.3011em;",[45,2481,2483,2486],{"style":2482},"top:-2.55em;margin-left:0em;margin-right:0.05em;",[45,2484],{"className":2485,"style":301},[300],[45,2487,2489],{"className":2488},[305,306,307,308],[45,2490,184],{"className":2491},[89,308],[45,2493,428],{"className":2494},[427],[45,2496,2498],{"className":2497},[289],[45,2499,2501],{"className":2500,"style":435},[293],[45,2502],{},[45,2504,2402],{"className":2505},[2506],"mpunct",[45,2508],{"className":2509,"style":2510},[232],"margin-right:0.1667em;",[45,2512,2514,2517],{"className":2513},[89],[45,2515,2397],{"className":2516},[89,90],[45,2518,2520],{"className":2519},[281],[45,2521,2523,2543],{"className":2522},[285,397],[45,2524,2526,2540],{"className":2525},[289],[45,2527,2529],{"className":2528,"style":2479},[293],[45,2530,2531,2534],{"style":2482},[45,2532],{"className":2533,"style":301},[300],[45,2535,2537],{"className":2536},[305,306,307,308],[45,2538,1947],{"className":2539},[89,308],[45,2541,428],{"className":2542},[427],[45,2544,2546],{"className":2545},[289],[45,2547,2549],{"className":2548,"style":435},[293],[45,2550],{},[45,2552,2402],{"className":2553},[2506],[45,2555],{"className":2556,"style":2510},[232],[45,2558,2413],{"className":2559},[2560],"minner",[45,2562],{"className":2563,"style":2510},[232],[45,2565],{"className":2566,"style":2510},[232],[45,2568,2402],{"className":2569},[2506],[45,2571],{"className":2572,"style":2510},[232],[45,2574,2576,2579],{"className":2575},[89],[45,2577,2397],{"className":2578},[89,90],[45,2580,2582],{"className":2581},[281],[45,2583,2585,2606],{"className":2584},[285,397],[45,2586,2588,2603],{"className":2587},[289],[45,2589,2592],{"className":2590,"style":2591},[293],"height:0.1514em;",[45,2593,2594,2597],{"style":2482},[45,2595],{"className":2596,"style":301},[300],[45,2598,2600],{"className":2599},[305,306,307,308],[45,2601,67],{"className":2602},[89,90,308],[45,2604,428],{"className":2605},[427],[45,2607,2609],{"className":2608},[289],[45,2610,2612],{"className":2611,"style":435},[293],[45,2613],{},[45,2615,2427],{"className":2616},[228],[16,2618,2619,2620,671],{},"Suppose we have model parameters: ",[45,2621,2623,2638],{"className":2622,"translate":49},[48],[45,2624,2626],{"className":2625},[53],[55,2627,2628],{"xmlns":57},[59,2629,2630,2635],{},[62,2631,2632],{},[65,2633,2634],{},"θ",[69,2636,2637],{"encoding":71},"\\theta",[45,2639,2641],{"className":2640,"ariaHidden":76},[75],[45,2642,2644,2647],{"className":2643},[80],[45,2645],{"className":2646,"style":2307},[84],[45,2648,2634],{"className":2649,"style":1707},[89,90],[16,2651,2652,2653,2656],{},"The ",[668,2654,2655],{},"likelihood function"," is defined as:",[45,2658,2660],{"className":2659,"translate":49},[327],[45,2661,2663,2698],{"className":2662,"translate":49},[48],[45,2664,2666],{"className":2665},[53],[55,2667,2668],{"xmlns":57,"display":336},[59,2669,2670,2695],{},[62,2671,2672,2675,2677,2679,2681,2683,2685,2687,2689,2691,2693],{},[65,2673,2674],{},"L",[169,2676,172],{"stretchy":171},[65,2678,2634],{},[169,2680,177],{"stretchy":171},[169,2682,359],{},[65,2684,346],{},[169,2686,172],{"stretchy":171},[65,2688,1659],{},[65,2690,1663],{"mathvariant":1662},[65,2692,2634],{},[169,2694,177],{"stretchy":171},[69,2696,2697],{"encoding":71},"L(\\theta) = P(D|\\theta)",[45,2699,2701,2728],{"className":2700,"ariaHidden":76},[75],[45,2702,2704,2707,2710,2713,2716,2719,2722,2725],{"className":2703},[80],[45,2705],{"className":2706,"style":214},[84],[45,2708,2674],{"className":2709},[89,90],[45,2711,172],{"className":2712},[221],[45,2714,2634],{"className":2715,"style":1707},[89,90],[45,2717,177],{"className":2718},[228],[45,2720],{"className":2721,"style":233},[232],[45,2723,359],{"className":2724},[237],[45,2726],{"className":2727,"style":233},[232],[45,2729,2731,2734,2737,2740,2743,2746,2749],{"className":2730},[80],[45,2732],{"className":2733,"style":214},[84],[45,2735,346],{"className":2736,"style":390},[89,90],[45,2738,172],{"className":2739},[221],[45,2741,1659],{"className":2742,"style":1707},[89,90],[45,2744,1663],{"className":2745},[89],[45,2747,2634],{"className":2748,"style":1707},[89,90],[45,2750,177],{"className":2751},[228],[16,2753,2754,2757],{},[668,2755,2756],{},"Maximum Likelihood Estimation (MLE)"," is:",[45,2759,2761],{"className":2760,"translate":49},[327],[45,2762,2764,2838],{"className":2763,"translate":49},[48],[45,2765,2767],{"className":2766},[53],[55,2768,2769],{"xmlns":57,"display":336},[59,2770,2771,2835],{},[62,2772,2773,2779,2781,2784,2787,2799,2801,2803,2805,2807,2809,2811,2813,2815,2817,2827,2829,2831,2833],{},[2271,2774,2775,2777],{"accent":76},[65,2776,2634],{},[169,2778,2277],{},[169,2780,359],{},[65,2782,2783],{},"arg",[169,2785,2786],{},"⁡",[2788,2789,2790,2797],"munder",{},[62,2791,2792,2795],{},[65,2793,2794],{},"max",[169,2796,2786],{},[65,2798,2634],{},[65,2800,346],{},[169,2802,172],{"stretchy":171},[65,2804,1659],{},[65,2806,1663],{"mathvariant":1662},[65,2808,2634],{},[169,2810,177],{"stretchy":171},[169,2812,359],{},[65,2814,2783],{},[169,2816,2786],{},[2788,2818,2819,2825],{},[62,2820,2821,2823],{},[65,2822,2794],{},[169,2824,2786],{},[65,2826,2634],{},[65,2828,2674],{},[169,2830,172],{"stretchy":171},[65,2832,2634],{},[169,2834,177],{"stretchy":171},[69,2836,2837],{"encoding":71},"\\hat\\theta = \\arg\\max_\\theta P(D|\\theta) = \\arg\\max_\\theta L(\\theta)",[45,2839,2841,2889,2985],{"className":2840,"ariaHidden":76},[75],[45,2842,2844,2848,2880,2883,2886],{"className":2843},[80],[45,2845],{"className":2846,"style":2847},[84],"height:0.9579em;",[45,2849,2851],{"className":2850},[89,2297],[45,2852,2854],{"className":2853},[285],[45,2855,2857],{"className":2856},[289],[45,2858,2860,2868],{"className":2859,"style":2847},[293],[45,2861,2862,2865],{"style":2310},[45,2863],{"className":2864,"style":2314},[300],[45,2866,2634],{"className":2867,"style":1707},[89,90],[45,2869,2871,2874],{"style":2870},"top:-3.2634em;",[45,2872],{"className":2873,"style":2314},[300],[45,2875,2877],{"className":2876,"style":2327},[2326],[45,2878,2277],{"className":2879},[89],[45,2881],{"className":2882,"style":233},[232],[45,2884,359],{"className":2885},[237],[45,2887],{"className":2888,"style":233},[232],[45,2890,2892,2896,2905,2908,2955,2958,2961,2964,2967,2970,2973,2976,2979,2982],{"className":2891},[80],[45,2893],{"className":2894,"style":2895},[84],"height:1.5021em;vertical-align:-0.7521em;",[45,2897,2900,2901],{"className":2898},[2899],"mop","ar",[45,2902,2904],{"style":2903},"margin-right:0.0139em;","g",[45,2906],{"className":2907,"style":2510},[232],[45,2909,2912],{"className":2910},[2899,2911],"op-limits",[45,2913,2915,2946],{"className":2914},[285,397],[45,2916,2918,2943],{"className":2917},[289],[45,2919,2921,2933],{"className":2920,"style":85},[293],[45,2922,2924,2927],{"style":2923},"top:-2.3479em;margin-left:0em;",[45,2925],{"className":2926,"style":2314},[300],[45,2928,2930],{"className":2929},[305,306,307,308],[45,2931,2634],{"className":2932,"style":1707},[89,90,308],[45,2934,2935,2938],{"style":2310},[45,2936],{"className":2937,"style":2314},[300],[45,2939,2940],{},[45,2941,2794],{"className":2942},[2899],[45,2944,428],{"className":2945},[427],[45,2947,2949],{"className":2948},[289],[45,2950,2953],{"className":2951,"style":2952},[293],"height:0.7521em;",[45,2954],{},[45,2956],{"className":2957,"style":2510},[232],[45,2959,346],{"className":2960,"style":390},[89,90],[45,2962,172],{"className":2963},[221],[45,2965,1659],{"className":2966,"style":1707},[89,90],[45,2968,1663],{"className":2969},[89],[45,2971,2634],{"className":2972,"style":1707},[89,90],[45,2974,177],{"className":2975},[228],[45,2977],{"className":2978,"style":233},[232],[45,2980,359],{"className":2981},[237],[45,2983],{"className":2984,"style":233},[232],[45,2986,2988,2991,2996,2999,3043,3046,3049,3052,3055],{"className":2987},[80],[45,2989],{"className":2990,"style":2895},[84],[45,2992,2900,2994],{"className":2993},[2899],[45,2995,2904],{"style":2903},[45,2997],{"className":2998,"style":2510},[232],[45,3000,3002],{"className":3001},[2899,2911],[45,3003,3005,3035],{"className":3004},[285,397],[45,3006,3008,3032],{"className":3007},[289],[45,3009,3011,3022],{"className":3010,"style":85},[293],[45,3012,3013,3016],{"style":2923},[45,3014],{"className":3015,"style":2314},[300],[45,3017,3019],{"className":3018},[305,306,307,308],[45,3020,2634],{"className":3021,"style":1707},[89,90,308],[45,3023,3024,3027],{"style":2310},[45,3025],{"className":3026,"style":2314},[300],[45,3028,3029],{},[45,3030,2794],{"className":3031},[2899],[45,3033,428],{"className":3034},[427],[45,3036,3038],{"className":3037},[289],[45,3039,3041],{"className":3040,"style":2952},[293],[45,3042],{},[45,3044],{"className":3045,"style":2510},[232],[45,3047,2674],{"className":3048},[89,90],[45,3050,172],{"className":3051},[221],[45,3053,2634],{"className":3054,"style":1707},[89,90],[45,3056,177],{"className":3057},[228],[16,3059,3060,3061,3064,3065,1348,3119,3163,3164,3167],{},"Many people feel confused when first learning this:",[3062,3063],"br",{},"\nSince ",[45,3066,3068,3092],{"className":3067,"translate":49},[48],[45,3069,3071],{"className":3070},[53],[55,3072,3073],{"xmlns":57},[59,3074,3075,3089],{},[62,3076,3077,3079,3081,3083,3085,3087],{},[65,3078,346],{},[169,3080,172],{"stretchy":171},[65,3082,1659],{},[65,3084,1663],{"mathvariant":1662},[65,3086,2634],{},[169,3088,177],{"stretchy":171},[69,3090,3091],{"encoding":71},"P(D|\\theta)",[45,3093,3095],{"className":3094,"ariaHidden":76},[75],[45,3096,3098,3101,3104,3107,3110,3113,3116],{"className":3097},[80],[45,3099],{"className":3100,"style":214},[84],[45,3102,346],{"className":3103,"style":390},[89,90],[45,3105,172],{"className":3106},[221],[45,3108,1659],{"className":3109,"style":1707},[89,90],[45,3111,1663],{"className":3112},[89],[45,3114,2634],{"className":3115,"style":1707},[89,90],[45,3117,177],{"className":3118},[228],[45,3120,3122,3142],{"className":3121,"translate":49},[48],[45,3123,3125],{"className":3124},[53],[55,3126,3127],{"xmlns":57},[59,3128,3129,3139],{},[62,3130,3131,3133,3135,3137],{},[65,3132,2674],{},[169,3134,172],{"stretchy":171},[65,3136,2634],{},[169,3138,177],{"stretchy":171},[69,3140,3141],{"encoding":71},"L(\\theta)",[45,3143,3145],{"className":3144,"ariaHidden":76},[75],[45,3146,3148,3151,3154,3157,3160],{"className":3147},[80],[45,3149],{"className":3150,"style":214},[84],[45,3152,2674],{"className":3153},[89,90],[45,3155,172],{"className":3156},[221],[45,3158,2634],{"className":3159,"style":1707},[89,90],[45,3161,177],{"className":3162},[228]," have ",[668,3165,3166],{},"exactly the same expression",", why change the name?",[16,3169,3170,3171,3174],{},"The key lies in ",[668,3172,3173],{},"different perspectives",":",[1289,3176,3177,3308],{},[1292,3178,3179,3182,3183,3236,3237,3239,3240,3268,3269,3272,3273,3301,3302,671,3305,3307],{},[668,3180,3181],{},"Probability"," ",[45,3184,3186,3209],{"className":3185,"translate":49},[48],[45,3187,3189],{"className":3188},[53],[55,3190,3191],{"xmlns":57},[59,3192,3193,3207],{},[62,3194,3195,3197,3199,3201,3203,3205],{},[65,3196,346],{},[169,3198,172],{"stretchy":171},[65,3200,1659],{},[65,3202,1663],{"mathvariant":1662},[65,3204,2634],{},[169,3206,177],{"stretchy":171},[69,3208,3091],{"encoding":71},[45,3210,3212],{"className":3211,"ariaHidden":76},[75],[45,3213,3215,3218,3221,3224,3227,3230,3233],{"className":3214},[80],[45,3216],{"className":3217,"style":214},[84],[45,3219,346],{"className":3220,"style":390},[89,90],[45,3222,172],{"className":3223},[221],[45,3225,1659],{"className":3226,"style":1707},[89,90],[45,3228,1663],{"className":3229},[89],[45,3231,2634],{"className":3232,"style":1707},[89,90],[45,3234,177],{"className":3235},[228]," perspective:",[3062,3238],{},"\nThe parameter ",[45,3241,3243,3256],{"className":3242,"translate":49},[48],[45,3244,3246],{"className":3245},[53],[55,3247,3248],{"xmlns":57},[59,3249,3250,3254],{},[62,3251,3252],{},[65,3253,2634],{},[69,3255,2637],{"encoding":71},[45,3257,3259],{"className":3258,"ariaHidden":76},[75],[45,3260,3262,3265],{"className":3261},[80],[45,3263],{"className":3264,"style":2307},[84],[45,3266,2634],{"className":3267,"style":1707},[89,90]," is a ",[668,3270,3271],{},"fixed known value",", while the data ",[45,3274,3276,3289],{"className":3275,"translate":49},[48],[45,3277,3279],{"className":3278},[53],[55,3280,3281],{"xmlns":57},[59,3282,3283,3287],{},[62,3284,3285],{},[65,3286,1659],{},[69,3288,1659],{"encoding":71},[45,3290,3292],{"className":3291,"ariaHidden":76},[75],[45,3293,3295,3298],{"className":3294},[80],[45,3296],{"className":3297,"style":148},[84],[45,3299,1659],{"className":3300,"style":1707},[89,90]," is ",[668,3303,3304],{},"random",[3062,3306],{},"\nWe ask: \"Given the parameter, how likely are different data outcomes?\"",[1292,3309,3310,3182,3313,3236,3356,3358,3359,3268,3387,3390,3391,3268,3419,671,3422,3424],{},[668,3311,3312],{},"Likelihood",[45,3314,3316,3335],{"className":3315,"translate":49},[48],[45,3317,3319],{"className":3318},[53],[55,3320,3321],{"xmlns":57},[59,3322,3323,3333],{},[62,3324,3325,3327,3329,3331],{},[65,3326,2674],{},[169,3328,172],{"stretchy":171},[65,3330,2634],{},[169,3332,177],{"stretchy":171},[69,3334,3141],{"encoding":71},[45,3336,3338],{"className":3337,"ariaHidden":76},[75],[45,3339,3341,3344,3347,3350,3353],{"className":3340},[80],[45,3342],{"className":3343,"style":214},[84],[45,3345,2674],{"className":3346},[89,90],[45,3348,172],{"className":3349},[221],[45,3351,2634],{"className":3352,"style":1707},[89,90],[45,3354,177],{"className":3355},[228],[3062,3357],{},"\nThe data ",[45,3360,3362,3375],{"className":3361,"translate":49},[48],[45,3363,3365],{"className":3364},[53],[55,3366,3367],{"xmlns":57},[59,3368,3369,3373],{},[62,3370,3371],{},[65,3372,1659],{},[69,3374,1659],{"encoding":71},[45,3376,3378],{"className":3377,"ariaHidden":76},[75],[45,3379,3381,3384],{"className":3380},[80],[45,3382],{"className":3383,"style":148},[84],[45,3385,1659],{"className":3386,"style":1707},[89,90],[668,3388,3389],{},"fixed observed value",", while the parameter ",[45,3392,3394,3407],{"className":3393,"translate":49},[48],[45,3395,3397],{"className":3396},[53],[55,3398,3399],{"xmlns":57},[59,3400,3401,3405],{},[62,3402,3403],{},[65,3404,2634],{},[69,3406,2637],{"encoding":71},[45,3408,3410],{"className":3409,"ariaHidden":76},[75],[45,3411,3413,3416],{"className":3412},[80],[45,3414],{"className":3415,"style":2307},[84],[45,3417,2634],{"className":3418,"style":1707},[89,90],[668,3420,3421],{},"varying unknown quantity",[3062,3423],{},"\nWe ask: \"Under different parameter values, how likely is this observed data?\"",[16,3426,3427],{},"According to the conditional formula:",[45,3429,3431],{"className":3430,"translate":49},[327],[45,3432,3434,3495],{"className":3433,"translate":49},[48],[45,3435,3437],{"className":3436},[53],[55,3438,3439],{"xmlns":57,"display":336},[59,3440,3441,3492],{},[62,3442,3443,3445,3447,3449,3451,3453,3455,3457,3476,3478,3480,3486,3488,3490],{},[65,3444,346],{},[169,3446,172],{"stretchy":171},[65,3448,1659],{},[65,3450,1663],{"mathvariant":1662},[65,3452,2634],{},[169,3454,177],{"stretchy":171},[169,3456,359],{},[2788,3458,3459,3462],{},[169,3460,3461],{},"∏",[62,3463,3464,3471,3474],{},[342,3465,3466,3468],{},[65,3467,2397],{},[65,3469,3470],{},"i",[169,3472,3473],{},"∈",[65,3475,1659],{},[65,3477,346],{},[169,3479,172],{"stretchy":171},[342,3481,3482,3484],{},[65,3483,2397],{},[65,3485,3470],{},[65,3487,1663],{"mathvariant":1662},[65,3489,2634],{},[169,3491,177],{"stretchy":171},[69,3493,3494],{"encoding":71},"P(D|\\theta)\n=\n\\prod_{x_i \\in D}\nP(x_i|\\theta)",[45,3496,3498,3531],{"className":3497,"ariaHidden":76},[75],[45,3499,3501,3504,3507,3510,3513,3516,3519,3522,3525,3528],{"className":3500},[80],[45,3502],{"className":3503,"style":214},[84],[45,3505,346],{"className":3506,"style":390},[89,90],[45,3508,172],{"className":3509},[221],[45,3511,1659],{"className":3512,"style":1707},[89,90],[45,3514,1663],{"className":3515},[89],[45,3517,2634],{"className":3518,"style":1707},[89,90],[45,3520,177],{"className":3521},[228],[45,3523],{"className":3524,"style":233},[232],[45,3526,359],{"className":3527},[237],[45,3529],{"className":3530,"style":233},[232],[45,3532,3534,3538,3641,3644,3647,3650,3691,3694,3697],{"className":3533},[80],[45,3535],{"className":3536,"style":3537},[84],"height:2.4444em;vertical-align:-1.3944em;",[45,3539,3541],{"className":3540},[2899,2911],[45,3542,3544,3632],{"className":3543},[285,397],[45,3545,3547,3629],{"className":3546},[289],[45,3548,3551,3616],{"className":3549,"style":3550},[293],"height:1.05em;",[45,3552,3554,3558],{"style":3553},"top:-1.8557em;margin-left:0em;",[45,3555],{"className":3556,"style":3557},[300],"height:3.05em;",[45,3559,3561],{"className":3560},[305,306,307,308],[45,3562,3564,3610,3613],{"className":3563},[89,308],[45,3565,3567,3570],{"className":3566},[89,308],[45,3568,2397],{"className":3569},[89,90,308],[45,3571,3573],{"className":3572},[281],[45,3574,3576,3601],{"className":3575},[285,397],[45,3577,3579,3598],{"className":3578},[289],[45,3580,3583],{"className":3581,"style":3582},[293],"height:0.3281em;",[45,3584,3586,3590],{"style":3585},"top:-2.357em;margin-left:0em;margin-right:0.0714em;",[45,3587],{"className":3588,"style":3589},[300],"height:2.5em;",[45,3591,3595],{"className":3592},[305,3593,3594,308],"reset-size3","size1",[45,3596,3470],{"className":3597},[89,90,308],[45,3599,428],{"className":3600},[427],[45,3602,3604],{"className":3603},[289],[45,3605,3608],{"className":3606,"style":3607},[293],"height:0.143em;",[45,3609],{},[45,3611,3473],{"className":3612},[237,308],[45,3614,1659],{"className":3615,"style":1707},[89,90,308],[45,3617,3619,3622],{"style":3618},"top:-3.05em;",[45,3620],{"className":3621,"style":3557},[300],[45,3623,3624],{},[45,3625,3461],{"className":3626},[2899,3627,3628],"op-symbol","large-op",[45,3630,428],{"className":3631},[427],[45,3633,3635],{"className":3634},[289],[45,3636,3639],{"className":3637,"style":3638},[293],"height:1.3944em;",[45,3640],{},[45,3642],{"className":3643,"style":2510},[232],[45,3645,346],{"className":3646,"style":390},[89,90],[45,3648,172],{"className":3649},[221],[45,3651,3653,3656],{"className":3652},[89],[45,3654,2397],{"className":3655},[89,90],[45,3657,3659],{"className":3658},[281],[45,3660,3662,3683],{"className":3661},[285,397],[45,3663,3665,3680],{"className":3664},[289],[45,3666,3669],{"className":3667,"style":3668},[293],"height:0.3117em;",[45,3670,3671,3674],{"style":2482},[45,3672],{"className":3673,"style":301},[300],[45,3675,3677],{"className":3676},[305,306,307,308],[45,3678,3470],{"className":3679},[89,90,308],[45,3681,428],{"className":3682},[427],[45,3684,3686],{"className":3685},[289],[45,3687,3689],{"className":3688,"style":435},[293],[45,3690],{},[45,3692,1663],{"className":3693},[89],[45,3695,2634],{"className":3696,"style":1707},[89,90],[45,3698,177],{"className":3699},[228],[16,3701,3702,3703,3731],{},"If the data ",[45,3704,3706,3719],{"className":3705,"translate":49},[48],[45,3707,3709],{"className":3708},[53],[55,3710,3711],{"xmlns":57},[59,3712,3713,3717],{},[62,3714,3715],{},[65,3716,1659],{},[69,3718,1659],{"encoding":71},[45,3720,3722],{"className":3721,"ariaHidden":76},[75],[45,3723,3725,3728],{"className":3724},[80],[45,3726],{"className":3727,"style":148},[84],[45,3729,1659],{"className":3730,"style":1707},[89,90]," contains many samples, multiplying many small numbers yields a result approaching zero, which can cause numerical underflow in computers.",[16,3733,3734],{},"Thus we take the logarithm, converting multiplication into addition:",[45,3736,3738],{"className":3737,"translate":49},[327],[45,3739,3741,3824],{"className":3740,"translate":49},[48],[45,3742,3744],{"className":3743},[53],[55,3745,3746],{"xmlns":57,"display":336},[59,3747,3748,3821],{},[62,3749,3750,3753,3755,3757,3759,3761,3763,3765,3768,3770,3772,3774,3776,3778,3780,3782,3784,3801,3803,3805,3807,3809,3815,3817,3819],{},[65,3751,3752],{"mathvariant":1662},"ℓ",[169,3754,172],{"stretchy":171},[65,3756,2634],{},[65,3758,1663],{"mathvariant":1662},[65,3760,1659],{},[169,3762,177],{"stretchy":171},[169,3764,359],{},[65,3766,3767],{},"log",[169,3769,2786],{},[65,3771,346],{},[169,3773,172],{"stretchy":171},[65,3775,1659],{},[65,3777,1663],{"mathvariant":1662},[65,3779,2634],{},[169,3781,177],{"stretchy":171},[169,3783,359],{},[2788,3785,3786,3789],{},[169,3787,3788],{},"∑",[62,3790,3791,3797,3799],{},[342,3792,3793,3795],{},[65,3794,2397],{},[65,3796,3470],{},[169,3798,3473],{},[65,3800,1659],{},[65,3802,3767],{},[169,3804,2786],{},[65,3806,346],{},[169,3808,172],{"stretchy":171},[342,3810,3811,3813],{},[65,3812,2397],{},[65,3814,3470],{},[65,3816,1663],{"mathvariant":1662},[65,3818,2634],{},[169,3820,177],{"stretchy":171},[69,3822,3823],{"encoding":71},"\\ell(\\theta|D)=\n\\log P(D|\\theta)\n=\n\\sum_{x_i \\in D}\n\\log P(x_i|\\theta)",[45,3825,3827,3860,3902],{"className":3826,"ariaHidden":76},[75],[45,3828,3830,3833,3836,3839,3842,3845,3848,3851,3854,3857],{"className":3829},[80],[45,3831],{"className":3832,"style":214},[84],[45,3834,3752],{"className":3835},[89],[45,3837,172],{"className":3838},[221],[45,3840,2634],{"className":3841,"style":1707},[89,90],[45,3843,1663],{"className":3844},[89],[45,3846,1659],{"className":3847,"style":1707},[89,90],[45,3849,177],{"className":3850},[228],[45,3852],{"className":3853,"style":233},[232],[45,3855,359],{"className":3856},[237],[45,3858],{"className":3859,"style":233},[232],[45,3861,3863,3866,3872,3875,3878,3881,3884,3887,3890,3893,3896,3899],{"className":3862},[80],[45,3864],{"className":3865,"style":214},[84],[45,3867,3869,3870],{"className":3868},[2899],"lo",[45,3871,2904],{"style":2903},[45,3873],{"className":3874,"style":2510},[232],[45,3876,346],{"className":3877,"style":390},[89,90],[45,3879,172],{"className":3880},[221],[45,3882,1659],{"className":3883,"style":1707},[89,90],[45,3885,1663],{"className":3886},[89],[45,3888,2634],{"className":3889,"style":1707},[89,90],[45,3891,177],{"className":3892},[228],[45,3894],{"className":3895,"style":233},[232],[45,3897,359],{"className":3898},[237],[45,3900],{"className":3901,"style":233},[232],[45,3903,3905,3908,3998,4001,4006,4009,4012,4015,4055,4058,4061],{"className":3904},[80],[45,3906],{"className":3907,"style":3537},[84],[45,3909,3911],{"className":3910},[2899,2911],[45,3912,3914,3990],{"className":3913},[285,397],[45,3915,3917,3987],{"className":3916},[289],[45,3918,3920,3977],{"className":3919,"style":3550},[293],[45,3921,3922,3925],{"style":3553},[45,3923],{"className":3924,"style":3557},[300],[45,3926,3928],{"className":3927},[305,306,307,308],[45,3929,3931,3971,3974],{"className":3930},[89,308],[45,3932,3934,3937],{"className":3933},[89,308],[45,3935,2397],{"className":3936},[89,90,308],[45,3938,3940],{"className":3939},[281],[45,3941,3943,3963],{"className":3942},[285,397],[45,3944,3946,3960],{"className":3945},[289],[45,3947,3949],{"className":3948,"style":3582},[293],[45,3950,3951,3954],{"style":3585},[45,3952],{"className":3953,"style":3589},[300],[45,3955,3957],{"className":3956},[305,3593,3594,308],[45,3958,3470],{"className":3959},[89,90,308],[45,3961,428],{"className":3962},[427],[45,3964,3966],{"className":3965},[289],[45,3967,3969],{"className":3968,"style":3607},[293],[45,3970],{},[45,3972,3473],{"className":3973},[237,308],[45,3975,1659],{"className":3976,"style":1707},[89,90,308],[45,3978,3979,3982],{"style":3618},[45,3980],{"className":3981,"style":3557},[300],[45,3983,3984],{},[45,3985,3788],{"className":3986},[2899,3627,3628],[45,3988,428],{"className":3989},[427],[45,3991,3993],{"className":3992},[289],[45,3994,3996],{"className":3995,"style":3638},[293],[45,3997],{},[45,3999],{"className":4000,"style":2510},[232],[45,4002,3869,4004],{"className":4003},[2899],[45,4005,2904],{"style":2903},[45,4007],{"className":4008,"style":2510},[232],[45,4010,346],{"className":4011,"style":390},[89,90],[45,4013,172],{"className":4014},[221],[45,4016,4018,4021],{"className":4017},[89],[45,4019,2397],{"className":4020},[89,90],[45,4022,4024],{"className":4023},[281],[45,4025,4027,4047],{"className":4026},[285,397],[45,4028,4030,4044],{"className":4029},[289],[45,4031,4033],{"className":4032,"style":3668},[293],[45,4034,4035,4038],{"style":2482},[45,4036],{"className":4037,"style":301},[300],[45,4039,4041],{"className":4040},[305,306,307,308],[45,4042,3470],{"className":4043},[89,90,308],[45,4045,428],{"className":4046},[427],[45,4048,4050],{"className":4049},[289],[45,4051,4053],{"className":4052,"style":435},[293],[45,4054],{},[45,4056,1663],{"className":4057},[89],[45,4059,2634],{"className":4060,"style":1707},[89,90],[45,4062,177],{"className":4063},[228],[16,4065,4066],{},"After taking the logarithm, the values stay within an acceptable range, making optimization easier.",[16,4068,4069],{},"Therefore, almost all practical training optimizes:",[45,4071,4073],{"className":4072,"translate":49},[327],[45,4074,4076,4110],{"className":4075,"translate":49},[48],[45,4077,4079],{"className":4078},[53],[55,4080,4081],{"xmlns":57,"display":336},[59,4082,4083,4107],{},[62,4084,4085,4095,4097,4099,4101,4103,4105],{},[2788,4086,4087,4093],{},[62,4088,4089,4091],{},[65,4090,2794],{},[169,4092,2786],{},[65,4094,2634],{},[65,4096,3752],{"mathvariant":1662},[169,4098,172],{"stretchy":171},[65,4100,2634],{},[65,4102,1663],{"mathvariant":1662},[65,4104,1659],{},[169,4106,177],{"stretchy":171},[69,4108,4109],{"encoding":71},"\\max_\\theta\n\\ell(\\theta|D)",[45,4111,4113],{"className":4112,"ariaHidden":76},[75],[45,4114,4116,4119,4163,4166,4169,4172,4175,4178,4181],{"className":4115},[80],[45,4117],{"className":4118,"style":2895},[84],[45,4120,4122],{"className":4121},[2899,2911],[45,4123,4125,4155],{"className":4124},[285,397],[45,4126,4128,4152],{"className":4127},[289],[45,4129,4131,4142],{"className":4130,"style":85},[293],[45,4132,4133,4136],{"style":2923},[45,4134],{"className":4135,"style":2314},[300],[45,4137,4139],{"className":4138},[305,306,307,308],[45,4140,2634],{"className":4141,"style":1707},[89,90,308],[45,4143,4144,4147],{"style":2310},[45,4145],{"className":4146,"style":2314},[300],[45,4148,4149],{},[45,4150,2794],{"className":4151},[2899],[45,4153,428],{"className":4154},[427],[45,4156,4158],{"className":4157},[289],[45,4159,4161],{"className":4160,"style":2952},[293],[45,4162],{},[45,4164],{"className":4165,"style":2510},[232],[45,4167,3752],{"className":4168},[89],[45,4170,172],{"className":4171},[221],[45,4173,2634],{"className":4174,"style":1707},[89,90],[45,4176,1663],{"className":4177},[89],[45,4179,1659],{"className":4180,"style":1707},[89,90],[45,4182,177],{"className":4183},[228],[16,4185,4186,4187,671],{},"This form is called ",[668,4188,4189],{},"Log-Likelihood",[16,4191,4192,4193,4302],{},"In the likelihood function, we need to find the parameter ",[45,4194,4196,4219],{"className":4195,"translate":49},[48],[45,4197,4199],{"className":4198},[53],[55,4200,4201],{"xmlns":57},[59,4202,4203,4216],{},[62,4204,4205],{},[342,4206,4207,4213],{},[2271,4208,4209,4211],{"accent":76},[65,4210,2634],{},[169,4212,2277],{},[348,4214,4215],{},"MLE",[69,4217,4218],{"encoding":71},"\\hat{\\theta}_\\text{MLE}",[45,4220,4222],{"className":4221,"ariaHidden":76},[75],[45,4223,4225,4229],{"className":4224},[80],[45,4226],{"className":4227,"style":4228},[84],"height:1.1079em;vertical-align:-0.15em;",[45,4230,4232,4263],{"className":4231},[89],[45,4233,4235],{"className":4234},[89,2297],[45,4236,4238],{"className":4237},[285],[45,4239,4241],{"className":4240},[289],[45,4242,4244,4252],{"className":4243,"style":2847},[293],[45,4245,4246,4249],{"style":2310},[45,4247],{"className":4248,"style":2314},[300],[45,4250,2634],{"className":4251,"style":1707},[89,90],[45,4253,4254,4257],{"style":2870},[45,4255],{"className":4256,"style":2314},[300],[45,4258,4260],{"className":4259,"style":2327},[2326],[45,4261,2277],{"className":4262},[89],[45,4264,4266],{"className":4265},[281],[45,4267,4269,4294],{"className":4268},[285,397],[45,4270,4272,4291],{"className":4271},[289],[45,4273,4276],{"className":4274,"style":4275},[293],"height:0.3283em;",[45,4277,4279,4282],{"style":4278},"top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;",[45,4280],{"className":4281,"style":301},[300],[45,4283,4285],{"className":4284},[305,306,307,308],[45,4286,4288],{"className":4287},[89,420,308],[45,4289,4215],{"className":4290},[89,308],[45,4292,428],{"className":4293},[427],[45,4295,4297],{"className":4296},[289],[45,4298,4300],{"className":4299,"style":435},[293],[45,4301],{}," that maximizes the likelihood.",[11,4304,4306],{"id":4305},"solving-for-parameters","Solving for Parameters",[16,4308,4309,4310,4364,4365,4482],{},"If ",[45,4311,4313,4337],{"className":4312,"translate":49},[48],[45,4314,4316],{"className":4315},[53],[55,4317,4318],{"xmlns":57},[59,4319,4320,4334],{},[62,4321,4322,4324,4326,4328,4330,4332],{},[65,4323,3752],{"mathvariant":1662},[169,4325,172],{"stretchy":171},[65,4327,2634],{},[65,4329,1663],{"mathvariant":1662},[65,4331,1659],{},[169,4333,177],{"stretchy":171},[69,4335,4336],{"encoding":71},"\\ell(\\theta|D)",[45,4338,4340],{"className":4339,"ariaHidden":76},[75],[45,4341,4343,4346,4349,4352,4355,4358,4361],{"className":4342},[80],[45,4344],{"className":4345,"style":214},[84],[45,4347,3752],{"className":4348},[89],[45,4350,172],{"className":4351},[221],[45,4353,2634],{"className":4354,"style":1707},[89,90],[45,4356,1663],{"className":4357},[89],[45,4359,1659],{"className":4360,"style":1707},[89,90],[45,4362,177],{"className":4363},[228]," is differentiable, ",[45,4366,4368,4398],{"className":4367,"translate":49},[48],[45,4369,4371],{"className":4370},[53],[55,4372,4373],{"xmlns":57},[59,4374,4375,4395],{},[62,4376,4377],{},[342,4378,4379,4385],{},[2271,4380,4381,4383],{"accent":76},[65,4382,2634],{},[169,4384,2277],{},[62,4386,4387,4390,4392],{},[65,4388,4389],{"mathvariant":1662},"M",[65,4391,2674],{"mathvariant":1662},[65,4393,4394],{"mathvariant":1662},"E",[69,4396,4397],{"encoding":71},"\\hat{\\theta}_{\\rm MLE}",[45,4399,4401],{"className":4400,"ariaHidden":76},[75],[45,4402,4404,4407],{"className":4403},[80],[45,4405],{"className":4406,"style":4228},[84],[45,4408,4410,4441],{"className":4409},[89],[45,4411,4413],{"className":4412},[89,2297],[45,4414,4416],{"className":4415},[285],[45,4417,4419],{"className":4418},[289],[45,4420,4422,4430],{"className":4421,"style":2847},[293],[45,4423,4424,4427],{"style":2310},[45,4425],{"className":4426,"style":2314},[300],[45,4428,2634],{"className":4429,"style":1707},[89,90],[45,4431,4432,4435],{"style":2870},[45,4433],{"className":4434,"style":2314},[300],[45,4436,4438],{"className":4437,"style":2327},[2326],[45,4439,2277],{"className":4440},[89],[45,4442,4444],{"className":4443},[281],[45,4445,4447,4474],{"className":4446},[285,397],[45,4448,4450,4471],{"className":4449},[289],[45,4451,4453],{"className":4452,"style":4275},[293],[45,4454,4455,4458],{"style":4278},[45,4456],{"className":4457,"style":301},[300],[45,4459,4461],{"className":4460},[305,306,307,308],[45,4462,4464],{"className":4463},[89,308],[45,4465,4467],{"className":4466},[89,308],[45,4468,4215],{"className":4469},[89,4470,308],"mathrm",[45,4472,428],{"className":4473},[427],[45,4475,4477],{"className":4476},[289],[45,4478,4480],{"className":4479,"style":435},[293],[45,4481],{}," typically satisfies:",[45,4484,4486],{"className":4485,"translate":49},[327],[45,4487,4489,4532],{"className":4488,"translate":49},[48],[45,4490,4492],{"className":4491},[53],[55,4493,4494],{"xmlns":57,"display":336},[59,4495,4496,4529],{},[62,4497,4498,4524,4526],{},[4499,4500,4501,4518],"mfrac",{},[62,4502,4503,4506,4508,4510,4512,4514,4516],{},[65,4504,4505],{"mathvariant":1662},"∂",[65,4507,3752],{"mathvariant":1662},[169,4509,172],{"stretchy":171},[65,4511,2634],{},[65,4513,1663],{"mathvariant":1662},[65,4515,1659],{},[169,4517,177],{"stretchy":171},[62,4519,4520,4522],{},[65,4521,4505],{"mathvariant":1662},[65,4523,2634],{},[169,4525,359],{},[182,4527,4528],{},"0",[69,4530,4531],{"encoding":71},"\\frac{\\partial \\ell(\\theta|D)}{\\partial \\theta} = 0",[45,4533,4535,4642],{"className":4534,"ariaHidden":76},[75],[45,4536,4538,4542,4633,4636,4639],{"className":4537},[80],[45,4539],{"className":4540,"style":4541},[84],"height:2.113em;vertical-align:-0.686em;",[45,4543,4545,4549,4630],{"className":4544},[89],[45,4546],{"className":4547},[221,4548],"nulldelimiter",[45,4550,4552],{"className":4551},[4499],[45,4553,4555,4621],{"className":4554},[285,397],[45,4556,4558,4618],{"className":4557},[289],[45,4559,4562,4577,4588],{"className":4560,"style":4561},[293],"height:1.427em;",[45,4563,4565,4568],{"style":4564},"top:-2.314em;",[45,4566],{"className":4567,"style":2314},[300],[45,4569,4571,4574],{"className":4570},[89],[45,4572,4505],{"className":4573,"style":961},[89],[45,4575,2634],{"className":4576,"style":1707},[89,90],[45,4578,4580,4583],{"style":4579},"top:-3.23em;",[45,4581],{"className":4582,"style":2314},[300],[45,4584],{"className":4585,"style":4587},[4586],"frac-line","border-bottom-width:0.04em;",[45,4589,4591,4594],{"style":4590},"top:-3.677em;",[45,4592],{"className":4593,"style":2314},[300],[45,4595,4597,4600,4603,4606,4609,4612,4615],{"className":4596},[89],[45,4598,4505],{"className":4599,"style":961},[89],[45,4601,3752],{"className":4602},[89],[45,4604,172],{"className":4605},[221],[45,4607,2634],{"className":4608,"style":1707},[89,90],[45,4610,1663],{"className":4611},[89],[45,4613,1659],{"className":4614,"style":1707},[89,90],[45,4616,177],{"className":4617},[228],[45,4619,428],{"className":4620},[427],[45,4622,4624],{"className":4623},[289],[45,4625,4628],{"className":4626,"style":4627},[293],"height:0.686em;",[45,4629],{},[45,4631],{"className":4632},[228,4548],[45,4634],{"className":4635,"style":233},[232],[45,4637,359],{"className":4638},[237],[45,4640],{"className":4641,"style":233},[232],[45,4643,4645,4648],{"className":4644},[80],[45,4646],{"className":4647,"style":1631},[84],[45,4649,4528],{"className":4650},[89],[16,4652,4653],{},"And the second-order condition ensures a maximum:",[45,4655,4657],{"className":4656,"translate":49},[327],[45,4658,4660,4709],{"className":4659,"translate":49},[48],[45,4661,4663],{"className":4662},[53],[55,4664,4665],{"xmlns":57,"display":336},[59,4666,4667,4706],{},[62,4668,4669,4701,4704],{},[4499,4670,4671,4691],{},[62,4672,4673,4679,4681,4683,4685,4687,4689],{},[191,4674,4675,4677],{},[65,4676,4505],{"mathvariant":1662},[182,4678,1947],{},[65,4680,3752],{"mathvariant":1662},[169,4682,172],{"stretchy":171},[65,4684,2634],{},[65,4686,1663],{"mathvariant":1662},[65,4688,1659],{},[169,4690,177],{"stretchy":171},[62,4692,4693,4695],{},[65,4694,4505],{"mathvariant":1662},[191,4696,4697,4699],{},[65,4698,2634],{},[182,4700,1947],{},[169,4702,4703],{},"\u003C",[182,4705,4528],{},[69,4707,4708],{"encoding":71},"\\frac{\\partial^2 \\ell(\\theta|D)}{\\partial \\theta^2} \u003C 0",[45,4710,4712,4867],{"className":4711,"ariaHidden":76},[75],[45,4713,4715,4719,4858,4861,4864],{"className":4714},[80],[45,4716],{"className":4717,"style":4718},[84],"height:2.1771em;vertical-align:-0.686em;",[45,4720,4722,4725,4855],{"className":4721},[89],[45,4723],{"className":4724},[221,4548],[45,4726,4728],{"className":4727},[4499],[45,4729,4731,4847],{"className":4730},[285,397],[45,4732,4734,4844],{"className":4733},[289],[45,4735,4738,4780,4788],{"className":4736,"style":4737},[293],"height:1.4911em;",[45,4739,4740,4743],{"style":4564},[45,4741],{"className":4742,"style":2314},[300],[45,4744,4746,4749],{"className":4745},[89],[45,4747,4505],{"className":4748,"style":961},[89],[45,4750,4752,4755],{"className":4751},[89],[45,4753,2634],{"className":4754,"style":1707},[89,90],[45,4756,4758],{"className":4757},[281],[45,4759,4761],{"className":4760},[285],[45,4762,4764],{"className":4763},[289],[45,4765,4768],{"className":4766,"style":4767},[293],"height:0.7401em;",[45,4769,4771,4774],{"style":4770},"top:-2.989em;margin-right:0.05em;",[45,4772],{"className":4773,"style":301},[300],[45,4775,4777],{"className":4776},[305,306,307,308],[45,4778,1947],{"className":4779},[89,308],[45,4781,4782,4785],{"style":4579},[45,4783],{"className":4784,"style":2314},[300],[45,4786],{"className":4787,"style":4587},[4586],[45,4789,4790,4793],{"style":4590},[45,4791],{"className":4792,"style":2314},[300],[45,4794,4796,4826,4829,4832,4835,4838,4841],{"className":4795},[89],[45,4797,4799,4802],{"className":4798},[89],[45,4800,4505],{"className":4801,"style":961},[89],[45,4803,4805],{"className":4804},[281],[45,4806,4808],{"className":4807},[285],[45,4809,4811],{"className":4810},[289],[45,4812,4815],{"className":4813,"style":4814},[293],"height:0.8141em;",[45,4816,4817,4820],{"style":296},[45,4818],{"className":4819,"style":301},[300],[45,4821,4823],{"className":4822},[305,306,307,308],[45,4824,1947],{"className":4825},[89,308],[45,4827,3752],{"className":4828},[89],[45,4830,172],{"className":4831},[221],[45,4833,2634],{"className":4834,"style":1707},[89,90],[45,4836,1663],{"className":4837},[89],[45,4839,1659],{"className":4840,"style":1707},[89,90],[45,4842,177],{"className":4843},[228],[45,4845,428],{"className":4846},[427],[45,4848,4850],{"className":4849},[289],[45,4851,4853],{"className":4852,"style":4627},[293],[45,4854],{},[45,4856],{"className":4857},[228,4548],[45,4859],{"className":4860,"style":233},[232],[45,4862,4703],{"className":4863},[237],[45,4865],{"className":4866,"style":233},[232],[45,4868,4870,4873],{"className":4869},[80],[45,4871],{"className":4872,"style":1631},[84],[45,4874,4528],{"className":4875},[89],[16,4877,4878],{},"Take the simplest Bernoulli distribution as an example:",[1289,4880,4881,5000],{},[1292,4882,4883,4884],{},"Data ",[45,4885,4887,4917],{"className":4886,"translate":49},[48],[45,4888,4890],{"className":4889},[53],[55,4891,4892],{"xmlns":57},[59,4893,4894,4914],{},[62,4895,4896,4902,4904,4906,4908,4910,4912],{},[342,4897,4898,4900],{},[65,4899,2397],{},[65,4901,3470],{},[169,4903,3473],{},[169,4905,2392],{"stretchy":171},[182,4907,4528],{},[169,4909,2402],{"separator":76},[182,4911,184],{},[169,4913,2427],{"stretchy":171},[69,4915,4916],{"encoding":71},"x_i \\in \\{0,1\\}",[45,4918,4920,4976],{"className":4919,"ariaHidden":76},[75],[45,4921,4923,4927,4967,4970,4973],{"className":4922},[80],[45,4924],{"className":4925,"style":4926},[84],"height:0.6891em;vertical-align:-0.15em;",[45,4928,4930,4933],{"className":4929},[89],[45,4931,2397],{"className":4932},[89,90],[45,4934,4936],{"className":4935},[281],[45,4937,4939,4959],{"className":4938},[285,397],[45,4940,4942,4956],{"className":4941},[289],[45,4943,4945],{"className":4944,"style":3668},[293],[45,4946,4947,4950],{"style":2482},[45,4948],{"className":4949,"style":301},[300],[45,4951,4953],{"className":4952},[305,306,307,308],[45,4954,3470],{"className":4955},[89,90,308],[45,4957,428],{"className":4958},[427],[45,4960,4962],{"className":4961},[289],[45,4963,4965],{"className":4964,"style":435},[293],[45,4966],{},[45,4968],{"className":4969,"style":233},[232],[45,4971,3473],{"className":4972},[237],[45,4974],{"className":4975,"style":233},[232],[45,4977,4979,4982,4985,4988,4991,4994,4997],{"className":4978},[80],[45,4980],{"className":4981,"style":214},[84],[45,4983,2392],{"className":4984},[221],[45,4986,4528],{"className":4987},[89],[45,4989,2402],{"className":4990},[2506],[45,4992],{"className":4993,"style":2510},[232],[45,4995,184],{"className":4996},[89],[45,4998,2427],{"className":4999},[228],[1292,5001,5002,5003],{},"Model: ",[45,5004,5006,5057],{"className":5005,"translate":49},[48],[45,5007,5009],{"className":5008},[53],[55,5010,5011],{"xmlns":57},[59,5012,5013,5054],{},[62,5014,5015,5017,5019,5022,5024,5026,5028,5030,5032,5034,5036,5038,5040,5042,5044,5046,5048,5050,5052],{},[65,5016,346],{},[169,5018,172],{"stretchy":171},[65,5020,5021],{},"X",[169,5023,359],{},[182,5025,184],{},[169,5027,177],{"stretchy":171},[169,5029,359],{},[65,5031,16],{},[169,5033,2402],{"separator":76},[65,5035,346],{},[169,5037,172],{"stretchy":171},[65,5039,5021],{},[169,5041,359],{},[182,5043,4528],{},[169,5045,177],{"stretchy":171},[169,5047,359],{},[182,5049,184],{},[169,5051,187],{},[65,5053,16],{},[69,5055,5056],{"encoding":71},"P(X=1)=p, P(X=0)=1-p",[45,5058,5060,5085,5106,5139,5160,5178],{"className":5059,"ariaHidden":76},[75],[45,5061,5063,5066,5069,5072,5076,5079,5082],{"className":5062},[80],[45,5064],{"className":5065,"style":214},[84],[45,5067,346],{"className":5068,"style":390},[89,90],[45,5070,172],{"className":5071},[221],[45,5073,5021],{"className":5074,"style":5075},[89,90],"margin-right:0.0785em;",[45,5077],{"className":5078,"style":233},[232],[45,5080,359],{"className":5081},[237],[45,5083],{"className":5084,"style":233},[232],[45,5086,5088,5091,5094,5097,5100,5103],{"className":5087},[80],[45,5089],{"className":5090,"style":214},[84],[45,5092,184],{"className":5093},[89],[45,5095,177],{"className":5096},[228],[45,5098],{"className":5099,"style":233},[232],[45,5101,359],{"className":5102},[237],[45,5104],{"className":5105,"style":233},[232],[45,5107,5109,5112,5115,5118,5121,5124,5127,5130,5133,5136],{"className":5108},[80],[45,5110],{"className":5111,"style":214},[84],[45,5113,16],{"className":5114},[89,90],[45,5116,2402],{"className":5117},[2506],[45,5119],{"className":5120,"style":2510},[232],[45,5122,346],{"className":5123,"style":390},[89,90],[45,5125,172],{"className":5126},[221],[45,5128,5021],{"className":5129,"style":5075},[89,90],[45,5131],{"className":5132,"style":233},[232],[45,5134,359],{"className":5135},[237],[45,5137],{"className":5138,"style":233},[232],[45,5140,5142,5145,5148,5151,5154,5157],{"className":5141},[80],[45,5143],{"className":5144,"style":214},[84],[45,5146,4528],{"className":5147},[89],[45,5149,177],{"className":5150},[228],[45,5152],{"className":5153,"style":233},[232],[45,5155,359],{"className":5156},[237],[45,5158],{"className":5159,"style":233},[232],[45,5161,5163,5166,5169,5172,5175],{"className":5162},[80],[45,5164],{"className":5165,"style":247},[84],[45,5167,184],{"className":5168},[89],[45,5170],{"className":5171,"style":254},[232],[45,5173,187],{"className":5174},[258],[45,5176],{"className":5177,"style":254},[232],[45,5179,5181,5184],{"className":5180},[80],[45,5182],{"className":5183,"style":117},[84],[45,5185,16],{"className":5186},[89,90],[16,5188,5189],{},"We have the likelihood function:",[45,5191,5193],{"className":5192,"translate":49},[327],[45,5194,5196,5271],{"className":5195,"translate":49},[48],[45,5197,5199],{"className":5198},[53],[55,5200,5201],{"xmlns":57,"display":336},[59,5202,5203,5268],{},[62,5204,5205,5207,5209,5211,5213,5215,5217,5219,5234,5244,5246,5248,5250,5252],{},[65,5206,2674],{},[169,5208,172],{"stretchy":171},[65,5210,16],{},[65,5212,1663],{"mathvariant":1662},[65,5214,1659],{},[169,5216,177],{"stretchy":171},[169,5218,359],{},[5220,5221,5222,5224,5232],"munderover",{},[169,5223,3461],{},[62,5225,5226,5228,5230],{},[65,5227,3470],{},[169,5229,359],{},[182,5231,184],{},[65,5233,67],{},[191,5235,5236,5238],{},[65,5237,16],{},[342,5239,5240,5242],{},[65,5241,2397],{},[65,5243,3470],{},[169,5245,172],{"stretchy":171},[182,5247,184],{},[169,5249,187],{},[65,5251,16],{},[191,5253,5254,5256],{},[169,5255,177],{"stretchy":171},[62,5257,5258,5260,5262],{},[182,5259,184],{},[169,5261,187],{},[342,5263,5264,5266],{},[65,5265,2397],{},[65,5267,3470],{},[69,5269,5270],{"encoding":71},"L(p|D) = \\prod_{i=1}^n p^{x_i} (1-p)^{1-x_i}",[45,5272,5274,5307,5469],{"className":5273,"ariaHidden":76},[75],[45,5275,5277,5280,5283,5286,5289,5292,5295,5298,5301,5304],{"className":5276},[80],[45,5278],{"className":5279,"style":214},[84],[45,5281,2674],{"className":5282},[89,90],[45,5284,172],{"className":5285},[221],[45,5287,16],{"className":5288},[89,90],[45,5290,1663],{"className":5291},[89],[45,5293,1659],{"className":5294,"style":1707},[89,90],[45,5296,177],{"className":5297},[228],[45,5299],{"className":5300,"style":233},[232],[45,5302,359],{"className":5303},[237],[45,5305],{"className":5306,"style":233},[232],[45,5308,5310,5314,5382,5385,5454,5457,5460,5463,5466],{"className":5309},[80],[45,5311],{"className":5312,"style":5313},[84],"height:2.9291em;vertical-align:-1.2777em;",[45,5315,5317],{"className":5316},[2899,2911],[45,5318,5320,5373],{"className":5319},[285,397],[45,5321,5323,5370],{"className":5322},[289],[45,5324,5327,5348,5358],{"className":5325,"style":5326},[293],"height:1.6514em;",[45,5328,5330,5333],{"style":5329},"top:-1.8723em;margin-left:0em;",[45,5331],{"className":5332,"style":3557},[300],[45,5334,5336],{"className":5335},[305,306,307,308],[45,5337,5339,5342,5345],{"className":5338},[89,308],[45,5340,3470],{"className":5341},[89,90,308],[45,5343,359],{"className":5344},[237,308],[45,5346,184],{"className":5347},[89,308],[45,5349,5350,5353],{"style":3618},[45,5351],{"className":5352,"style":3557},[300],[45,5354,5355],{},[45,5356,3461],{"className":5357},[2899,3627,3628],[45,5359,5361,5364],{"style":5360},"top:-4.3em;margin-left:0em;",[45,5362],{"className":5363,"style":3557},[300],[45,5365,5367],{"className":5366},[305,306,307,308],[45,5368,67],{"className":5369},[89,90,308],[45,5371,428],{"className":5372},[427],[45,5374,5376],{"className":5375},[289],[45,5377,5380],{"className":5378,"style":5379},[293],"height:1.2777em;",[45,5381],{},[45,5383],{"className":5384,"style":2510},[232],[45,5386,5388,5391],{"className":5387},[89],[45,5389,16],{"className":5390},[89,90],[45,5392,5394],{"className":5393},[281],[45,5395,5397],{"className":5396},[285],[45,5398,5400],{"className":5399},[289],[45,5401,5403],{"className":5402,"style":489},[293],[45,5404,5405,5408],{"style":492},[45,5406],{"className":5407,"style":301},[300],[45,5409,5411],{"className":5410},[305,306,307,308],[45,5412,5414],{"className":5413},[89,308],[45,5415,5417,5420],{"className":5416},[89,308],[45,5418,2397],{"className":5419},[89,90,308],[45,5421,5423],{"className":5422},[281],[45,5424,5426,5446],{"className":5425},[285,397],[45,5427,5429,5443],{"className":5428},[289],[45,5430,5432],{"className":5431,"style":3582},[293],[45,5433,5434,5437],{"style":3585},[45,5435],{"className":5436,"style":3589},[300],[45,5438,5440],{"className":5439},[305,3593,3594,308],[45,5441,3470],{"className":5442},[89,90,308],[45,5444,428],{"className":5445},[427],[45,5447,5449],{"className":5448},[289],[45,5450,5452],{"className":5451,"style":3607},[293],[45,5453],{},[45,5455,172],{"className":5456},[221],[45,5458,184],{"className":5459},[89],[45,5461],{"className":5462,"style":254},[232],[45,5464,187],{"className":5465},[258],[45,5467],{"className":5468,"style":254},[232],[45,5470,5472,5476,5479],{"className":5471},[80],[45,5473],{"className":5474,"style":5475},[84],"height:1.1141em;vertical-align:-0.25em;",[45,5477,16],{"className":5478},[89,90],[45,5480,5482,5485],{"className":5481},[228],[45,5483,177],{"className":5484},[228],[45,5486,5488],{"className":5487},[281],[45,5489,5491],{"className":5490},[285],[45,5492,5494],{"className":5493},[289],[45,5495,5497],{"className":5496,"style":1750},[293],[45,5498,5499,5502],{"style":492},[45,5500],{"className":5501,"style":301},[300],[45,5503,5505],{"className":5504},[305,306,307,308],[45,5506,5508,5511,5514],{"className":5507},[89,308],[45,5509,184],{"className":5510},[89,308],[45,5512,187],{"className":5513},[258,308],[45,5515,5517,5520],{"className":5516},[89,308],[45,5518,2397],{"className":5519},[89,90,308],[45,5521,5523],{"className":5522},[281],[45,5524,5526,5546],{"className":5525},[285,397],[45,5527,5529,5543],{"className":5528},[289],[45,5530,5532],{"className":5531,"style":3582},[293],[45,5533,5534,5537],{"style":3585},[45,5535],{"className":5536,"style":3589},[300],[45,5538,5540],{"className":5539},[305,3593,3594,308],[45,5541,3470],{"className":5542},[89,90,308],[45,5544,428],{"className":5545},[427],[45,5547,5549],{"className":5548},[289],[45,5550,5552],{"className":5551,"style":3607},[293],[45,5553],{},[16,5555,5556],{},"Convert to log-likelihood:",[45,5558,5560],{"className":5559,"translate":49},[327],[45,5561,5563,5646],{"className":5562,"translate":49},[48],[45,5564,5566],{"className":5565},[53],[55,5567,5568],{"xmlns":57,"display":336},[59,5569,5570,5643],{},[62,5571,5572,5574,5576,5578,5580,5582,5584,5586,5600,5606,5608,5610,5612,5615,5617,5619,5621,5627,5629,5631,5633,5635,5637,5639,5641],{},[65,5573,3752],{"mathvariant":1662},[169,5575,172],{"stretchy":171},[65,5577,16],{},[65,5579,1663],{"mathvariant":1662},[65,5581,1659],{},[169,5583,177],{"stretchy":171},[169,5585,359],{},[5220,5587,5588,5590,5598],{},[169,5589,3788],{},[62,5591,5592,5594,5596],{},[65,5593,3470],{},[169,5595,359],{},[182,5597,184],{},[65,5599,67],{},[342,5601,5602,5604],{},[65,5603,2397],{},[65,5605,3470],{},[65,5607,3767],{},[169,5609,2786],{},[65,5611,16],{},[169,5613,5614],{},"+",[169,5616,172],{"stretchy":171},[182,5618,184],{},[169,5620,187],{},[342,5622,5623,5625],{},[65,5624,2397],{},[65,5626,3470],{},[169,5628,177],{"stretchy":171},[65,5630,3767],{},[169,5632,2786],{},[169,5634,172],{"stretchy":171},[182,5636,184],{},[169,5638,187],{},[65,5640,16],{},[169,5642,177],{"stretchy":171},[69,5644,5645],{"encoding":71},"\\ell(p|D) = \\sum_{i=1}^n x_i \\log p + (1-x_i) \\log (1-p)",[45,5647,5649,5682,5818,5839,5911],{"className":5648,"ariaHidden":76},[75],[45,5650,5652,5655,5658,5661,5664,5667,5670,5673,5676,5679],{"className":5651},[80],[45,5653],{"className":5654,"style":214},[84],[45,5656,3752],{"className":5657},[89],[45,5659,172],{"className":5660},[221],[45,5662,16],{"className":5663},[89,90],[45,5665,1663],{"className":5666},[89],[45,5668,1659],{"className":5669,"style":1707},[89,90],[45,5671,177],{"className":5672},[228],[45,5674],{"className":5675,"style":233},[232],[45,5677,359],{"className":5678},[237],[45,5680],{"className":5681,"style":233},[232],[45,5683,5685,5688,5752,5755,5795,5798,5803,5806,5809,5812,5815],{"className":5684},[80],[45,5686],{"className":5687,"style":5313},[84],[45,5689,5691],{"className":5690},[2899,2911],[45,5692,5694,5744],{"className":5693},[285,397],[45,5695,5697,5741],{"className":5696},[289],[45,5698,5700,5720,5730],{"className":5699,"style":5326},[293],[45,5701,5702,5705],{"style":5329},[45,5703],{"className":5704,"style":3557},[300],[45,5706,5708],{"className":5707},[305,306,307,308],[45,5709,5711,5714,5717],{"className":5710},[89,308],[45,5712,3470],{"className":5713},[89,90,308],[45,5715,359],{"className":5716},[237,308],[45,5718,184],{"className":5719},[89,308],[45,5721,5722,5725],{"style":3618},[45,5723],{"className":5724,"style":3557},[300],[45,5726,5727],{},[45,5728,3788],{"className":5729},[2899,3627,3628],[45,5731,5732,5735],{"style":5360},[45,5733],{"className":5734,"style":3557},[300],[45,5736,5738],{"className":5737},[305,306,307,308],[45,5739,67],{"className":5740},[89,90,308],[45,5742,428],{"className":5743},[427],[45,5745,5747],{"className":5746},[289],[45,5748,5750],{"className":5749,"style":5379},[293],[45,5751],{},[45,5753],{"className":5754,"style":2510},[232],[45,5756,5758,5761],{"className":5757},[89],[45,5759,2397],{"className":5760},[89,90],[45,5762,5764],{"className":5763},[281],[45,5765,5767,5787],{"className":5766},[285,397],[45,5768,5770,5784],{"className":5769},[289],[45,5771,5773],{"className":5772,"style":3668},[293],[45,5774,5775,5778],{"style":2482},[45,5776],{"className":5777,"style":301},[300],[45,5779,5781],{"className":5780},[305,306,307,308],[45,5782,3470],{"className":5783},[89,90,308],[45,5785,428],{"className":5786},[427],[45,5788,5790],{"className":5789},[289],[45,5791,5793],{"className":5792,"style":435},[293],[45,5794],{},[45,5796],{"className":5797,"style":2510},[232],[45,5799,3869,5801],{"className":5800},[2899],[45,5802,2904],{"style":2903},[45,5804],{"className":5805,"style":2510},[232],[45,5807,16],{"className":5808},[89,90],[45,5810],{"className":5811,"style":254},[232],[45,5813,5614],{"className":5814},[258],[45,5816],{"className":5817,"style":254},[232],[45,5819,5821,5824,5827,5830,5833,5836],{"className":5820},[80],[45,5822],{"className":5823,"style":214},[84],[45,5825,172],{"className":5826},[221],[45,5828,184],{"className":5829},[89],[45,5831],{"className":5832,"style":254},[232],[45,5834,187],{"className":5835},[258],[45,5837],{"className":5838,"style":254},[232],[45,5840,5842,5845,5885,5888,5891,5896,5899,5902,5905,5908],{"className":5841},[80],[45,5843],{"className":5844,"style":214},[84],[45,5846,5848,5851],{"className":5847},[89],[45,5849,2397],{"className":5850},[89,90],[45,5852,5854],{"className":5853},[281],[45,5855,5857,5877],{"className":5856},[285,397],[45,5858,5860,5874],{"className":5859},[289],[45,5861,5863],{"className":5862,"style":3668},[293],[45,5864,5865,5868],{"style":2482},[45,5866],{"className":5867,"style":301},[300],[45,5869,5871],{"className":5870},[305,306,307,308],[45,5872,3470],{"className":5873},[89,90,308],[45,5875,428],{"className":5876},[427],[45,5878,5880],{"className":5879},[289],[45,5881,5883],{"className":5882,"style":435},[293],[45,5884],{},[45,5886,177],{"className":5887},[228],[45,5889],{"className":5890,"style":2510},[232],[45,5892,3869,5894],{"className":5893},[2899],[45,5895,2904],{"style":2903},[45,5897,172],{"className":5898},[221],[45,5900,184],{"className":5901},[89],[45,5903],{"className":5904,"style":254},[232],[45,5906,187],{"className":5907},[258],[45,5909],{"className":5910,"style":254},[232],[45,5912,5914,5917,5920],{"className":5913},[80],[45,5915],{"className":5916,"style":214},[84],[45,5918,16],{"className":5919},[89,90],[45,5921,177],{"className":5922},[228],[16,5924,5925,5926,5954],{},"Take the derivative with respect to ",[45,5927,5929,5942],{"className":5928,"translate":49},[48],[45,5930,5932],{"className":5931},[53],[55,5933,5934],{"xmlns":57},[59,5935,5936,5940],{},[62,5937,5938],{},[65,5939,16],{},[69,5941,16],{"encoding":71},[45,5943,5945],{"className":5944,"ariaHidden":76},[75],[45,5946,5948,5951],{"className":5947},[80],[45,5949],{"className":5950,"style":117},[84],[45,5952,16],{"className":5953},[89,90]," and set to 0:",[45,5956,5958],{"className":5957,"translate":49},[327],[45,5959,5961,6033],{"className":5960,"translate":49},[48],[45,5962,5964],{"className":5963},[53],[55,5965,5966],{"xmlns":57,"display":336},[59,5967,5968,6030],{},[62,5969,5970,5984,5986,6000,6002,6026,6028],{},[4499,5971,5972,5978],{},[62,5973,5974,5976],{},[65,5975,4505],{"mathvariant":1662},[65,5977,3752],{"mathvariant":1662},[62,5979,5980,5982],{},[65,5981,4505],{"mathvariant":1662},[65,5983,16],{},[169,5985,359],{},[4499,5987,5988,5998],{},[62,5989,5990,5992],{},[169,5991,3788],{},[342,5993,5994,5996],{},[65,5995,2397],{},[65,5997,3470],{},[65,5999,16],{},[169,6001,187],{},[4499,6003,6004,6018],{},[62,6005,6006,6008,6010,6012],{},[65,6007,67],{},[169,6009,187],{},[169,6011,3788],{},[342,6013,6014,6016],{},[65,6015,2397],{},[65,6017,3470],{},[62,6019,6020,6022,6024],{},[182,6021,184],{},[169,6023,187],{},[65,6025,16],{},[169,6027,359],{},[182,6029,4528],{},[69,6031,6032],{"encoding":71},"\\frac{\\partial \\ell}{\\partial p} = \\frac{\\sum x_i}{p} - \\frac{n-\\sum x_i}{1-p} = 0",[45,6034,6036,6122,6245,6389],{"className":6035,"ariaHidden":76},[75],[45,6037,6039,6043,6113,6116,6119],{"className":6038},[80],[45,6040],{"className":6041,"style":6042},[84],"height:2.2519em;vertical-align:-0.8804em;",[45,6044,6046,6049,6110],{"className":6045},[89],[45,6047],{"className":6048},[221,4548],[45,6050,6052],{"className":6051},[4499],[45,6053,6055,6101],{"className":6054},[285,397],[45,6056,6058,6098],{"className":6057},[289],[45,6059,6062,6076,6084],{"className":6060,"style":6061},[293],"height:1.3714em;",[45,6063,6064,6067],{"style":4564},[45,6065],{"className":6066,"style":2314},[300],[45,6068,6070,6073],{"className":6069},[89],[45,6071,4505],{"className":6072,"style":961},[89],[45,6074,16],{"className":6075},[89,90],[45,6077,6078,6081],{"style":4579},[45,6079],{"className":6080,"style":2314},[300],[45,6082],{"className":6083,"style":4587},[4586],[45,6085,6086,6089],{"style":4590},[45,6087],{"className":6088,"style":2314},[300],[45,6090,6092,6095],{"className":6091},[89],[45,6093,4505],{"className":6094,"style":961},[89],[45,6096,3752],{"className":6097},[89],[45,6099,428],{"className":6100},[427],[45,6102,6104],{"className":6103},[289],[45,6105,6108],{"className":6106,"style":6107},[293],"height:0.8804em;",[45,6109],{},[45,6111],{"className":6112},[228,4548],[45,6114],{"className":6115,"style":233},[232],[45,6117,359],{"className":6118},[237],[45,6120],{"className":6121,"style":233},[232],[45,6123,6125,6129,6236,6239,6242],{"className":6124},[80],[45,6126],{"className":6127,"style":6128},[84],"height:2.3074em;vertical-align:-0.8804em;",[45,6130,6132,6135,6233],{"className":6131},[89],[45,6133],{"className":6134},[221,4548],[45,6136,6138],{"className":6137},[4499],[45,6139,6141,6225],{"className":6140},[285,397],[45,6142,6144,6222],{"className":6143},[289],[45,6145,6147,6158,6166],{"className":6146,"style":4561},[293],[45,6148,6149,6152],{"style":4564},[45,6150],{"className":6151,"style":2314},[300],[45,6153,6155],{"className":6154},[89],[45,6156,16],{"className":6157},[89,90],[45,6159,6160,6163],{"style":4579},[45,6161],{"className":6162,"style":2314},[300],[45,6164],{"className":6165,"style":4587},[4586],[45,6167,6168,6171],{"style":4590},[45,6169],{"className":6170,"style":2314},[300],[45,6172,6174,6179,6182],{"className":6173},[89],[45,6175,3788],{"className":6176,"style":6178},[2899,3627,6177],"small-op","position:relative;top:0em;",[45,6180],{"className":6181,"style":2510},[232],[45,6183,6185,6188],{"className":6184},[89],[45,6186,2397],{"className":6187},[89,90],[45,6189,6191],{"className":6190},[281],[45,6192,6194,6214],{"className":6193},[285,397],[45,6195,6197,6211],{"className":6196},[289],[45,6198,6200],{"className":6199,"style":3668},[293],[45,6201,6202,6205],{"style":2482},[45,6203],{"className":6204,"style":301},[300],[45,6206,6208],{"className":6207},[305,306,307,308],[45,6209,3470],{"className":6210},[89,90,308],[45,6212,428],{"className":6213},[427],[45,6215,6217],{"className":6216},[289],[45,6218,6220],{"className":6219,"style":435},[293],[45,6221],{},[45,6223,428],{"className":6224},[427],[45,6226,6228],{"className":6227},[289],[45,6229,6231],{"className":6230,"style":6107},[293],[45,6232],{},[45,6234],{"className":6235},[228,4548],[45,6237],{"className":6238,"style":254},[232],[45,6240,187],{"className":6241},[258],[45,6243],{"className":6244,"style":254},[232],[45,6246,6248,6251,6380,6383,6386],{"className":6247},[80],[45,6249],{"className":6250,"style":6128},[84],[45,6252,6254,6257,6377],{"className":6253},[89],[45,6255],{"className":6256},[221,4548],[45,6258,6260],{"className":6259},[4499],[45,6261,6263,6369],{"className":6262},[285,397],[45,6264,6266,6366],{"className":6265},[289],[45,6267,6269,6292,6300],{"className":6268,"style":4561},[293],[45,6270,6271,6274],{"style":4564},[45,6272],{"className":6273,"style":2314},[300],[45,6275,6277,6280,6283,6286,6289],{"className":6276},[89],[45,6278,184],{"className":6279},[89],[45,6281],{"className":6282,"style":254},[232],[45,6284,187],{"className":6285},[258],[45,6287],{"className":6288,"style":254},[232],[45,6290,16],{"className":6291},[89,90],[45,6293,6294,6297],{"style":4579},[45,6295],{"className":6296,"style":2314},[300],[45,6298],{"className":6299,"style":4587},[4586],[45,6301,6302,6305],{"style":4590},[45,6303],{"className":6304,"style":2314},[300],[45,6306,6308,6311,6314,6317,6320,6323,6326],{"className":6307},[89],[45,6309,67],{"className":6310},[89,90],[45,6312],{"className":6313,"style":254},[232],[45,6315,187],{"className":6316},[258],[45,6318],{"className":6319,"style":254},[232],[45,6321,3788],{"className":6322,"style":6178},[2899,3627,6177],[45,6324],{"className":6325,"style":2510},[232],[45,6327,6329,6332],{"className":6328},[89],[45,6330,2397],{"className":6331},[89,90],[45,6333,6335],{"className":6334},[281],[45,6336,6338,6358],{"className":6337},[285,397],[45,6339,6341,6355],{"className":6340},[289],[45,6342,6344],{"className":6343,"style":3668},[293],[45,6345,6346,6349],{"style":2482},[45,6347],{"className":6348,"style":301},[300],[45,6350,6352],{"className":6351},[305,306,307,308],[45,6353,3470],{"className":6354},[89,90,308],[45,6356,428],{"className":6357},[427],[45,6359,6361],{"className":6360},[289],[45,6362,6364],{"className":6363,"style":435},[293],[45,6365],{},[45,6367,428],{"className":6368},[427],[45,6370,6372],{"className":6371},[289],[45,6373,6375],{"className":6374,"style":6107},[293],[45,6376],{},[45,6378],{"className":6379},[228,4548],[45,6381],{"className":6382,"style":233},[232],[45,6384,359],{"className":6385},[237],[45,6387],{"className":6388,"style":233},[232],[45,6390,6392,6395],{"className":6391},[80],[45,6393],{"className":6394,"style":1631},[84],[45,6396,4528],{"className":6397},[89],[16,6399,6400],{},"Solve to obtain:",[45,6402,6404],{"className":6403,"translate":49},[327],[45,6405,6407,6463],{"className":6406,"translate":49},[48],[45,6408,6410],{"className":6409},[53],[55,6411,6412],{"xmlns":57,"display":336},[59,6413,6414,6460],{},[62,6415,6416,6432,6434],{},[342,6417,6418,6424],{},[2271,6419,6420,6422],{"accent":76},[65,6421,16],{},[169,6423,2277],{},[62,6425,6426,6428,6430],{},[65,6427,4389],{"mathvariant":1662},[65,6429,2674],{"mathvariant":1662},[65,6431,4394],{"mathvariant":1662},[169,6433,359],{},[4499,6435,6436,6458],{},[62,6437,6438,6452],{},[5220,6439,6440,6442,6450],{},[169,6441,3788],{},[62,6443,6444,6446,6448],{},[65,6445,3470],{},[169,6447,359],{},[182,6449,184],{},[65,6451,67],{},[342,6453,6454,6456],{},[65,6455,2397],{},[65,6457,3470],{},[65,6459,67],{},[69,6461,6462],{"encoding":71},"\\hat{p}_{\\rm MLE} = \\frac{\\sum_{i=1}^n x_i}{n}",[45,6464,6466,6566],{"className":6465,"ariaHidden":76},[75],[45,6467,6469,6472,6557,6560,6563],{"className":6468},[80],[45,6470],{"className":6471,"style":925},[84],[45,6473,6475,6517],{"className":6474},[89],[45,6476,6478],{"className":6477},[89,2297],[45,6479,6481,6509],{"className":6480},[285,397],[45,6482,6484,6506],{"className":6483},[289],[45,6485,6487,6495],{"className":6486,"style":2307},[293],[45,6488,6489,6492],{"style":2310},[45,6490],{"className":6491,"style":2314},[300],[45,6493,16],{"className":6494},[89,90],[45,6496,6497,6500],{"style":2310},[45,6498],{"className":6499,"style":2314},[300],[45,6501,6503],{"className":6502,"style":2327},[2326],[45,6504,2277],{"className":6505},[89],[45,6507,428],{"className":6508},[427],[45,6510,6512],{"className":6511},[289],[45,6513,6515],{"className":6514,"style":2340},[293],[45,6516],{},[45,6518,6520],{"className":6519},[281],[45,6521,6523,6549],{"className":6522},[285,397],[45,6524,6526,6546],{"className":6525},[289],[45,6527,6529],{"className":6528,"style":4275},[293],[45,6530,6531,6534],{"style":2482},[45,6532],{"className":6533,"style":301},[300],[45,6535,6537],{"className":6536},[305,306,307,308],[45,6538,6540],{"className":6539},[89,308],[45,6541,6543],{"className":6542},[89,308],[45,6544,4215],{"className":6545},[89,4470,308],[45,6547,428],{"className":6548},[427],[45,6550,6552],{"className":6551},[289],[45,6553,6555],{"className":6554,"style":435},[293],[45,6556],{},[45,6558],{"className":6559,"style":233},[232],[45,6561,359],{"className":6562},[237],[45,6564],{"className":6565,"style":233},[232],[45,6567,6569,6573],{"className":6568},[80],[45,6570],{"className":6571,"style":6572},[84],"height:2.18em;vertical-align:-0.686em;",[45,6574,6576,6579,6738],{"className":6575},[89],[45,6577],{"className":6578},[221,4548],[45,6580,6582],{"className":6581},[4499],[45,6583,6585,6730],{"className":6584},[285,397],[45,6586,6588,6727],{"className":6587},[289],[45,6589,6592,6603,6611],{"className":6590,"style":6591},[293],"height:1.494em;",[45,6593,6594,6597],{"style":4564},[45,6595],{"className":6596,"style":2314},[300],[45,6598,6600],{"className":6599},[89],[45,6601,67],{"className":6602},[89,90],[45,6604,6605,6608],{"style":4579},[45,6606],{"className":6607,"style":2314},[300],[45,6609],{"className":6610,"style":4587},[4586],[45,6612,6614,6617],{"style":6613},"top:-3.6897em;",[45,6615],{"className":6616,"style":2314},[300],[45,6618,6620,6684,6687],{"className":6619},[89],[45,6621,6623,6626],{"className":6622},[2899],[45,6624,3788],{"className":6625,"style":6178},[2899,3627,6177],[45,6627,6629],{"className":6628},[281],[45,6630,6632,6675],{"className":6631},[285,397],[45,6633,6635,6672],{"className":6634},[289],[45,6636,6639,6660],{"className":6637,"style":6638},[293],"height:0.8043em;",[45,6640,6642,6645],{"style":6641},"top:-2.4003em;margin-left:0em;margin-right:0.05em;",[45,6643],{"className":6644,"style":301},[300],[45,6646,6648],{"className":6647},[305,306,307,308],[45,6649,6651,6654,6657],{"className":6650},[89,308],[45,6652,3470],{"className":6653},[89,90,308],[45,6655,359],{"className":6656},[237,308],[45,6658,184],{"className":6659},[89,308],[45,6661,6663,6666],{"style":6662},"top:-3.2029em;margin-right:0.05em;",[45,6664],{"className":6665,"style":301},[300],[45,6667,6669],{"className":6668},[305,306,307,308],[45,6670,67],{"className":6671},[89,90,308],[45,6673,428],{"className":6674},[427],[45,6676,6678],{"className":6677},[289],[45,6679,6682],{"className":6680,"style":6681},[293],"height:0.2997em;",[45,6683],{},[45,6685],{"className":6686,"style":2510},[232],[45,6688,6690,6693],{"className":6689},[89],[45,6691,2397],{"className":6692},[89,90],[45,6694,6696],{"className":6695},[281],[45,6697,6699,6719],{"className":6698},[285,397],[45,6700,6702,6716],{"className":6701},[289],[45,6703,6705],{"className":6704,"style":3668},[293],[45,6706,6707,6710],{"style":2482},[45,6708],{"className":6709,"style":301},[300],[45,6711,6713],{"className":6712},[305,306,307,308],[45,6714,3470],{"className":6715},[89,90,308],[45,6717,428],{"className":6718},[427],[45,6720,6722],{"className":6721},[289],[45,6723,6725],{"className":6724,"style":435},[293],[45,6726],{},[45,6728,428],{"className":6729},[427],[45,6731,6733],{"className":6732},[289],[45,6734,6736],{"className":6735,"style":4627},[293],[45,6737],{},[45,6739],{"className":6740},[228,4548],[16,6742,6743,6744,6772,6773,6776],{},"In neural networks, the MLE parameter ",[45,6745,6747,6760],{"className":6746,"translate":49},[48],[45,6748,6750],{"className":6749},[53],[55,6751,6752],{"xmlns":57},[59,6753,6754,6758],{},[62,6755,6756],{},[65,6757,2634],{},[69,6759,2637],{"encoding":71},[45,6761,6763],{"className":6762,"ariaHidden":76},[75],[45,6764,6766,6769],{"className":6765},[80],[45,6767],{"className":6768,"style":2307},[84],[45,6770,2634],{"className":6771,"style":1707},[89,90]," corresponds to the network's ",[668,6774,6775],{},"weights and biases",", typically denoted as:",[45,6778,6780],{"className":6779,"translate":49},[327],[45,6781,6783,6854],{"className":6782,"translate":49},[48],[45,6784,6786],{"className":6785},[53],[55,6787,6788],{"xmlns":57,"display":336},[59,6789,6790,6851],{},[62,6791,6792,6794,6796],{},[65,6793,2634],{},[169,6795,359],{},[62,6797,6798,6805,6807,6814,6816,6822,6824,6830,6832,6835,6837,6843,6845],{},[342,6799,6800,6803],{},[65,6801,6802],{},"W",[182,6804,184],{},[169,6806,2402],{"separator":76},[342,6808,6809,6812],{},[65,6810,6811],{},"b",[182,6813,184],{},[169,6815,2402],{"separator":76},[342,6817,6818,6820],{},[65,6819,6802],{},[182,6821,1947],{},[169,6823,2402],{"separator":76},[342,6825,6826,6828],{},[65,6827,6811],{},[182,6829,1947],{},[169,6831,2402],{"separator":76},[169,6833,6834],{},"…",[169,6836,2402],{"separator":76},[342,6838,6839,6841],{},[65,6840,6802],{},[65,6842,2674],{},[169,6844,2402],{"separator":76},[342,6846,6847,6849],{},[65,6848,6811],{},[65,6850,2674],{},[69,6852,6853],{"encoding":71},"\\theta = {W_1, b_1, W_2, b_2, \\dots, W_L, b_L}",[45,6855,6857,6875],{"className":6856,"ariaHidden":76},[75],[45,6858,6860,6863,6866,6869,6872],{"className":6859},[80],[45,6861],{"className":6862,"style":2307},[84],[45,6864,2634],{"className":6865,"style":1707},[89,90],[45,6867],{"className":6868,"style":233},[232],[45,6870,359],{"className":6871},[237],[45,6873],{"className":6874,"style":233},[232],[45,6876,6878,6881],{"className":6877},[80],[45,6879],{"className":6880,"style":925},[84],[45,6882,6884,6924,6927,6930,6970,6973,6976,7016,7019,7022,7062,7065,7068,7071,7074,7077,7080,7120,7123,7126],{"className":6883},[89],[45,6885,6887,6890],{"className":6886},[89],[45,6888,6802],{"className":6889,"style":390},[89,90],[45,6891,6893],{"className":6892},[281],[45,6894,6896,6916],{"className":6895},[285,397],[45,6897,6899,6913],{"className":6898},[289],[45,6900,6902],{"className":6901,"style":2479},[293],[45,6903,6904,6907],{"style":407},[45,6905],{"className":6906,"style":301},[300],[45,6908,6910],{"className":6909},[305,306,307,308],[45,6911,184],{"className":6912},[89,308],[45,6914,428],{"className":6915},[427],[45,6917,6919],{"className":6918},[289],[45,6920,6922],{"className":6921,"style":435},[293],[45,6923],{},[45,6925,2402],{"className":6926},[2506],[45,6928],{"className":6929,"style":2510},[232],[45,6931,6933,6936],{"className":6932},[89],[45,6934,6811],{"className":6935},[89,90],[45,6937,6939],{"className":6938},[281],[45,6940,6942,6962],{"className":6941},[285,397],[45,6943,6945,6959],{"className":6944},[289],[45,6946,6948],{"className":6947,"style":2479},[293],[45,6949,6950,6953],{"style":2482},[45,6951],{"className":6952,"style":301},[300],[45,6954,6956],{"className":6955},[305,306,307,308],[45,6957,184],{"className":6958},[89,308],[45,6960,428],{"className":6961},[427],[45,6963,6965],{"className":6964},[289],[45,6966,6968],{"className":6967,"style":435},[293],[45,6969],{},[45,6971,2402],{"className":6972},[2506],[45,6974],{"className":6975,"style":2510},[232],[45,6977,6979,6982],{"className":6978},[89],[45,6980,6802],{"className":6981,"style":390},[89,90],[45,6983,6985],{"className":6984},[281],[45,6986,6988,7008],{"className":6987},[285,397],[45,6989,6991,7005],{"className":6990},[289],[45,6992,6994],{"className":6993,"style":2479},[293],[45,6995,6996,6999],{"style":407},[45,6997],{"className":6998,"style":301},[300],[45,7000,7002],{"className":7001},[305,306,307,308],[45,7003,1947],{"className":7004},[89,308],[45,7006,428],{"className":7007},[427],[45,7009,7011],{"className":7010},[289],[45,7012,7014],{"className":7013,"style":435},[293],[45,7015],{},[45,7017,2402],{"className":7018},[2506],[45,7020],{"className":7021,"style":2510},[232],[45,7023,7025,7028],{"className":7024},[89],[45,7026,6811],{"className":7027},[89,90],[45,7029,7031],{"className":7030},[281],[45,7032,7034,7054],{"className":7033},[285,397],[45,7035,7037,7051],{"className":7036},[289],[45,7038,7040],{"className":7039,"style":2479},[293],[45,7041,7042,7045],{"style":2482},[45,7043],{"className":7044,"style":301},[300],[45,7046,7048],{"className":7047},[305,306,307,308],[45,7049,1947],{"className":7050},[89,308],[45,7052,428],{"className":7053},[427],[45,7055,7057],{"className":7056},[289],[45,7058,7060],{"className":7059,"style":435},[293],[45,7061],{},[45,7063,2402],{"className":7064},[2506],[45,7066],{"className":7067,"style":2510},[232],[45,7069,6834],{"className":7070},[2560],[45,7072],{"className":7073,"style":2510},[232],[45,7075,2402],{"className":7076},[2506],[45,7078],{"className":7079,"style":2510},[232],[45,7081,7083,7086],{"className":7082},[89],[45,7084,6802],{"className":7085,"style":390},[89,90],[45,7087,7089],{"className":7088},[281],[45,7090,7092,7112],{"className":7091},[285,397],[45,7093,7095,7109],{"className":7094},[289],[45,7096,7098],{"className":7097,"style":4275},[293],[45,7099,7100,7103],{"style":407},[45,7101],{"className":7102,"style":301},[300],[45,7104,7106],{"className":7105},[305,306,307,308],[45,7107,2674],{"className":7108},[89,90,308],[45,7110,428],{"className":7111},[427],[45,7113,7115],{"className":7114},[289],[45,7116,7118],{"className":7117,"style":435},[293],[45,7119],{},[45,7121,2402],{"className":7122},[2506],[45,7124],{"className":7125,"style":2510},[232],[45,7127,7129,7132],{"className":7128},[89],[45,7130,6811],{"className":7131},[89,90],[45,7133,7135],{"className":7134},[281],[45,7136,7138,7158],{"className":7137},[285,397],[45,7139,7141,7155],{"className":7140},[289],[45,7142,7144],{"className":7143,"style":4275},[293],[45,7145,7146,7149],{"style":2482},[45,7147],{"className":7148,"style":301},[300],[45,7150,7152],{"className":7151},[305,306,307,308],[45,7153,2674],{"className":7154},[89,90,308],[45,7156,428],{"className":7157},[427],[45,7159,7161],{"className":7160},[289],[45,7162,7164],{"className":7163,"style":435},[293],[45,7165],{},[16,7167,7168,7169,7197,7198,7268,7269,7298,7299,7370],{},"where ",[45,7170,7172,7185],{"className":7171,"translate":49},[48],[45,7173,7175],{"className":7174},[53],[55,7176,7177],{"xmlns":57},[59,7178,7179,7183],{},[62,7180,7181],{},[65,7182,2674],{},[69,7184,2674],{"encoding":71},[45,7186,7188],{"className":7187,"ariaHidden":76},[75],[45,7189,7191,7194],{"className":7190},[80],[45,7192],{"className":7193,"style":148},[84],[45,7195,2674],{"className":7196},[89,90]," is the number of layers, ",[45,7199,7201,7219],{"className":7200,"translate":49},[48],[45,7202,7204],{"className":7203},[53],[55,7205,7206],{"xmlns":57},[59,7207,7208,7216],{},[62,7209,7210],{},[342,7211,7212,7214],{},[65,7213,6802],{},[65,7215,3470],{},[69,7217,7218],{"encoding":71},"W_i",[45,7220,7222],{"className":7221,"ariaHidden":76},[75],[45,7223,7225,7228],{"className":7224},[80],[45,7226],{"className":7227,"style":564},[84],[45,7229,7231,7234],{"className":7230},[89],[45,7232,6802],{"className":7233,"style":390},[89,90],[45,7235,7237],{"className":7236},[281],[45,7238,7240,7260],{"className":7239},[285,397],[45,7241,7243,7257],{"className":7242},[289],[45,7244,7246],{"className":7245,"style":3668},[293],[45,7247,7248,7251],{"style":407},[45,7249],{"className":7250,"style":301},[300],[45,7252,7254],{"className":7253},[305,306,307,308],[45,7255,3470],{"className":7256},[89,90,308],[45,7258,428],{"className":7259},[427],[45,7261,7263],{"className":7262},[289],[45,7264,7266],{"className":7265,"style":435},[293],[45,7267],{}," is the weight matrix of the ",[45,7270,7272,7285],{"className":7271,"translate":49},[48],[45,7273,7275],{"className":7274},[53],[55,7276,7277],{"xmlns":57},[59,7278,7279,7283],{},[62,7280,7281],{},[65,7282,3470],{},[69,7284,3470],{"encoding":71},[45,7286,7288],{"className":7287,"ariaHidden":76},[75],[45,7289,7291,7295],{"className":7290},[80],[45,7292],{"className":7293,"style":7294},[84],"height:0.6595em;",[45,7296,3470],{"className":7297},[89,90],"-th layer, and ",[45,7300,7302,7320],{"className":7301,"translate":49},[48],[45,7303,7305],{"className":7304},[53],[55,7306,7307],{"xmlns":57},[59,7308,7309,7317],{},[62,7310,7311],{},[342,7312,7313,7315],{},[65,7314,6811],{},[65,7316,3470],{},[69,7318,7319],{"encoding":71},"b_i",[45,7321,7323],{"className":7322,"ariaHidden":76},[75],[45,7324,7326,7330],{"className":7325},[80],[45,7327],{"className":7328,"style":7329},[84],"height:0.8444em;vertical-align:-0.15em;",[45,7331,7333,7336],{"className":7332},[89],[45,7334,6811],{"className":7335},[89,90],[45,7337,7339],{"className":7338},[281],[45,7340,7342,7362],{"className":7341},[285,397],[45,7343,7345,7359],{"className":7344},[289],[45,7346,7348],{"className":7347,"style":3668},[293],[45,7349,7350,7353],{"style":2482},[45,7351],{"className":7352,"style":301},[300],[45,7354,7356],{"className":7355},[305,306,307,308],[45,7357,3470],{"className":7358},[89,90,308],[45,7360,428],{"className":7361},[427],[45,7363,7365],{"className":7364},[289],[45,7366,7368],{"className":7367,"style":435},[293],[45,7369],{}," is the bias vector.",[16,7372,7373,7374],{},"In other words: a neural network is a function family ",[45,7375,7377,7402],{"className":7376,"translate":49},[48],[45,7378,7380],{"className":7379},[53],[55,7381,7382],{"xmlns":57},[59,7383,7384,7399],{},[62,7385,7386,7393,7395,7397],{},[342,7387,7388,7391],{},[65,7389,7390],{},"f",[65,7392,2634],{},[169,7394,172],{"stretchy":171},[65,7396,2397],{},[169,7398,177],{"stretchy":171},[69,7400,7401],{"encoding":71},"f_\\theta(x)",[45,7403,7405],{"className":7404,"ariaHidden":76},[75],[45,7406,7408,7411,7453,7456,7459],{"className":7407},[80],[45,7409],{"className":7410,"style":214},[84],[45,7412,7414,7418],{"className":7413},[89],[45,7415,7390],{"className":7416,"style":7417},[89,90],"margin-right:0.1076em;",[45,7419,7421],{"className":7420},[281],[45,7422,7424,7445],{"className":7423},[285,397],[45,7425,7427,7442],{"className":7426},[289],[45,7428,7430],{"className":7429,"style":404},[293],[45,7431,7433,7436],{"style":7432},"top:-2.55em;margin-left:-0.1076em;margin-right:0.05em;",[45,7434],{"className":7435,"style":301},[300],[45,7437,7439],{"className":7438},[305,306,307,308],[45,7440,2634],{"className":7441,"style":1707},[89,90,308],[45,7443,428],{"className":7444},[427],[45,7446,7448],{"className":7447},[289],[45,7449,7451],{"className":7450,"style":435},[293],[45,7452],{},[45,7454,172],{"className":7455},[221],[45,7457,2397],{"className":7458},[89,90],[45,7460,177],{"className":7461},[228],[16,7463,7464],{},"It defines the conditional probability distribution:",[45,7466,7468],{"className":7467,"translate":49},[327],[45,7469,7471,7500],{"className":7470,"translate":49},[48],[45,7472,7474],{"className":7473},[53],[55,7475,7476],{"xmlns":57,"display":336},[59,7477,7478,7497],{},[62,7479,7480,7486,7488,7491,7493,7495],{},[342,7481,7482,7484],{},[65,7483,346],{},[65,7485,2634],{},[169,7487,172],{"stretchy":171},[65,7489,7490],{},"y",[65,7492,1663],{"mathvariant":1662},[65,7494,2397],{},[169,7496,177],{"stretchy":171},[69,7498,7499],{"encoding":71},"P_\\theta(y|x)",[45,7501,7503],{"className":7502,"ariaHidden":76},[75],[45,7504,7506,7509,7549,7552,7556,7559,7562],{"className":7505},[80],[45,7507],{"className":7508,"style":214},[84],[45,7510,7512,7515],{"className":7511},[89],[45,7513,346],{"className":7514,"style":390},[89,90],[45,7516,7518],{"className":7517},[281],[45,7519,7521,7541],{"className":7520},[285,397],[45,7522,7524,7538],{"className":7523},[289],[45,7525,7527],{"className":7526,"style":404},[293],[45,7528,7529,7532],{"style":407},[45,7530],{"className":7531,"style":301},[300],[45,7533,7535],{"className":7534},[305,306,307,308],[45,7536,2634],{"className":7537,"style":1707},[89,90,308],[45,7539,428],{"className":7540},[427],[45,7542,7544],{"className":7543},[289],[45,7545,7547],{"className":7546,"style":435},[293],[45,7548],{},[45,7550,172],{"className":7551},[221],[45,7553,7490],{"className":7554,"style":7555},[89,90],"margin-right:0.0359em;",[45,7557,1663],{"className":7558},[89],[45,7560,2397],{"className":7561},[89,90],[45,7563,177],{"className":7564},[228],[16,7566,7567,7570,7571,7599],{},[668,7568,7569],{},"Training a neural network"," = selecting parameters ",[45,7572,7574,7587],{"className":7573,"translate":49},[48],[45,7575,7577],{"className":7576},[53],[55,7578,7579],{"xmlns":57},[59,7580,7581,7585],{},[62,7582,7583],{},[65,7584,2634],{},[69,7586,2637],{"encoding":71},[45,7588,7590],{"className":7589,"ariaHidden":76},[75],[45,7591,7593,7596],{"className":7592},[80],[45,7594],{"className":7595,"style":2307},[84],[45,7597,2634],{"className":7598,"style":1707},[89,90]," that maximize the probability of the observed data:",[45,7601,7603],{"className":7602,"translate":49},[327],[45,7604,7606,7680],{"className":7605,"translate":49},[48],[45,7607,7609],{"className":7608},[53],[55,7610,7611],{"xmlns":57,"display":336},[59,7612,7613,7677],{},[62,7614,7615,7631,7633,7635,7637,7647,7653,7659,7661,7667,7669,7675],{},[342,7616,7617,7623],{},[2271,7618,7619,7621],{"accent":76},[65,7620,2634],{},[169,7622,2277],{},[62,7624,7625,7627,7629],{},[65,7626,4389],{"mathvariant":1662},[65,7628,2674],{"mathvariant":1662},[65,7630,4394],{"mathvariant":1662},[169,7632,359],{},[65,7634,2783],{},[169,7636,2786],{},[2788,7638,7639,7645],{},[62,7640,7641,7643],{},[65,7642,2794],{},[169,7644,2786],{},[65,7646,2634],{},[2788,7648,7649,7651],{},[169,7650,3461],{},[65,7652,3470],{},[342,7654,7655,7657],{},[65,7656,346],{},[65,7658,2634],{},[169,7660,172],{"stretchy":171},[342,7662,7663,7665],{},[65,7664,7490],{},[65,7666,3470],{},[65,7668,1663],{"mathvariant":1662},[342,7670,7671,7673],{},[65,7672,2397],{},[65,7674,3470],{},[169,7676,177],{"stretchy":171},[69,7678,7679],{"encoding":71},"\\hat{\\theta}_{\\rm MLE} = \\arg\\max_\\theta \\prod_i P_\\theta(y_i | x_i)",[45,7681,7683,7772],{"className":7682,"ariaHidden":76},[75],[45,7684,7686,7689,7763,7766,7769],{"className":7685},[80],[45,7687],{"className":7688,"style":4228},[84],[45,7690,7692,7723],{"className":7691},[89],[45,7693,7695],{"className":7694},[89,2297],[45,7696,7698],{"className":7697},[285],[45,7699,7701],{"className":7700},[289],[45,7702,7704,7712],{"className":7703,"style":2847},[293],[45,7705,7706,7709],{"style":2310},[45,7707],{"className":7708,"style":2314},[300],[45,7710,2634],{"className":7711,"style":1707},[89,90],[45,7713,7714,7717],{"style":2870},[45,7715],{"className":7716,"style":2314},[300],[45,7718,7720],{"className":7719,"style":2327},[2326],[45,7721,2277],{"className":7722},[89],[45,7724,7726],{"className":7725},[281],[45,7727,7729,7755],{"className":7728},[285,397],[45,7730,7732,7752],{"className":7731},[289],[45,7733,7735],{"className":7734,"style":4275},[293],[45,7736,7737,7740],{"style":4278},[45,7738],{"className":7739,"style":301},[300],[45,7741,7743],{"className":7742},[305,306,307,308],[45,7744,7746],{"className":7745},[89,308],[45,7747,7749],{"className":7748},[89,308],[45,7750,4215],{"className":7751},[89,4470,308],[45,7753,428],{"className":7754},[427],[45,7756,7758],{"className":7757},[289],[45,7759,7761],{"className":7760,"style":435},[293],[45,7762],{},[45,7764],{"className":7765,"style":233},[232],[45,7767,359],{"className":7768},[237],[45,7770],{"className":7771,"style":233},[232],[45,7773,7775,7779,7784,7787,7831,7834,7878,7881,7921,7924,7965,7968,8008],{"className":7774},[80],[45,7776],{"className":7777,"style":7778},[84],"height:2.3277em;vertical-align:-1.2777em;",[45,7780,2900,7782],{"className":7781},[2899],[45,7783,2904],{"style":2903},[45,7785],{"className":7786,"style":2510},[232],[45,7788,7790],{"className":7789},[2899,2911],[45,7791,7793,7823],{"className":7792},[285,397],[45,7794,7796,7820],{"className":7795},[289],[45,7797,7799,7810],{"className":7798,"style":85},[293],[45,7800,7801,7804],{"style":2923},[45,7802],{"className":7803,"style":2314},[300],[45,7805,7807],{"className":7806},[305,306,307,308],[45,7808,2634],{"className":7809,"style":1707},[89,90,308],[45,7811,7812,7815],{"style":2310},[45,7813],{"className":7814,"style":2314},[300],[45,7816,7817],{},[45,7818,2794],{"className":7819},[2899],[45,7821,428],{"className":7822},[427],[45,7824,7826],{"className":7825},[289],[45,7827,7829],{"className":7828,"style":2952},[293],[45,7830],{},[45,7832],{"className":7833,"style":2510},[232],[45,7835,7837],{"className":7836},[2899,2911],[45,7838,7840,7870],{"className":7839},[285,397],[45,7841,7843,7867],{"className":7842},[289],[45,7844,7846,7857],{"className":7845,"style":3550},[293],[45,7847,7848,7851],{"style":5329},[45,7849],{"className":7850,"style":3557},[300],[45,7852,7854],{"className":7853},[305,306,307,308],[45,7855,3470],{"className":7856},[89,90,308],[45,7858,7859,7862],{"style":3618},[45,7860],{"className":7861,"style":3557},[300],[45,7863,7864],{},[45,7865,3461],{"className":7866},[2899,3627,3628],[45,7868,428],{"className":7869},[427],[45,7871,7873],{"className":7872},[289],[45,7874,7876],{"className":7875,"style":5379},[293],[45,7877],{},[45,7879],{"className":7880,"style":2510},[232],[45,7882,7884,7887],{"className":7883},[89],[45,7885,346],{"className":7886,"style":390},[89,90],[45,7888,7890],{"className":7889},[281],[45,7891,7893,7913],{"className":7892},[285,397],[45,7894,7896,7910],{"className":7895},[289],[45,7897,7899],{"className":7898,"style":404},[293],[45,7900,7901,7904],{"style":407},[45,7902],{"className":7903,"style":301},[300],[45,7905,7907],{"className":7906},[305,306,307,308],[45,7908,2634],{"className":7909,"style":1707},[89,90,308],[45,7911,428],{"className":7912},[427],[45,7914,7916],{"className":7915},[289],[45,7917,7919],{"className":7918,"style":435},[293],[45,7920],{},[45,7922,172],{"className":7923},[221],[45,7925,7927,7930],{"className":7926},[89],[45,7928,7490],{"className":7929,"style":7555},[89,90],[45,7931,7933],{"className":7932},[281],[45,7934,7936,7957],{"className":7935},[285,397],[45,7937,7939,7954],{"className":7938},[289],[45,7940,7942],{"className":7941,"style":3668},[293],[45,7943,7945,7948],{"style":7944},"top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;",[45,7946],{"className":7947,"style":301},[300],[45,7949,7951],{"className":7950},[305,306,307,308],[45,7952,3470],{"className":7953},[89,90,308],[45,7955,428],{"className":7956},[427],[45,7958,7960],{"className":7959},[289],[45,7961,7963],{"className":7962,"style":435},[293],[45,7964],{},[45,7966,1663],{"className":7967},[89],[45,7969,7971,7974],{"className":7970},[89],[45,7972,2397],{"className":7973},[89,90],[45,7975,7977],{"className":7976},[281],[45,7978,7980,8000],{"className":7979},[285,397],[45,7981,7983,7997],{"className":7982},[289],[45,7984,7986],{"className":7985,"style":3668},[293],[45,7987,7988,7991],{"style":2482},[45,7989],{"className":7990,"style":301},[300],[45,7992,7994],{"className":7993},[305,306,307,308],[45,7995,3470],{"className":7996},[89,90,308],[45,7998,428],{"className":7999},[427],[45,8001,8003],{"className":8002},[289],[45,8004,8006],{"className":8005,"style":435},[293],[45,8007],{},[45,8009,177],{"className":8010},[228],[16,8012,8013],{},"Take classification as an example: a binary classification neural network outputs:",[45,8015,8017],{"className":8016,"translate":49},[327],[45,8018,8020,8059],{"className":8019,"translate":49},[48],[45,8021,8023],{"className":8022},[53],[55,8024,8025],{"xmlns":57,"display":336},[59,8026,8027,8056],{},[62,8028,8029,8035,8037,8040,8042,8048,8050,8052,8054],{},[2271,8030,8031,8033],{"accent":76},[65,8032,7490],{},[169,8034,2277],{},[169,8036,359],{},[65,8038,8039],{},"σ",[169,8041,172],{"stretchy":171},[342,8043,8044,8046],{},[65,8045,7390],{},[65,8047,2634],{},[169,8049,172],{"stretchy":171},[65,8051,2397],{},[169,8053,177],{"stretchy":171},[169,8055,177],{"stretchy":171},[69,8057,8058],{"encoding":71},"\\hat{y} = \\sigma(f_\\theta(x))",[45,8060,8062,8120],{"className":8061,"ariaHidden":76},[75],[45,8063,8065,8068,8111,8114,8117],{"className":8064},[80],[45,8066],{"className":8067,"style":925},[84],[45,8069,8071],{"className":8070},[89,2297],[45,8072,8074,8103],{"className":8073},[285,397],[45,8075,8077,8100],{"className":8076},[289],[45,8078,8080,8088],{"className":8079,"style":2307},[293],[45,8081,8082,8085],{"style":2310},[45,8083],{"className":8084,"style":2314},[300],[45,8086,7490],{"className":8087,"style":7555},[89,90],[45,8089,8090,8093],{"style":2310},[45,8091],{"className":8092,"style":2314},[300],[45,8094,8097],{"className":8095,"style":8096},[2326],"left:-0.1944em;",[45,8098,2277],{"className":8099},[89],[45,8101,428],{"className":8102},[427],[45,8104,8106],{"className":8105},[289],[45,8107,8109],{"className":8108,"style":2340},[293],[45,8110],{},[45,8112],{"className":8113,"style":233},[232],[45,8115,359],{"className":8116},[237],[45,8118],{"className":8119,"style":233},[232],[45,8121,8123,8126,8129,8132,8172,8175,8178],{"className":8122},[80],[45,8124],{"className":8125,"style":214},[84],[45,8127,8039],{"className":8128,"style":7555},[89,90],[45,8130,172],{"className":8131},[221],[45,8133,8135,8138],{"className":8134},[89],[45,8136,7390],{"className":8137,"style":7417},[89,90],[45,8139,8141],{"className":8140},[281],[45,8142,8144,8164],{"className":8143},[285,397],[45,8145,8147,8161],{"className":8146},[289],[45,8148,8150],{"className":8149,"style":404},[293],[45,8151,8152,8155],{"style":7432},[45,8153],{"className":8154,"style":301},[300],[45,8156,8158],{"className":8157},[305,306,307,308],[45,8159,2634],{"className":8160,"style":1707},[89,90,308],[45,8162,428],{"className":8163},[427],[45,8165,8167],{"className":8166},[289],[45,8168,8170],{"className":8169,"style":435},[293],[45,8171],{},[45,8173,172],{"className":8174},[221],[45,8176,2397],{"className":8177},[89,90],[45,8179,8181],{"className":8180},[228],"))",[16,8183,8184],{},"We see that maximizing the likelihood can be written as minimizing the cross-entropy loss function as the cost function:",[45,8186,8188],{"className":8187,"translate":49},[327],[45,8189,8191,8290],{"className":8190,"translate":49},[48],[45,8192,8194],{"className":8193},[53],[55,8195,8196],{"xmlns":57,"display":336},[59,8197,8198,8287],{},[62,8199,8200,8216,8218,8220,8222,8232,8238,8255,8257,8259,8261,8271],{},[342,8201,8202,8208],{},[2271,8203,8204,8206],{"accent":76},[65,8205,2634],{},[169,8207,2277],{},[62,8209,8210,8212,8214],{},[65,8211,4389],{"mathvariant":1662},[65,8213,2674],{"mathvariant":1662},[65,8215,4394],{"mathvariant":1662},[169,8217,359],{},[65,8219,2783],{},[169,8221,2786],{},[2788,8223,8224,8230],{},[62,8225,8226,8228],{},[65,8227,2794],{},[169,8229,2786],{},[65,8231,2634],{},[2788,8233,8234,8236],{},[169,8235,3461],{},[65,8237,3470],{},[8239,8240,8241,8247,8249],"msubsup",{},[2271,8242,8243,8245],{"accent":76},[65,8244,7490],{},[169,8246,2277],{},[65,8248,3470],{},[342,8250,8251,8253],{},[65,8252,7490],{},[65,8254,3470],{},[169,8256,172],{"stretchy":171},[182,8258,184],{},[169,8260,187],{},[342,8262,8263,8269],{},[2271,8264,8265,8267],{"accent":76},[65,8266,7490],{},[169,8268,2277],{},[65,8270,3470],{},[191,8272,8273,8275],{},[169,8274,177],{"stretchy":171},[62,8276,8277,8279,8281],{},[182,8278,184],{},[169,8280,187],{},[342,8282,8283,8285],{},[65,8284,7490],{},[65,8286,3470],{},[69,8288,8289],{"encoding":71},"\\hat{\\theta}_{\\rm MLE} = \\arg\\max_\\theta \\prod_i \\hat{y}_i^{y_i} (1-\\hat{y}_i)^{1-y_i}",[45,8291,8293,8382,8640],{"className":8292,"ariaHidden":76},[75],[45,8294,8296,8299,8373,8376,8379],{"className":8295},[80],[45,8297],{"className":8298,"style":4228},[84],[45,8300,8302,8333],{"className":8301},[89],[45,8303,8305],{"className":8304},[89,2297],[45,8306,8308],{"className":8307},[285],[45,8309,8311],{"className":8310},[289],[45,8312,8314,8322],{"className":8313,"style":2847},[293],[45,8315,8316,8319],{"style":2310},[45,8317],{"className":8318,"style":2314},[300],[45,8320,2634],{"className":8321,"style":1707},[89,90],[45,8323,8324,8327],{"style":2870},[45,8325],{"className":8326,"style":2314},[300],[45,8328,8330],{"className":8329,"style":2327},[2326],[45,8331,2277],{"className":8332},[89],[45,8334,8336],{"className":8335},[281],[45,8337,8339,8365],{"className":8338},[285,397],[45,8340,8342,8362],{"className":8341},[289],[45,8343,8345],{"className":8344,"style":4275},[293],[45,8346,8347,8350],{"style":4278},[45,8348],{"className":8349,"style":301},[300],[45,8351,8353],{"className":8352},[305,306,307,308],[45,8354,8356],{"className":8355},[89,308],[45,8357,8359],{"className":8358},[89,308],[45,8360,4215],{"className":8361},[89,4470,308],[45,8363,428],{"className":8364},[427],[45,8366,8368],{"className":8367},[289],[45,8369,8371],{"className":8370,"style":435},[293],[45,8372],{},[45,8374],{"className":8375,"style":233},[232],[45,8377,359],{"className":8378},[237],[45,8380],{"className":8381,"style":233},[232],[45,8383,8385,8388,8393,8396,8440,8443,8487,8490,8625,8628,8631,8634,8637],{"className":8384},[80],[45,8386],{"className":8387,"style":7778},[84],[45,8389,2900,8391],{"className":8390},[2899],[45,8392,2904],{"style":2903},[45,8394],{"className":8395,"style":2510},[232],[45,8397,8399],{"className":8398},[2899,2911],[45,8400,8402,8432],{"className":8401},[285,397],[45,8403,8405,8429],{"className":8404},[289],[45,8406,8408,8419],{"className":8407,"style":85},[293],[45,8409,8410,8413],{"style":2923},[45,8411],{"className":8412,"style":2314},[300],[45,8414,8416],{"className":8415},[305,306,307,308],[45,8417,2634],{"className":8418,"style":1707},[89,90,308],[45,8420,8421,8424],{"style":2310},[45,8422],{"className":8423,"style":2314},[300],[45,8425,8426],{},[45,8427,2794],{"className":8428},[2899],[45,8430,428],{"className":8431},[427],[45,8433,8435],{"className":8434},[289],[45,8436,8438],{"className":8437,"style":2952},[293],[45,8439],{},[45,8441],{"className":8442,"style":2510},[232],[45,8444,8446],{"className":8445},[2899,2911],[45,8447,8449,8479],{"className":8448},[285,397],[45,8450,8452,8476],{"className":8451},[289],[45,8453,8455,8466],{"className":8454,"style":3550},[293],[45,8456,8457,8460],{"style":5329},[45,8458],{"className":8459,"style":3557},[300],[45,8461,8463],{"className":8462},[305,306,307,308],[45,8464,3470],{"className":8465},[89,90,308],[45,8467,8468,8471],{"style":3618},[45,8469],{"className":8470,"style":3557},[300],[45,8472,8473],{},[45,8474,3461],{"className":8475},[2899,3627,3628],[45,8477,428],{"className":8478},[427],[45,8480,8482],{"className":8481},[289],[45,8483,8485],{"className":8484,"style":5379},[293],[45,8486],{},[45,8488],{"className":8489,"style":2510},[232],[45,8491,8493,8535],{"className":8492},[89],[45,8494,8496],{"className":8495},[89,2297],[45,8497,8499,8527],{"className":8498},[285,397],[45,8500,8502,8524],{"className":8501},[289],[45,8503,8505,8513],{"className":8504,"style":2307},[293],[45,8506,8507,8510],{"style":2310},[45,8508],{"className":8509,"style":2314},[300],[45,8511,7490],{"className":8512,"style":7555},[89,90],[45,8514,8515,8518],{"style":2310},[45,8516],{"className":8517,"style":2314},[300],[45,8519,8521],{"className":8520,"style":8096},[2326],[45,8522,2277],{"className":8523},[89],[45,8525,428],{"className":8526},[427],[45,8528,8530],{"className":8529},[289],[45,8531,8533],{"className":8532,"style":2340},[293],[45,8534],{},[45,8536,8538],{"className":8537},[281],[45,8539,8541,8616],{"className":8540},[285,397],[45,8542,8544,8613],{"className":8543},[289],[45,8545,8548,8560],{"className":8546,"style":8547},[293],"height:0.7823em;",[45,8549,8551,8554],{"style":8550},"top:-2.4231em;margin-left:-0.0359em;margin-right:0.05em;",[45,8552],{"className":8553,"style":301},[300],[45,8555,8557],{"className":8556},[305,306,307,308],[45,8558,3470],{"className":8559},[89,90,308],[45,8561,8563,8566],{"style":8562},"top:-3.1809em;margin-right:0.05em;",[45,8564],{"className":8565,"style":301},[300],[45,8567,8569],{"className":8568},[305,306,307,308],[45,8570,8572],{"className":8571},[89,308],[45,8573,8575,8578],{"className":8574},[89,308],[45,8576,7490],{"className":8577,"style":7555},[89,90,308],[45,8579,8581],{"className":8580},[281],[45,8582,8584,8605],{"className":8583},[285,397],[45,8585,8587,8602],{"className":8586},[289],[45,8588,8590],{"className":8589,"style":3582},[293],[45,8591,8593,8596],{"style":8592},"top:-2.357em;margin-left:-0.0359em;margin-right:0.0714em;",[45,8594],{"className":8595,"style":3589},[300],[45,8597,8599],{"className":8598},[305,3593,3594,308],[45,8600,3470],{"className":8601},[89,90,308],[45,8603,428],{"className":8604},[427],[45,8606,8608],{"className":8607},[289],[45,8609,8611],{"className":8610,"style":3607},[293],[45,8612],{},[45,8614,428],{"className":8615},[427],[45,8617,8619],{"className":8618},[289],[45,8620,8623],{"className":8621,"style":8622},[293],"height:0.2769em;",[45,8624],{},[45,8626,172],{"className":8627},[221],[45,8629,184],{"className":8630},[89],[45,8632],{"className":8633,"style":254},[232],[45,8635,187],{"className":8636},[258],[45,8638],{"className":8639,"style":254},[232],[45,8641,8643,8646,8725],{"className":8642},[80],[45,8644],{"className":8645,"style":5475},[84],[45,8647,8649,8691],{"className":8648},[89],[45,8650,8652],{"className":8651},[89,2297],[45,8653,8655,8683],{"className":8654},[285,397],[45,8656,8658,8680],{"className":8657},[289],[45,8659,8661,8669],{"className":8660,"style":2307},[293],[45,8662,8663,8666],{"style":2310},[45,8664],{"className":8665,"style":2314},[300],[45,8667,7490],{"className":8668,"style":7555},[89,90],[45,8670,8671,8674],{"style":2310},[45,8672],{"className":8673,"style":2314},[300],[45,8675,8677],{"className":8676,"style":8096},[2326],[45,8678,2277],{"className":8679},[89],[45,8681,428],{"className":8682},[427],[45,8684,8686],{"className":8685},[289],[45,8687,8689],{"className":8688,"style":2340},[293],[45,8690],{},[45,8692,8694],{"className":8693},[281],[45,8695,8697,8717],{"className":8696},[285,397],[45,8698,8700,8714],{"className":8699},[289],[45,8701,8703],{"className":8702,"style":3668},[293],[45,8704,8705,8708],{"style":7944},[45,8706],{"className":8707,"style":301},[300],[45,8709,8711],{"className":8710},[305,306,307,308],[45,8712,3470],{"className":8713},[89,90,308],[45,8715,428],{"className":8716},[427],[45,8718,8720],{"className":8719},[289],[45,8721,8723],{"className":8722,"style":435},[293],[45,8724],{},[45,8726,8728,8731],{"className":8727},[228],[45,8729,177],{"className":8730},[228],[45,8732,8734],{"className":8733},[281],[45,8735,8737],{"className":8736},[285],[45,8738,8740],{"className":8739},[289],[45,8741,8743],{"className":8742,"style":1750},[293],[45,8744,8745,8748],{"style":492},[45,8746],{"className":8747,"style":301},[300],[45,8749,8751],{"className":8750},[305,306,307,308],[45,8752,8754,8757,8760],{"className":8753},[89,308],[45,8755,184],{"className":8756},[89,308],[45,8758,187],{"className":8759},[258,308],[45,8761,8763,8766],{"className":8762},[89,308],[45,8764,7490],{"className":8765,"style":7555},[89,90,308],[45,8767,8769],{"className":8768},[281],[45,8770,8772,8792],{"className":8771},[285,397],[45,8773,8775,8789],{"className":8774},[289],[45,8776,8778],{"className":8777,"style":3582},[293],[45,8779,8780,8783],{"style":8592},[45,8781],{"className":8782,"style":3589},[300],[45,8784,8786],{"className":8785},[305,3593,3594,308],[45,8787,3470],{"className":8788},[89,90,308],[45,8790,428],{"className":8791},[427],[45,8793,8795],{"className":8794},[289],[45,8796,8798],{"className":8797,"style":3607},[293],[45,8799],{},[16,8801,8802],{},"Finally, we obtain the cross-entropy loss:",[45,8804,8806],{"className":8805,"translate":49},[327],[45,8807,8809,8904],{"className":8808,"translate":49},[48],[45,8810,8812],{"className":8811},[53],[55,8813,8814],{"xmlns":57,"display":336},[59,8815,8816,8901],{},[62,8817,8818,8820,8822,8824,8826,8828,8830,8836,8840,8846,8848,8850,8860,8862,8864,8866,8868,8874,8876,8878,8880,8882,8884,8886,8896,8898],{},[65,8819,2674],{},[169,8821,172],{"stretchy":171},[65,8823,2634],{},[169,8825,177],{"stretchy":171},[169,8827,359],{},[169,8829,187],{},[2788,8831,8832,8834],{},[169,8833,3788],{},[65,8835,3470],{},[169,8837,8839],{"fence":171,"stretchy":76,"minsize":8838,"maxsize":8838},"1.8em","[",[342,8841,8842,8844],{},[65,8843,7490],{},[65,8845,3470],{},[65,8847,3767],{},[169,8849,2786],{},[342,8851,8852,8858],{},[2271,8853,8854,8856],{"accent":76},[65,8855,7490],{},[169,8857,2277],{},[65,8859,3470],{},[169,8861,5614],{},[169,8863,172],{"stretchy":171},[182,8865,184],{},[169,8867,187],{},[342,8869,8870,8872],{},[65,8871,7490],{},[65,8873,3470],{},[169,8875,177],{"stretchy":171},[65,8877,3767],{},[169,8879,2786],{},[169,8881,172],{"stretchy":171},[182,8883,184],{},[169,8885,187],{},[342,8887,8888,8894],{},[2271,8889,8890,8892],{"accent":76},[65,8891,7490],{},[169,8893,2277],{},[65,8895,3470],{},[169,8897,177],{"stretchy":171},[169,8899,8900],{"fence":171,"stretchy":76,"minsize":8838,"maxsize":8838},"]",[69,8902,8903],{"encoding":71},"L(\\theta) = - \\sum_i \\Big[ y_i \\log \\hat{y}_i + (1-y_i) \\log (1-\\hat{y}_i) \\Big]",[45,8905,8907,8934,9141,9162,9234],{"className":8906,"ariaHidden":76},[75],[45,8908,8910,8913,8916,8919,8922,8925,8928,8931],{"className":8909},[80],[45,8911],{"className":8912,"style":214},[84],[45,8914,2674],{"className":8915},[89,90],[45,8917,172],{"className":8918},[221],[45,8920,2634],{"className":8921,"style":1707},[89,90],[45,8923,177],{"className":8924},[228],[45,8926],{"className":8927,"style":233},[232],[45,8929,359],{"className":8930},[237],[45,8932],{"className":8933,"style":233},[232],[45,8935,8937,8941,8944,8947,8991,8994,9002,9042,9045,9050,9053,9132,9135,9138],{"className":8936},[80],[45,8938],{"className":8939,"style":8940},[84],"height:2.4277em;vertical-align:-1.2777em;",[45,8942,187],{"className":8943},[89],[45,8945],{"className":8946,"style":2510},[232],[45,8948,8950],{"className":8949},[2899,2911],[45,8951,8953,8983],{"className":8952},[285,397],[45,8954,8956,8980],{"className":8955},[289],[45,8957,8959,8970],{"className":8958,"style":3550},[293],[45,8960,8961,8964],{"style":5329},[45,8962],{"className":8963,"style":3557},[300],[45,8965,8967],{"className":8966},[305,306,307,308],[45,8968,3470],{"className":8969},[89,90,308],[45,8971,8972,8975],{"style":3618},[45,8973],{"className":8974,"style":3557},[300],[45,8976,8977],{},[45,8978,3788],{"className":8979},[2899,3627,3628],[45,8981,428],{"className":8982},[427],[45,8984,8986],{"className":8985},[289],[45,8987,8989],{"className":8988,"style":5379},[293],[45,8990],{},[45,8992],{"className":8993,"style":2510},[232],[45,8995,8997],{"className":8996},[89],[45,8998,8839],{"className":8999},[9000,9001],"delimsizing","size2",[45,9003,9005,9008],{"className":9004},[89],[45,9006,7490],{"className":9007,"style":7555},[89,90],[45,9009,9011],{"className":9010},[281],[45,9012,9014,9034],{"className":9013},[285,397],[45,9015,9017,9031],{"className":9016},[289],[45,9018,9020],{"className":9019,"style":3668},[293],[45,9021,9022,9025],{"style":7944},[45,9023],{"className":9024,"style":301},[300],[45,9026,9028],{"className":9027},[305,306,307,308],[45,9029,3470],{"className":9030},[89,90,308],[45,9032,428],{"className":9033},[427],[45,9035,9037],{"className":9036},[289],[45,9038,9040],{"className":9039,"style":435},[293],[45,9041],{},[45,9043],{"className":9044,"style":2510},[232],[45,9046,3869,9048],{"className":9047},[2899],[45,9049,2904],{"style":2903},[45,9051],{"className":9052,"style":2510},[232],[45,9054,9056,9098],{"className":9055},[89],[45,9057,9059],{"className":9058},[89,2297],[45,9060,9062,9090],{"className":9061},[285,397],[45,9063,9065,9087],{"className":9064},[289],[45,9066,9068,9076],{"className":9067,"style":2307},[293],[45,9069,9070,9073],{"style":2310},[45,9071],{"className":9072,"style":2314},[300],[45,9074,7490],{"className":9075,"style":7555},[89,90],[45,9077,9078,9081],{"style":2310},[45,9079],{"className":9080,"style":2314},[300],[45,9082,9084],{"className":9083,"style":8096},[2326],[45,9085,2277],{"className":9086},[89],[45,9088,428],{"className":9089},[427],[45,9091,9093],{"className":9092},[289],[45,9094,9096],{"className":9095,"style":2340},[293],[45,9097],{},[45,9099,9101],{"className":9100},[281],[45,9102,9104,9124],{"className":9103},[285,397],[45,9105,9107,9121],{"className":9106},[289],[45,9108,9110],{"className":9109,"style":3668},[293],[45,9111,9112,9115],{"style":7944},[45,9113],{"className":9114,"style":301},[300],[45,9116,9118],{"className":9117},[305,306,307,308],[45,9119,3470],{"className":9120},[89,90,308],[45,9122,428],{"className":9123},[427],[45,9125,9127],{"className":9126},[289],[45,9128,9130],{"className":9129,"style":435},[293],[45,9131],{},[45,9133],{"className":9134,"style":254},[232],[45,9136,5614],{"className":9137},[258],[45,9139],{"className":9140,"style":254},[232],[45,9142,9144,9147,9150,9153,9156,9159],{"className":9143},[80],[45,9145],{"className":9146,"style":214},[84],[45,9148,172],{"className":9149},[221],[45,9151,184],{"className":9152},[89],[45,9154],{"className":9155,"style":254},[232],[45,9157,187],{"className":9158},[258],[45,9160],{"className":9161,"style":254},[232],[45,9163,9165,9168,9208,9211,9214,9219,9222,9225,9228,9231],{"className":9164},[80],[45,9166],{"className":9167,"style":214},[84],[45,9169,9171,9174],{"className":9170},[89],[45,9172,7490],{"className":9173,"style":7555},[89,90],[45,9175,9177],{"className":9176},[281],[45,9178,9180,9200],{"className":9179},[285,397],[45,9181,9183,9197],{"className":9182},[289],[45,9184,9186],{"className":9185,"style":3668},[293],[45,9187,9188,9191],{"style":7944},[45,9189],{"className":9190,"style":301},[300],[45,9192,9194],{"className":9193},[305,306,307,308],[45,9195,3470],{"className":9196},[89,90,308],[45,9198,428],{"className":9199},[427],[45,9201,9203],{"className":9202},[289],[45,9204,9206],{"className":9205,"style":435},[293],[45,9207],{},[45,9209,177],{"className":9210},[228],[45,9212],{"className":9213,"style":2510},[232],[45,9215,3869,9217],{"className":9216},[2899],[45,9218,2904],{"style":2903},[45,9220,172],{"className":9221},[221],[45,9223,184],{"className":9224},[89],[45,9226],{"className":9227,"style":254},[232],[45,9229,187],{"className":9230},[258],[45,9232],{"className":9233,"style":254},[232],[45,9235,9237,9241,9320,9323],{"className":9236},[80],[45,9238],{"className":9239,"style":9240},[84],"height:1.8em;vertical-align:-0.65em;",[45,9242,9244,9286],{"className":9243},[89],[45,9245,9247],{"className":9246},[89,2297],[45,9248,9250,9278],{"className":9249},[285,397],[45,9251,9253,9275],{"className":9252},[289],[45,9254,9256,9264],{"className":9255,"style":2307},[293],[45,9257,9258,9261],{"style":2310},[45,9259],{"className":9260,"style":2314},[300],[45,9262,7490],{"className":9263,"style":7555},[89,90],[45,9265,9266,9269],{"style":2310},[45,9267],{"className":9268,"style":2314},[300],[45,9270,9272],{"className":9271,"style":8096},[2326],[45,9273,2277],{"className":9274},[89],[45,9276,428],{"className":9277},[427],[45,9279,9281],{"className":9280},[289],[45,9282,9284],{"className":9283,"style":2340},[293],[45,9285],{},[45,9287,9289],{"className":9288},[281],[45,9290,9292,9312],{"className":9291},[285,397],[45,9293,9295,9309],{"className":9294},[289],[45,9296,9298],{"className":9297,"style":3668},[293],[45,9299,9300,9303],{"style":7944},[45,9301],{"className":9302,"style":301},[300],[45,9304,9306],{"className":9305},[305,306,307,308],[45,9307,3470],{"className":9308},[89,90,308],[45,9310,428],{"className":9311},[427],[45,9313,9315],{"className":9314},[289],[45,9316,9318],{"className":9317,"style":435},[293],[45,9319],{},[45,9321,177],{"className":9322},[228],[45,9324,9326],{"className":9325},[89],[45,9327,8900],{"className":9328},[9000,9001],[16,9330,9331,9332,9360],{},"Here ",[45,9333,9335,9348],{"className":9334,"translate":49},[48],[45,9336,9338],{"className":9337},[53],[55,9339,9340],{"xmlns":57},[59,9341,9342,9346],{},[62,9343,9344],{},[65,9345,2634],{},[69,9347,2637],{"encoding":71},[45,9349,9351],{"className":9350,"ariaHidden":76},[75],[45,9352,9354,9357],{"className":9353},[80],[45,9355],{"className":9356,"style":2307},[84],[45,9358,2634],{"className":9359,"style":1707},[89,90]," represents all weights and biases.",[16,9362,9363],{},"Take regression as another example: suppose",[45,9365,9367],{"className":9366,"translate":49},[327],[45,9368,9370,9445],{"className":9369,"translate":49},[48],[45,9371,9373],{"className":9372},[53],[55,9374,9375],{"xmlns":57,"display":336},[59,9376,9377,9442],{},[62,9378,9379,9385,9387,9393,9395,9401,9403,9405,9412,9414,9417,9423,9426,9428,9430,9432,9434,9440],{},[342,9380,9381,9383],{},[65,9382,7490],{},[65,9384,3470],{},[169,9386,359],{},[342,9388,9389,9391],{},[65,9390,7390],{},[65,9392,2634],{},[169,9394,172],{"stretchy":171},[342,9396,9397,9399],{},[65,9398,2397],{},[65,9400,3470],{},[169,9402,177],{"stretchy":171},[169,9404,5614],{},[342,9406,9407,9410],{},[65,9408,9409],{},"ϵ",[65,9411,3470],{},[169,9413,2402],{"separator":76},[232,9415],{"width":9416},"1em",[342,9418,9419,9421],{},[65,9420,9409],{},[65,9422,3470],{},[169,9424,9425],{},"∼",[65,9427,136],{},[169,9429,172],{"stretchy":171},[182,9431,4528],{},[169,9433,2402],{"separator":76},[191,9435,9436,9438],{},[65,9437,8039],{},[182,9439,1947],{},[169,9441,177],{"stretchy":171},[69,9443,9444],{"encoding":71},"y_i = f_\\theta(x_i) + \\epsilon_i, \\quad \\epsilon_i \\sim N(0, \\sigma^2)",[45,9446,9448,9503,9604,9709],{"className":9447,"ariaHidden":76},[75],[45,9449,9451,9454,9494,9497,9500],{"className":9450},[80],[45,9452],{"className":9453,"style":117},[84],[45,9455,9457,9460],{"className":9456},[89],[45,9458,7490],{"className":9459,"style":7555},[89,90],[45,9461,9463],{"className":9462},[281],[45,9464,9466,9486],{"className":9465},[285,397],[45,9467,9469,9483],{"className":9468},[289],[45,9470,9472],{"className":9471,"style":3668},[293],[45,9473,9474,9477],{"style":7944},[45,9475],{"className":9476,"style":301},[300],[45,9478,9480],{"className":9479},[305,306,307,308],[45,9481,3470],{"className":9482},[89,90,308],[45,9484,428],{"className":9485},[427],[45,9487,9489],{"className":9488},[289],[45,9490,9492],{"className":9491,"style":435},[293],[45,9493],{},[45,9495],{"className":9496,"style":233},[232],[45,9498,359],{"className":9499},[237],[45,9501],{"className":9502,"style":233},[232],[45,9504,9506,9509,9549,9552,9592,9595,9598,9601],{"className":9505},[80],[45,9507],{"className":9508,"style":214},[84],[45,9510,9512,9515],{"className":9511},[89],[45,9513,7390],{"className":9514,"style":7417},[89,90],[45,9516,9518],{"className":9517},[281],[45,9519,9521,9541],{"className":9520},[285,397],[45,9522,9524,9538],{"className":9523},[289],[45,9525,9527],{"className":9526,"style":404},[293],[45,9528,9529,9532],{"style":7432},[45,9530],{"className":9531,"style":301},[300],[45,9533,9535],{"className":9534},[305,306,307,308],[45,9536,2634],{"className":9537,"style":1707},[89,90,308],[45,9539,428],{"className":9540},[427],[45,9542,9544],{"className":9543},[289],[45,9545,9547],{"className":9546,"style":435},[293],[45,9548],{},[45,9550,172],{"className":9551},[221],[45,9553,9555,9558],{"className":9554},[89],[45,9556,2397],{"className":9557},[89,90],[45,9559,9561],{"className":9560},[281],[45,9562,9564,9584],{"className":9563},[285,397],[45,9565,9567,9581],{"className":9566},[289],[45,9568,9570],{"className":9569,"style":3668},[293],[45,9571,9572,9575],{"style":2482},[45,9573],{"className":9574,"style":301},[300],[45,9576,9578],{"className":9577},[305,306,307,308],[45,9579,3470],{"className":9580},[89,90,308],[45,9582,428],{"className":9583},[427],[45,9585,9587],{"className":9586},[289],[45,9588,9590],{"className":9589,"style":435},[293],[45,9591],{},[45,9593,177],{"className":9594},[228],[45,9596],{"className":9597,"style":254},[232],[45,9599,5614],{"className":9600},[258],[45,9602],{"className":9603,"style":254},[232],[45,9605,9607,9610,9650,9653,9657,9660,9700,9703,9706],{"className":9606},[80],[45,9608],{"className":9609,"style":117},[84],[45,9611,9613,9616],{"className":9612},[89],[45,9614,9409],{"className":9615},[89,90],[45,9617,9619],{"className":9618},[281],[45,9620,9622,9642],{"className":9621},[285,397],[45,9623,9625,9639],{"className":9624},[289],[45,9626,9628],{"className":9627,"style":3668},[293],[45,9629,9630,9633],{"style":2482},[45,9631],{"className":9632,"style":301},[300],[45,9634,9636],{"className":9635},[305,306,307,308],[45,9637,3470],{"className":9638},[89,90,308],[45,9640,428],{"className":9641},[427],[45,9643,9645],{"className":9644},[289],[45,9646,9648],{"className":9647,"style":435},[293],[45,9649],{},[45,9651,2402],{"className":9652},[2506],[45,9654],{"className":9655,"style":9656},[232],"margin-right:1em;",[45,9658],{"className":9659,"style":2510},[232],[45,9661,9663,9666],{"className":9662},[89],[45,9664,9409],{"className":9665},[89,90],[45,9667,9669],{"className":9668},[281],[45,9670,9672,9692],{"className":9671},[285,397],[45,9673,9675,9689],{"className":9674},[289],[45,9676,9678],{"className":9677,"style":3668},[293],[45,9679,9680,9683],{"style":2482},[45,9681],{"className":9682,"style":301},[300],[45,9684,9686],{"className":9685},[305,306,307,308],[45,9687,3470],{"className":9688},[89,90,308],[45,9690,428],{"className":9691},[427],[45,9693,9695],{"className":9694},[289],[45,9696,9698],{"className":9697,"style":435},[293],[45,9699],{},[45,9701],{"className":9702,"style":233},[232],[45,9704,9425],{"className":9705},[237],[45,9707],{"className":9708,"style":233},[232],[45,9710,9712,9715,9718,9721,9724,9727,9730,9759],{"className":9711},[80],[45,9713],{"className":9714,"style":5475},[84],[45,9716,136],{"className":9717,"style":152},[89,90],[45,9719,172],{"className":9720},[221],[45,9722,4528],{"className":9723},[89],[45,9725,2402],{"className":9726},[2506],[45,9728],{"className":9729,"style":2510},[232],[45,9731,9733,9736],{"className":9732},[89],[45,9734,8039],{"className":9735,"style":7555},[89,90],[45,9737,9739],{"className":9738},[281],[45,9740,9742],{"className":9741},[285],[45,9743,9745],{"className":9744},[289],[45,9746,9748],{"className":9747,"style":1750},[293],[45,9749,9750,9753],{"style":492},[45,9751],{"className":9752,"style":301},[300],[45,9754,9756],{"className":9755},[305,306,307,308],[45,9757,1947],{"className":9758},[89,308],[45,9760,177],{"className":9761},[228],[16,9763,9764],{},"Log-likelihood:",[45,9766,9768],{"className":9767,"translate":49},[327],[45,9769,9771,9852],{"className":9770,"translate":49},[48],[45,9772,9774],{"className":9773},[53],[55,9775,9776],{"xmlns":57,"display":336},[59,9777,9778,9849],{},[62,9779,9780,9782,9784,9786,9788,9790,9792,9806,9812,9814,9820,9822,9828,9830,9836,9838,9844,9846],{},[65,9781,3752],{"mathvariant":1662},[169,9783,172],{"stretchy":171},[65,9785,2634],{},[169,9787,177],{"stretchy":171},[169,9789,359],{},[169,9791,187],{},[4499,9793,9794,9796],{},[182,9795,184],{},[62,9797,9798,9800],{},[182,9799,1947],{},[191,9801,9802,9804],{},[65,9803,8039],{},[182,9805,1947],{},[2788,9807,9808,9810],{},[169,9809,3788],{},[65,9811,3470],{},[169,9813,172],{"stretchy":171},[342,9815,9816,9818],{},[65,9817,7490],{},[65,9819,3470],{},[169,9821,187],{},[342,9823,9824,9826],{},[65,9825,7390],{},[65,9827,2634],{},[169,9829,172],{"stretchy":171},[342,9831,9832,9834],{},[65,9833,2397],{},[65,9835,3470],{},[169,9837,177],{"stretchy":171},[191,9839,9840,9842],{},[169,9841,177],{"stretchy":171},[182,9843,1947],{},[169,9845,5614],{},[65,9847,9848],{},"C",[69,9850,9851],{"encoding":71},"\\ell(\\theta) = -\\frac{1}{2\\sigma^2} \\sum_i (y_i - f_\\theta(x_i))^2 + C",[45,9853,9855,9882,10083,10213],{"className":9854,"ariaHidden":76},[75],[45,9856,9858,9861,9864,9867,9870,9873,9876,9879],{"className":9857},[80],[45,9859],{"className":9860,"style":214},[84],[45,9862,3752],{"className":9863},[89],[45,9865,172],{"className":9866},[221],[45,9868,2634],{"className":9869,"style":1707},[89,90],[45,9871,177],{"className":9872},[228],[45,9874],{"className":9875,"style":233},[232],[45,9877,359],{"className":9878},[237],[45,9880],{"className":9881,"style":233},[232],[45,9883,9885,9889,9892,9984,9987,10031,10034,10074,10077,10080],{"className":9884},[80],[45,9886],{"className":9887,"style":9888},[84],"height:2.5991em;vertical-align:-1.2777em;",[45,9890,187],{"className":9891},[89],[45,9893,9895,9898,9981],{"className":9894},[89],[45,9896],{"className":9897},[221,4548],[45,9899,9901],{"className":9900},[4499],[45,9902,9904,9973],{"className":9903},[285,397],[45,9905,9907,9970],{"className":9906},[289],[45,9908,9911,9951,9959],{"className":9909,"style":9910},[293],"height:1.3214em;",[45,9912,9913,9916],{"style":4564},[45,9914],{"className":9915,"style":2314},[300],[45,9917,9919,9922],{"className":9918},[89],[45,9920,1947],{"className":9921},[89],[45,9923,9925,9928],{"className":9924},[89],[45,9926,8039],{"className":9927,"style":7555},[89,90],[45,9929,9931],{"className":9930},[281],[45,9932,9934],{"className":9933},[285],[45,9935,9937],{"className":9936},[289],[45,9938,9940],{"className":9939,"style":4767},[293],[45,9941,9942,9945],{"style":4770},[45,9943],{"className":9944,"style":301},[300],[45,9946,9948],{"className":9947},[305,306,307,308],[45,9949,1947],{"className":9950},[89,308],[45,9952,9953,9956],{"style":4579},[45,9954],{"className":9955,"style":2314},[300],[45,9957],{"className":9958,"style":4587},[4586],[45,9960,9961,9964],{"style":4590},[45,9962],{"className":9963,"style":2314},[300],[45,9965,9967],{"className":9966},[89],[45,9968,184],{"className":9969},[89],[45,9971,428],{"className":9972},[427],[45,9974,9976],{"className":9975},[289],[45,9977,9979],{"className":9978,"style":4627},[293],[45,9980],{},[45,9982],{"className":9983},[228,4548],[45,9985],{"className":9986,"style":2510},[232],[45,9988,9990],{"className":9989},[2899,2911],[45,9991,9993,10023],{"className":9992},[285,397],[45,9994,9996,10020],{"className":9995},[289],[45,9997,9999,10010],{"className":9998,"style":3550},[293],[45,10000,10001,10004],{"style":5329},[45,10002],{"className":10003,"style":3557},[300],[45,10005,10007],{"className":10006},[305,306,307,308],[45,10008,3470],{"className":10009},[89,90,308],[45,10011,10012,10015],{"style":3618},[45,10013],{"className":10014,"style":3557},[300],[45,10016,10017],{},[45,10018,3788],{"className":10019},[2899,3627,3628],[45,10021,428],{"className":10022},[427],[45,10024,10026],{"className":10025},[289],[45,10027,10029],{"className":10028,"style":5379},[293],[45,10030],{},[45,10032,172],{"className":10033},[221],[45,10035,10037,10040],{"className":10036},[89],[45,10038,7490],{"className":10039,"style":7555},[89,90],[45,10041,10043],{"className":10042},[281],[45,10044,10046,10066],{"className":10045},[285,397],[45,10047,10049,10063],{"className":10048},[289],[45,10050,10052],{"className":10051,"style":3668},[293],[45,10053,10054,10057],{"style":7944},[45,10055],{"className":10056,"style":301},[300],[45,10058,10060],{"className":10059},[305,306,307,308],[45,10061,3470],{"className":10062},[89,90,308],[45,10064,428],{"className":10065},[427],[45,10067,10069],{"className":10068},[289],[45,10070,10072],{"className":10071,"style":435},[293],[45,10073],{},[45,10075],{"className":10076,"style":254},[232],[45,10078,187],{"className":10079},[258],[45,10081],{"className":10082,"style":254},[232],[45,10084,10086,10089,10129,10132,10172,10175,10204,10207,10210],{"className":10085},[80],[45,10087],{"className":10088,"style":5475},[84],[45,10090,10092,10095],{"className":10091},[89],[45,10093,7390],{"className":10094,"style":7417},[89,90],[45,10096,10098],{"className":10097},[281],[45,10099,10101,10121],{"className":10100},[285,397],[45,10102,10104,10118],{"className":10103},[289],[45,10105,10107],{"className":10106,"style":404},[293],[45,10108,10109,10112],{"style":7432},[45,10110],{"className":10111,"style":301},[300],[45,10113,10115],{"className":10114},[305,306,307,308],[45,10116,2634],{"className":10117,"style":1707},[89,90,308],[45,10119,428],{"className":10120},[427],[45,10122,10124],{"className":10123},[289],[45,10125,10127],{"className":10126,"style":435},[293],[45,10128],{},[45,10130,172],{"className":10131},[221],[45,10133,10135,10138],{"className":10134},[89],[45,10136,2397],{"className":10137},[89,90],[45,10139,10141],{"className":10140},[281],[45,10142,10144,10164],{"className":10143},[285,397],[45,10145,10147,10161],{"className":10146},[289],[45,10148,10150],{"className":10149,"style":3668},[293],[45,10151,10152,10155],{"style":2482},[45,10153],{"className":10154,"style":301},[300],[45,10156,10158],{"className":10157},[305,306,307,308],[45,10159,3470],{"className":10160},[89,90,308],[45,10162,428],{"className":10163},[427],[45,10165,10167],{"className":10166},[289],[45,10168,10170],{"className":10169,"style":435},[293],[45,10171],{},[45,10173,177],{"className":10174},[228],[45,10176,10178,10181],{"className":10177},[228],[45,10179,177],{"className":10180},[228],[45,10182,10184],{"className":10183},[281],[45,10185,10187],{"className":10186},[285],[45,10188,10190],{"className":10189},[289],[45,10191,10193],{"className":10192,"style":1750},[293],[45,10194,10195,10198],{"style":492},[45,10196],{"className":10197,"style":301},[300],[45,10199,10201],{"className":10200},[305,306,307,308],[45,10202,1947],{"className":10203},[89,308],[45,10205],{"className":10206,"style":254},[232],[45,10208,5614],{"className":10209},[258],[45,10211],{"className":10212,"style":254},[232],[45,10214,10216,10219],{"className":10215},[80],[45,10217],{"className":10218,"style":148},[84],[45,10220,9848],{"className":10221,"style":10222},[89,90],"margin-right:0.0715em;",[16,10224,10225,10226,10254],{},"We see that the maximum likelihood objective is equivalent to minimizing the mean squared error, where ",[45,10227,10229,10242],{"className":10228,"translate":49},[48],[45,10230,10232],{"className":10231},[53],[55,10233,10234],{"xmlns":57},[59,10235,10236,10240],{},[62,10237,10238],{},[65,10239,2634],{},[69,10241,2637],{"encoding":71},[45,10243,10245],{"className":10244,"ariaHidden":76},[75],[45,10246,10248,10251],{"className":10247},[80],[45,10249],{"className":10250,"style":2307},[84],[45,10252,2634],{"className":10253,"style":1707},[89,90]," = the weights and biases of the neural network.",[16,10256,10257],{},"The hidden assumption here is that the errors follow a Gaussian distribution. If we instead assume a Laplace distribution, MLE would lead to the MAE loss. This explains why different tasks choose different loss functions — they correspond to different assumptions about the data distribution.",[1227,10259,10260],{},[16,10261,10262],{},[668,10263,10264],{},"In neural networks, maximum likelihood estimation means finding, among all possible combinations of weights and biases, the set that makes the training data most likely to occur.",[16,10266,10267,10268,671],{},"Therefore, every time we use gradient descent to update weights, we are essentially performing ",[668,10269,10270],{},"MLE optimization",{"title":1169,"searchDepth":1170,"depth":1170,"links":10272},[10273,10274,10275],{"id":1531,"depth":1170,"text":1532},{"id":2366,"depth":1170,"text":2367},{"id":4305,"depth":1170,"text":4306},{},"2026-06-08","\u002Fblog\u002F2026\u002F2026-06-08-explanation-of-neural-network-from-maximum-likelihood-estimation",{"title":1526,"description":1169},"blog\u002F2026\u002F2026-06-08-explanation-of-neural-network-from-maximum-likelihood-estimation","From a statistical learning perspective, modern neural networks can indeed be understood as a large-scale maximum likelihood estimation (MLE) process. Specifically, a neural network is a parameterized function, and the most common way to train a neural network is to perform maximum likelihood estimation on the data.",[10283,1200,10284],"mathematics","statistics","yLsQBVr2bqUThpxIvHATP8EzYTApmSlXPQtwW2rNZ7E",1786880520664]