DATE: <2024-11-20 Wed> Lets write a few paragraphs which will destroy the current LLM narrative (naive bullshit), while neither any single piece nor the whole article can be refuted. This is a high-level proper (classic Eastern) philosophy, which is many levels away from simple logical forms, but it still can be reduced to these, if one wants to. The Proper Philosophy isn’t dead, not even it is dying. It cannot, lmao....
Who haven’t seen the famous meme-movies about the markets? “The Big Shot”, “The Margin Call” and, of course, “The Wolf of Wall Street” (in which, remarkably, there were no mathematical models involved – just plain old bullshitting). And, when we saw that guy, the second place on a math Olympiad, we probably felt a pang of pain and regret. That could have been me. Math, it turns out, is not that complicated after all....
Once again I tried to through This meme video. Once again with the same results. I have seen these social patterns many times before – when people begin to use ill-defined, anthropomorphic and purely abstract concepts to construct familiar analogies and to invoke intuitions, so everything seems “right” and “logical”. Abidharma uses abstract terminology to produce a seemingly coherent system. It started from very reasonable abstractions of The Buddha, which illustrated his ideas with abstract notions like pealing off layers of an onion (to find literally nothing in the center) or a mixture of spices using for cooking, but very quickly went all to pure abstract bullshit....
This is sort of an answer to this question. https://news.ycombinator.com/item?id=38425475 So, what if you are late to the party? Unfortunately, nowadays it is even much harder to get through all the utter bullshit and hype, but there is a sort of a shortcut or “the Hard Way”. There are two and a half key figures: Geoffrey Hinton, who did most of the mathematical heavy lifting, Andrew Ng, who not just did all the derivations, but became the most famous practitioner, and Andrej Karpathy who us just a narcissistic asshole, similar to Lex Friedman....
The only reasonable and even proven correct point of applying deep-learning is that the algorithm “learns” so-called “hidden patterns”, which escape (literally) from a biologically limited human intelligence. The causality is so complex that even to list correctly the major factors (not missing anything relevant) is nearly an impossible task. Social media creates whole waves of bullshit, including the idiotic memes about the US interest rates, and other poorly understood and dynamic macro-economic factors....
Social media make us stupid. To be precise - they encourage production and emission of a useless verbiage as a from of virtue signaling. The cultural change is that being “wrong” is ok for some talking heads, and nowadays it is even possible to argue that “there is no wrong”, just an “imperfect information”, you know. The older cultures were better. They had a common sense notion of “you have no idea what you are talking about”....
DESCRIPTION: There is no “I” in your AI. Lets put all the memes and bullshit aside, for a moment and talk serously about GAI (hello, mr. Carmack, sir). There is what every “AI researcher” should know about Knowledge and Intelligence (yes, both capitalized). There is so-called “reality” prior to any knowledge or intelligence. Any reasonable thinker has been arrived at the ultimate reality. “I am That” (“That Thou Art”) of Chandogya Upanishad is the “end of knowledge” and the “arrival” to the ultimate “truth”, with implies existence of “That” and one being just a sub-process (a wave) in It....
These are just assorted notes for now, which shall become something ready to be formalized. Non-bullshit The objective is to train a NN which captures subtle recurrent patterns among many well-chosen (and well-defined) features. The proper set of features that, in turn, captures the most relevant aspects of reality is what determines the distinction between a modest success or a total failure of this ML approach. All the features should be actual “measurements” of something real, like “Open Interest” or the “Long/Short ratio” and other obvious measurements like “Volume”....
Overview A valid (less wrong) intuitive metaphor is that we “learn” a “surface” which will match (will cover, up to the last wrinkle) the whole actual Himalaya. This notion “generalizes to any number of dimensions” meme (differences, distances and derivatives do not care about Mind’s abstract bullshit) The Himalayas (Truth) has to be “out there”. A good generalization is a bucket-sort, which can be thought off as a specific example of a classification problem....
Using the standard formalisms only within appropriate contexts Fitting (using an optimization method) a weighted sum, a line or a curve or a whole “plane” to match the data. Statistics This is, by definition, observing, categorizing and counting observations (which also means “measuring”) that already happened. There is no notion of any potential or possible outcomes. We just observe and count (measure). My favorite example is from archery, when one just measures the distances between hits (arrows), sum them and divide by the number of shots fired (which is averaging of the distances)....