AI

Things begin to move way too fast, at least for me. There is some amazing fact, which shows that an intuitive understanding can also be valid. I remember reading the memest book of the time – “eXtreme Programming Explained” (by still sober and sane Kent Beck), and trying (probably just like everyone else at the time) to “distill” some grains of “wisdom” from the torrents of “mostly bullshit”. The real take away from the whole manuscript (at least for me) was that writing tests is what really change everything, in particular, they are, indeed, allow quick, confident and “cheap” refactoring and thus really “facilitate change” and, yes, actually speed up the development process. ...

For a quite long time already I sort of “ran in the background” this question – “how they bridge the semantic gap between verbiage and the code?” In the context of a well-written textbook (extremely rare, just 50 or so in existence) the code examples immediately follow or even intersperse the explanations. Most of the internet content is nothing like that. The semantically closest entities in the code are specially formatted comments that are used to generate [an appearance of] “documentation”, with a modern fashion to mention some core concepts being used. This kind of well-documented (enforced by the strict rules) code one would find in placed like Google’s monorepo and other megacorp inner code bases. Almost noting this strict can be found on Github (the primary source of all training). ...

https://karpathy.bearblog.dev/year-in-review-2025/ In this episode @karpathy blessed us all with another blogpost. While his wording is much more careful and even nuanced, there is still a lot of bullshit in it. It way less outrageous bullshit as in the Friedman poocast and around that time, but still. Here are some excerpts: With vibe coding, programming is not strictly reserved for highly trained professionals, it is something anyone can do. … But not only does vibe coding empower regular people to approach programming, it empowers trained professionals to write a lot more (vibe coded) software that would otherwise never be written. ...

Okay, lets look at the “better side” of things. The good thing about using LLMs is that you do not have to deal with Google Search and any fucking Social Media. Imagine a painfully typical scenario – you want to clarify or better understand something you already vaguely knew or at least aware of. You type a query into Google Search, and you get… a fucking CEO fucked-up list of Ad-infested links to various web pages – either the largest social media containment boards (StackOverflow, Reddit, Medium), or some CEO’d blogs, when you are either greeted with a wall of text (usually directly pasted from tutorials and docs), ads, pop-ups, and other distractions, or some narcissistic asshole’s low-effort over-verbose crappy verbiage about “how fucking smart he is”. ...

Just like a spontaneous, “natural and organic” continuation to the previous post (which implicitly confirm that it has properly captured at least some aspect of reality [as it is]). I have had to delete some nice movies in order to download and try that over-hyped “5M downloads” Nvidia’s meme-model Nemotron-3-Nano-30B" I have a small set of highly sophisticated prompts which I use to measure the apparent quality of a generated slop of the 4 major data-center-sized LLMs. ...

Everything is broken and idiots are everywhere. There is a clown which attention whoring, sorry, publicly arguing (and gaining a lot of unwarranted attention) that one shall vapecode in C. https://news.ycombinator.com/item?id=46207505 Basically, making such a claim is idiotic on so many levels that it is hard to know where to start. Almost the whole of classic non-bullshit programming language theory research is about how to correctly address C’s shortcomings and semantic issues, and how to avoid the inherent in the design of the language (and the ABI) problems. ...

So, it seems like this is the time to somehow sum up the current AI hype (way through the roof) and the immediate and long term consequences of it. First of all, a proper education – studying the fundamental underlying principles instead of particulars – which used to be an unofficial mantra of MIT, pays off again. One just sets particular constraints to a coding LLM and use it as a whole-data-center-powerful constraint satisfaction engine that spews out a slop, which then can be used for rapid prototyping and minimal-viable products. The properly constrained slop can even be used as the basis of a project, which then undergo proper continuous improvement and refinement by a human expert (who knows the whys). ...

I think I have seen this before. Once in Varanasi, wandering around book stalls (most titles already “tourist books”, – oversimplified and westernized “tantric” bullshit), I found a whole book by some local publisher which describes in a minute details one single Brahmanic ritual (an elaborate sacrifice) which last almost a whole day. Hundreds of ingredients are being burn in a precise sequence, or rather a simphony of chants. motions, gestures (mudras) and many other elaborate details. The priests (brahmans) definitely knew what they are doing and why exactly this way is the only proper way. ...

The thing I hate the most is when some of these fucking YouTube content “creators”, which decide to monetize an AI coding clickbait with low-effort subpar videos, say “aaand boom!” when another chunk of a slop has been spewed out by an AI. This “boom!” is an insult to the last 60 years of the programming languages research (including the math-based theory) and to the “old sages” which crafted their languages and standard libraries in the best possible, “just right”, perfect, in the sense of “nothing more to take away” by surveying all the available literature, non-bullshit papers and spending months of anguish and self-doubt. ...

So, things begin to move a lot faster and much bigger, and there is something to realize about this unpreceded AI bubble. We will consider only the underlaying fundamental principles, not the particular implementation details, “architectures” and what not.. There are four major aspects to any LLM model – the training process, the “architecture” (the structural shape) of a model, the " post-training tuning" (lobotomy) of the model and the inference process. ...