https://x.com/karpathy/status/2015883857489522876 – 5.5M views. What could we possible do?
Meanwhile, my 2 cents: The current Sonet 4.5 via the web-interface (free-tier) is what I have access to. It can generate very convincing verbiage, consistent with we could find in the best books. That only means it has been trained on the [pirated] books too.
Last iteration it wrote a very nice few pages summary of the Google’s testing practices, found in the public domain, like the SWE book, Abseil guidelines, the Testing blog, etc. This is only impressive because it managed to combine all these sources into a coherent narrative, without any major hallucinations.
The code, however, what it generated was a stale pasta from github repos – the lua configuration for Neovim (yeah, actually know what your are asking for first!). The config failed in a subtle ways, and I even could tell (not, really) from where the pieces were taken (I have seen tons of this shit across github).
No suprises here.
So, the hype. When you DO NOT KNOW your subject really well, you cannot actually evaluate the quality of the slop, and you are very impressed, just like our “messiah”, who is barely a coder. If, however, you have spent years on some niche, obscure domains, then you can see how crappy the slop actually is, compared to what it could be by just following some “common-sense” principles, outlined in the popular literature (“Modern Software Development”, “the Google SWE book”, “Domain Model Made Functional”, and what not). Yes, I am well-read Humbert.
Gemini3-pro, on the other hand, wrote almost no verbiage, but the code and the comments. The code was also a github pasta, but it felt way more “professional”.
I do not know how Google trains its models, but it is kind of obvious, that they just have to feed in their “30+ years” of best practices on the monorepo, especially that applied “leaning to the left” maxim, which is nothing but a clever summary of the essential TDD and XP practices, which are the most essential for writing good code.
Anyway, who are we compared to @karpathy? Just peasants.
And no, no miracles that Anthropic is promising to us every single day, in sight.