Giving the exponentially increasing amount of bullshit, like this one:

https://news.ycombinator.com/item?id=47521150

we have to perform regular sanity checks to avoid our understanding being polluted by this stormy sea of screaming bullshit (if you will).

First, nothing has been changed in principle.

  • this is still an abstract token-level (not even syntax) probabilistic modelling (predicting the next token, given the previous (and updated) state)
  • the “architectures” such as variation of “Transformers” is still a socially constructed Tantric bullshit, every one is parroting, but no one understands, leave alone validates
  • given these two actual properties above, any system, however large and “tuned” (nowadays it is RL – feeding the user’s prompts and the resulting slop back into the model as new training examples), everything they produce is still a slop, in principle.

There is nothing more to add or clarify.

There are specific problems when used for vape-coding:

  • the models still restart “from scratch” (reloading the saved state and the context) on every restart, so the “consistency” is a cognitive illusion (there are lots of them, and this is how it sells)/
  • the models still cannot, in principle, bridge the semantic fundamental gap between the code and the comments straight above it, so they systematically fuck up (delete) both and rewrite larger blocks.
  • one more time: when one actually observe how models fix the errors, they do not fix just “these few characters”, but spew out (rewrite) a whole (much larger than necessary) block, which “explains” how they actually work with code
  • the models fail to write proper unit and integration tests (according to the problem domain semantics) most of the time, generating simplistic “asserts” instead (no surprise here – they have been trained on such kind of crap).

The whole thing is still a sophisticated cognitive illusion, which overwhelm and confuse the mind of an observer. It feels like clear 10x, but when one scrutinises the diffs and analyses the semantics (between the mental model, the model expressed in maths (algebraic data types), the prompt itself and the output (the slop)) one sees subtle lies here and there. It is not the blatant hallucinations as before, but just lies, when the textual slop (usually a clear and reassuring manager-speak) does not correspond to the actual code right below it (I am Jack’s utter lack of surprise).

There is one more “philosophical” (of the Mind) observation. The slop generator has a fundamental difficulty of merging distant domains into a single coherent “semantic structure”, simply because there were no such connections in the training data.

When you, for example, prompt it to write Initial Algebras (which is just some GADT) and a corresponding Unique Homomorphism (which is just an Interpreter) in Rust (yes, because it has a high-level pure-functional subset, which can be made to achieve an ad-hoc referential transparency by explicitly banning mut and ref mut and rely on its strict move semantics) the resulting slop is just a usual imperative cap – variations of the low-level imperative amateurish low-effort crap it has been trained with.

And this is Gemini-3-pro. The Claude and Codex cannot even comprehend the prompt correctly (since they has been trained and heavily RL-tuned on some Python and Javascript crap from Github).

This problem is fundamental, easily observable, and have no solution (for merging together seemingly distant domains the level of tokens is, in principle, not enough).

But why, post more bullshit on social media, buy more AI stonks, and have a good time while it lasts.