Here we formulate (and analyze the implications of) The Fundamental Problem of LLM-assistant vapecoding.

The problem is this: There is, at the moment, no machinery to systematically and correctly bridge the gap between the generated slop (code) and the accompanied verbiage, which appear to define the semantics of the code.

What appears to be coherent and consistent is only a cognitive illusion, made out of familiar words, which refer to familiar concepts of the mind, which looks plausible [to the mind].

Here is how a human would do this.

  • Analyze the language used by the domain experts and extract the key terms, the entities and the kinds of entities, the individual “stories”, and explicit (and implicit) rules.
  • Model the “Domain” as a mathematical model, using Sets and Relations (mappings as pure functions).
  • List all possible states for each entity [to be in] and formally define a FSM, and then implement explicit state transition functions (transducers).
  • Represent these entities using only the proper Algebraic Data Types (sums, products. arrows) and encapsulate these behind a clear abstraction barrier – inside Abstract Data Types (ADT).
  • Identify the common traits between the Domain-Specific Abstract Data Types, and write down the corresponding Traits (type-classes)
  • Analyze the resulting Traits and add specific constraints (trait bounds) –the more the better.
  • Use the stateless abstract interfaces and the only public interfaces exported from an orthogonal, loose-coupled module, which does only one specific thing (but does it well).
  • Apply the XP and TDD practices and now that new meme-book “Modern Software Development” (which looks like an AI-assisted rip-off of all the previous painfully discovered practices).
  • A lot more subtleties, rules and heuristics.

No LLM in existence does anything like this, in principle. The actual algorithms being used (actually implemented) at any stage of the whole mathematical pipeline use “dumb” weights and have “no notion” of the underlying semantics of any kind.

One more time: the problem of connecting the semi-formal text to a formal code is non-trivial (the last 50 years of “research” and millions of man-hours were spent to just a few new memes) and by no means is “solved” by LLMs. the LLMs only succeed in producing a convincing appearance of everything being “right”.

What appears to be a “rigorous” and over-confidently presented statements about the code and the explanations are only a “cognitive illusion”, made out of familiar words, which refer to familiar concepts of the mind, which looks plausible [to the mind].

The meme-theory about multi-dimension topological clustering of “tokens” into “clusters, which capture the semantic distances” (Slutsker, and other abstract bullshitters) and the “next-token prediction” inference algorithm is not a substitute for the actual understanding of the underlying semantics of the domain. This shall be obvious even to our messiah @katpathy. He just chooses to ignore this obvious fact, because it does not fit his narrative of “AI will solve everything”.

There is actually nothing more to say about this. The problem is clear, and the implications are self-evident.

Just look carefully at the slop and try to actually validate the claims made in the verbiage. You will see that there is no actual “deductible” connection between the two, since the generation process is fundamentally at the level of weights only, and thus stochastic and “apparently probabilistic”.

This problem cannot be solved by “better LLMs” or “more data” or “more compute”, again, in principle.

But yes, we can use LLMs as “coding assistants” (the Glorified Autocomplete, indeed) to help us write boilerplate code, or to help us with “syntax” or “idiomatic usage” of a specific programming language (which is exactly what Andrej is bragging about on Twitter), but forget about the correctness (of any kind) of the generated code, or the “validity” of the claims made in the verbiage.

Have a nice day.