L. Tao on LLMs
I am being systematically cancelled on HN (as if I am a fucking russian – I am NOT – or something), but sometimes it feels ridiculous. LLM don’t “do maths” by definition (which in this context is the code). LLM “predict” the next symbol after “2+2=” and separately and independently (in principle) predict token-by-token of why that is true. There is absolutely no “reasoning” or “understanding” of any kind whatsoever, just as a parrot would do....