Vi

Oh, look DeepMind and OpenAI win gold at ICPC (codeforces.com) So, a model memorized a lot more of a convoluted stuff and been able to spit out a coherent code. While this is absolutely amazing considering that it works at the level of syntax, and that its “representation” captures only “possible relations among tokens” and posses no “understanding”, thinking leave alone reasoning capacities whatsoever. It just picks up one of the few next most probable tokens, given the previous “context” (a sequence of tokens “so far”). ...