DATE:
A lot in common with cooking, which is (arguably) the simplest and the most ancient form of engineering. (TODO: explain with examples). No one can learn to cook by watching a “food porn” on social networks. It is a “learn-by-doing (and making mistakes)” process.
Small, complete (Always Be Compiled) iterations, which conceptually corresponds to “recursive calls” of a spiral-shaped recursive process, which ends up at (converges to) a local optimum.
The analogy is a flawed analogy because recursive function calls are “suspended” and the actual computation occur “backwards” when recursion unwinds (after reaching and computing base-case).
What is truly remarkable, is that the whole shape of an unwinding recursion is just a single chain of nested function “returns”, from the “deepest” (the base case) back to the function call (and to the “caller” function).
Nesting is the only proper way to compose functions, and recursion is just a composition with itself.
Anyway, the world “recursion” is overloaded, just like any other abstract term. It is, however, useful analogy to visualize recursive process as a spiral which “ends” at the base-case.
And, surprise! a spiral is just a curved (instead of straight) line (a sequence of points), or a path. The insight is that each “cycle” is closer (to the optimum) but goes through the same phases or “stages”.
Draw a pie-chart and embed a spiral to its center. This picture is our inaccurate but very useful abstraction – sort of an intuitive understanding, which is NOT wrong in principle.
Without all this esoteric stuff, it is just what liberal arts normies call an “iterative, continuous refinement” (until there is nothing more to take away). Real artists know and understand the processes.
partial solutions like partial functions
Each branch of a sum-type is a distinct (non-overlapping) partition, so a corresponding clause (of a pattern-matching expression or a function defined by clauses) is a partial function itself.
The trick, William Potter, is that another clauses can be added (and even safely removed) at any time without breaking any other clauses (partial functions).
Nothing like this exists in any other art form. Even plain text is more difficult to modify (because it is less structured and you don’t know where to change), leave alone a canvas or a sculpture.(only from scratch)
The great Classic Languages have proper “mathematical” sum-types, not some decorated crappy enums..
So, unlike cooking, which is a continuous process, we can do iterations, which is the essence of a non-bullshit “Agile” (if “Agile” captured anything real, it is these distinct short iterations, which end up in a stable state of the whole project).
The closest analogy is a partially assembled complex artifact (an aircraft jet engine, say) that is always left in a stable state between shifts, except that we do not have the detailed top-down step-by-step instructions, and have to invent the engine as we assembling it and to change the parts on the go.
Most of cooking cannot be “paused”, while having a “stable intermediate form” (yes, the concept from the molecular biology). Properly structured software components can.
By “properly structured” I mean proper algebraic types (and ADTs) and layers of DSLs (Sussman).
Starting with types
“Properly structured from the start” means stating with types. At the most abstract level these are Algebraic Data Types – either sum, product or function types. This is literally “all you need”.
Every concept captured from a problem domain most of the time is either a sum or a product, with nesting, if necessary. “Things” are almost always a “compound” product type, while “possibilities” (or possible states) are clearly sum-types.
State transitions are naturally “connected arrows” – curried functions and function compositions. These arrows (or paths) will form a graph, which will be reduced by evaluation.
It is not a random coincidence that “graph reduction” is the only formal model of lazy evaluation of a pure functional language.
So, “sums”, “products” and “function composition” (which is nesting).
One concept – one ADT – one module
This is a discipline – just the right thing to do. Like in the classic languages (of the ML family) – a file that exports an abstract public (stateless) interface and a file that contains (and hides) the representation and implementation details. In Haskell everything declared in a single file (no header files).
This, again, is just the right thing to do. Even in Python.
Header files a great idea – sometimes you don’t have the sources, but you may have only the headers. This also “promises” that these interfaces are abstract, stateless and “stable” (do not change on a whim).
Stable abstract interfaces or ADTs (which, ideally, form layers of DLSs) are the central notions of the whole discipline.
just 4 universal forms (or shapes) of the data
- a sequence
- a tree
- an acyclic graph
- a table
the practice
Just like one cannot learn how to play a violin by watching others play and by reading about it, one cannot learn how to program by reading. It is a learn by doing discipline, just like music or sports or almost every other human skill (driving a bike or a car, etc).
The proper practice is to write a dummy module and its test file first, by writing down a function specification – what it is supposed to do (as we currently understand it), then write the types – the function signature.
The body should just return a dummy “default” value (like 0 or an """) and then immediately write a test that checks some actual property (according to the specification and the signature) related to this dummy value.
Make sure everything compiles, and all the tests passed. This forces you to thing in the only proper way of thinking – at the level of abstract interfaces and their almost-formal specifications. This is the true non-bullshit essence of programming.
Barbara Liskov’s books, Michael Clarkson’s “ABC” principle (and its amazing course) and the whole course about Systematic Program Design by Gregor Kiczales – all converge to this very practice (plus my emphasis for stable intermediate states when just a single clause or a single branch or path added, and it compiles and passes all the tests).
“All tests passed” is a powerful positive reinforcement and the fastest (and the earliest) feed back loop. Motivation fades away very quickly, so a set of proper /habits and a strict discipline (like of an athlete) is what carries one along the way.
The early feedback is the key for human (and even animal!) learning process..
One more time – one writes dummy modules and dummy tests first.Then the algebraic types (which represent a captured concept named by a module). The function specifications (of operations) and the function’s type-signatures first, with a body that returns a dummy value – the stub (which can be tested right away).
This is not some pedantry by some old guys. This practice and the whole process keeps you motivated and in a proper psychological state, so you could keep going. This has been discovered by a lot of trials and errors, just like singing in a church – it just works.