AUTHOR: <lngnmn2@yahoo.com>

Writing a semi-automated trading system is easy - just ask John Carmack to write it down for you.

Before that Eric Evans has to extract the domain knowledge using his DDD approach, and then some top mathematician, or maybe Bartosz Milewski, would build a consistent model from it.

Maybe it is even better to rather ask Simon Peyton Jones to write a matching engine, so you will have it pure-functional, with the most important parts formally proven in Coq (and then translated into Haskell code).

Done.

So, you do not have enough money to hire Carmack and Eveans? Then you have to do it all yourself. The only problem is you have to be an above average trader with actual experience, a domain analyst and model maker, a mathematician and a functional programmer at the same time, and a technical writer and a software tester too.

The “hire the top tallent to do things which you do not fully understand” meme, according to the tradition, goes, through Bell Labs and Los Alamos, all the way back to Henry Ford, but it seems that the ancient Indians knew it too.

So, lets do it all by yourself. What’s the problem?

Understanding the domain

The most difficult part of a good programming is to know what to write and why. The common scenario is that programmers write low-level unmaintainable crap according to how they understand the domain, which is usually even worse (more perverse) than amateur newbies.

You have to have some serious personal trading experience – the more painful and full of stupid errors – the better. Then, and only then, you will realize that almost everything you hear and read is nonsense, but some aspects are indeed crucial.

This experience will allow you to correctly “call bullshit” in what you read in the books and on public forums, which all are full of shit to the brim.

The skill is to extract the actual underlying principles by observing (and generalizing from) recurring patterns on the charts, while correctly mapping (associating) them to the actual social causes – events, current sentiments, memes and other social dynamics.

Sometimes the causes are market actions of institutions and malicious actors trying to manipulate the prince, but these actions are still within the current social environment (sentiments).

The main task, thus, is to extract the underlying principles (which describe the actual causality) from the commonly used idioms and metaphors (which codify the knowledge gained from experience of thousands of individuals).

  • “buy the dip, to ze moon! 100k EOY”
  • “fuck me, I have bought the top”
  • “he sold the bottom, LMAO”
  • “stoploses? never heard of ’em”
  • catching the falling knife
  • I am FOMOing into XXX

These are examples of actual sentences which contain the knowledge.

The entities

One has to extract, identify and formally define (with a precise mathematical specification) every involved entity, including “abstract concepts”, which people actually mention and use.

These are, as in any domain-specific language, “things” and “actions” – nouns and verbs, and there are deep hierarchies and layers of these.

Communication with an exchange (placing Orders) is a sub-language (and a sub-domain) in itself, and goes from high-level “buy” and “sell” order down to the underlying HTTP headers and to the entire RESTful protocol which an exchange public API provides.

The goal is to have a dictionary (a wiki) which defines all the used vocabulary, and zooms through the layers of complexity.

Partitioning

Every domain (an abstract “space”) is implicitly partitioned into distinct “compartments”, reflecting the universal “architecture of complexity” – a hierarchy of nested layers, partitioned by abstraction barriers.

In trading your Orders (which may or may not be filled) and the resulting open Positions are in its own “compartment”, while your trading Account is in another.

Everything related to the analysis if the Candles (market data) should live in its own separate subsystem, so is the communication API at a much lower level of abstraction.

The key is to stay at he highest level of abstraction and to model everything “just right” at this conceptual level. This, basically, has an one-to-one correspondence with the actions a person performs and the reasoning behind them.

The partitions will be “natural”, along the “delegation” boundaries (and person-to-person “interfaces”). One has to have a sketch (a picture) of the system with partitioned into distinct sub-systems.

High-level prototyping

Good programming begins with an exploratory, rapid high-level prototyping (in an interactive high-level language like LISP). Two times high-level here is not an accident. The purpose is to check, validate (and correct) our naive assumptions and expectations early.

There is a quote from a great person: #+BEGIN_QUITE What makes a good programmer? It is a matter of efficiency over the entire production of a program. The key is to reduce wasted effort at each stage.

Things that can help include thinking through your implementation before you start coding, coding in a way that eliminates errors before you test, doing rigorous testing so that errors are found early, and paying careful attention to modularity so that when errors are discovered, they can be corrected with minimal impact on the program as a whole. #+END_QUOTE

Modules

Each entity (or a concept) has to have its own software module which represents and implements it. This is where we go beyond abstract mathematics (of Sets and Relations) to the actual Programming.

Abstraction and Modularity together is the true essence of programming.

Abstraction means how we define (the behavior) and how we interact with (interfaces) our entities, NOT how we actually represent and implement them – this is another, clearly separate topic.

Data Types

At this level of abstraction each well-defined concept has to have its own algebraic data type (in a corresponding module).

The key point is that data-abstractions must non-leaking – completely opaque, “black-box” abstractions. Only the exported public interface (an API) is what is visible to the clients (at the “use-site”).

Again, this is the most important point in all programming - to establish proper “abstract” public interfaces, so the code is being partitioned along with these well-defined, non-leaking abstraction barriers.

Implementations

There are, of course, a mesh of relations between individual types and whole hidden hierarchies at the “implementation details” level.

Once we have a set up proper abstraction barriers (by exporting public interfaces) we could have more than one representation for every data-abstraction and more than one corresponding implementation, which could be safely changed and updated locally, without affection, leave alone breaking, anything else in the rest of the system.

This is the main principle of Modularity in programming, and this, of course, corresponds to ability to replace a faulty part of an assembly in our everyday life, or replacing a faulty protein in a cell biology.

Testing of assumptions and expectations

Languages

Ideally, we have to use the pure functional language Haskell (and sometimes Coq), but very few people can write a proper, non-bullshit Haskell, like in the books of Richard Bird. 99% of public Haskell code is an utter crap by unqualified, attention-seeking Chuds.

This is the most pragmatic decisions, because what we will obtain is a written down, executable rigorous formalism of our problem domain in a pure system of logic (System-F Omega).

Think of writing an executable mathematics or logic – things that people do with the modern proof-assistants.

The efficiency concerns of execution (running) of the resulting formalism is absolutely irrelevant here. It will be good-enough to place Orders and to track the open Positions.

At least it has to be a mostly-functional, statically typed language, like Ocaml, with simple well-defined semantics (and evaluation rules) which contain no hidden imperative mutable state.

Truly immutable and persistent data structures (as of Clojure) is the main criteria.

AI

Deep learning has two fundamental properties

  • It can approximate any computable function

  • without being explicitly programmed

    Notice the word “function”. This implies that the domain may even be infinite, but it has to be fixed.

    It also implies an injection or surjection, which restricts what can be actually “learned”.

    Again, the domain must be fixed (immutable), like a Set of numbers, and, in general, the relations between variables has to be “stable” (persistent).

    The key mathematical principle is the back-propagation algorithm, which updates the weights by calculating a partial derivative, which is a universal way.

    The simple and straightforward optimization algorithms, such as gradient descent, can also be easily traced back to reality.

    A valid intuitive metaphor is that we “learn” a “surface” which will match (will cover, up to the last wrinkle) the whole actual Himalaya.

    One more time (it cannot be over-emphasized) arbitrary inputs to a learning algorithm will result in “learning” an utter bullshit.

    Most of currently used systems feed it with wrong domains and git “hallucinations” as output.