Introduction

Abstraction

As programmers, we are heirs to a great mathematical tradition. Be careful when you meet someone who claims one can program without math. Legend has it that Ageōmétrētos mēdeìs eisítō was inscribed above the entrance to Plato's Academy, which is greek for "let no one ignorant of geometry enter". One explanation for the Ancient Greek emphasis on geometry is very practical: civilization cannot exist without taxes, and keeping track of taxes requires the ability to measure the geometry of land and count the arithmetic of money. A geometer, which translates to "land measurer" was a crucial occupation when floods erased all markings in dirt which demarcated ownership and real estate.

However, another explanation for their emphasis on the geometry of shape over the arithmetic of number is their discovery of the incommensurable of the square root of 2. Because the square root of 2 was non-measurable with integers (irrational), the mathematicians of Ancient Greece decided to found the basis of arithmetic in shape, rather than the other was around. This is an explanation for why Euclid in his Elements provided an axiomatization of geometry instead of arithmetic, which the world had to wait 2000 years later for Giuseppe Peano's axiomaticization of arithmetic. As programmers, we share the same dream as the Ancient Greeks: to reduce the world to discrete natural numbers and in fact, two specific natural numbers of 0 and 1.

Not only do we share the same dream, but we also share the same language. This is because logic and computation are closely intertwined with one another. As programmers, we are heirs to a great mathematical tradition. When asked for the first programming languge, some might say of Fortran, Lisp, Algol, perhaps even APL. Some might say it was actually Ada Lovelace's pseudo formalization of natural language to program Babbage's analytical engine. But in some sense, the original compiler writer and programming language dates back to 2000 years ago. The name of the compiler writer, and the language, is Aristotle, and his logic.

Aristotle was a philosopher; a lover of wisdom. Kant, who was ten times more distant from Aristotle than we are from him, even held that nothing significant had been added to Aristotle’s views in the intervening two millennia. His works spanned anything and everything as he was a synoptic thinker who unified all aspects to his theory and thought. But as programmers, we are interested in his work with compiler construction, or the development of his logic — identifying the formal structures embedded in natural language which constitute valid reasoning — which mathematicians like Euclid used to program the first algorithms known to the world. Today, very few would try to maintain that Aristotle's logic is adequate as a basis modern logic or the type systems in programming languages. But scholars have come to view Aristotle with new respect, not so much for the correctness of his results but for the remarkable similarity in spirit between much of his work and modern logic.

People confuse the work of a logician with that of metaphysics of ontology or epistemology. That is, to answer the questions of what is reality, and what can we know to be true. But in reality, logicians are not concerned with what is "true" or not "true". What they are actually concerned are the properties of systems that people use to make valid inferences from premises to conclusions. The etymology of infer is inferre, which is Latin for "carrying forward" or "bringing in". The logician asks the question of when is it acceptable to carry forward or bring in a conclusion from a premise? This is not too different from the compiler writer not being concerned with the programs that the programmer writes, but the programming language itself.

TODO: inference. induction. problem of induction. sunrise. bayes theorom. generalization.

Returning to the 21st century, classical programming languages can declare functions up front in order to model discrete and deterministic mathematical objects like the List, Tree, and Graph. But they do not experience the same success when describing continuous and stochastic phenomena such as images and language. In order to endow machines with human-level capabilities in image recognition for self driving, or language modeling for assistants, functions must be recovered from data. After your study, you will come to understand that machine learning is the mechanization of inference.

In your hands lays the hacker's guide to high performance machine learning.
Machine learning is the pursuit of endowing machines with the ability to learn with induction.
How this is precisely done will become clear to you soon.

Assembly

You have quite a journey ahead of you.
Are you ready to begin?