Afterword

This textbook focused on prediction and generation. There is also discovery and action.

Prerequisites

Familiarity with the design, training, and inference of machine learning is required in order to interpret and compile them. This is no different from learning how to program before implementing a programming language itself.

  • Cambridge ITPRNN: Information Theory, Pattern Recognition and Neural Networks by David Mackay
  • Tubingen ML4202: Probabilistic Machine Learning
  • Stanford CS109: Probability for Computer Scientists by Chris Piech
  • Stanford CS229: Machine Learning by Andrew Ng
  • Stanford CS224N: NLP with Deep Learning by Christopher Manning
  • Eureka: Neural Networks Zero to Hero by Andrej Karpathy
  • Stanford CS336: Language Modeling from Scratch by Percy Liang

Corequisites

SCTP follows the breadth-first spirit of SICP's accelerated introduction to computation. SICP iteratively deepens the meaning and semantics of computation by providing concise coverage on substitution, stack/heap, operational interpreter, and ends with a register machine. SCTP follows suit with blas/dnn operations, autograd, learning, and compilation. Along your adventure with tensor programs, you may find some of the following specialized courses useful to consult with.

  • Brown CS053: Coding the Matrix by Philip Klein
  • MIT 18.S096: Matrix Calculus by Alan Edelman and Steven Johnson
  • Stanford CS149: Parallel Computing by Kayvon Fatahalian
  • Berkeley CS267: Applications of Parallel Computers by Katthie Yellick
  • Berkeley CS265: Compiler Optimization by Max Willsey
  • Cornell CS4120: Compilers by Andrew Myers
  • Cornell CS6120: Advanced by Adrian Sampson
  • Cornell CS4787: Principles of Large-Scale Machine Learning by Chris De Sa
  • Cornell CS6787: Advanced Machine Learning Systems by Chris De Sa
  • Carnegie Mellon 18-447: Computer Architecture by Onur Mutlu
  • Carnegie Mellon 15-411: Compiler Design by Frank Pfenning
  • Carnegie Mellon 15-745: Optimizing Compilers by Phil Gibbons
  • Carnegie Mellon 10-414: Deep Learning Systems by Tianqi Chen
  • Rice COMP412: Compiler Construction by Keith Cooper
  • Rice COMP512: Advanced Compiler Construction by Keith Cooper