Afterword

To continue deepening your knowledge, the following courses are also a good next step. You might find this book complementary to your reading, since all three streams were woven into a single presentation for each concept presented throughout the book. Once you feel comfortable, you should graduate towards contributing to larger machine learning systems.

Good luck on your journey. I'll see you at work.

Tensor Programming

  • Cambridge: Information Theory, Pattern Recognition and Neural Networks by David Mackay
  • Tubingen ML4202: Probabilistic Machine Learning by Philipp Hennig
  • Stanford CS124: From Languages to Information by Dan Jurafsky
  • Stanford CS229: Machine Learning by Andrew Ng
  • Stanford CS230: Deep Learning by Andrew Ng
  • Stanford CS224N: NLP with Deep Learning by Christopher Manning
  • Eureka LLM101N: Neural Networks Zero to Hero by Andrej Karpathy
  • Stanford CS336: Language Modeling from Scratch by Percy Liang

Tensor Interpretation

  • UPenn STAT 4830: Numerical Optimization for Machine Learning by Damek Davis
  • MIT 18.S096: Matrix Calculus by Alan Edelman and Steven Johnson
  • MIT 6.172: Performance Engineering by Charles Leiserson and Julian Shun
  • MIT 6.S894: Accelerated Computing by Jonathan Ragan-Kelley
  • Berkeley CS267: Applications of Parallel Computers by Katthie Yellick
  • UIUC ECE408: Programming Massively Parallel Processors by Wen-mei Hwu
  • Stanford CS149: Parallel Computing by Kayvon Fatahalian
  • Stanford CS217: Hardware Accelerators for Machine Learning by Ardavan Pedram and Kunle Olukotun
  • Carnegie Mellon 18-447: Computer Architecture by Onur Mutlu
  • Carnegie Mellon 18-742: Parallel Computer Architecture by Onur Mutlu
  • ETH 227: Programming Heterogeneous Computing Systems with GPUs by Onur Mutlu

Tensor Compilation

  • Google DeepMind: How to Scale Your Model: A Systems View of LLMs on TPUs
  • HuggingFace: Ultra-Scale Playbook: Training LLMs on GPU Clusters
  • Berkeley CS265: Compiler Optimization by Max Willsey
  • Cornell CS4120: Compilers by Andrew Myers
  • Cornell CS6120: Advanced Compilers by Adrian Sampson
  • Cornell CS4787: Principles of Large-Scale Machine Learning by Chris De Sa
  • Cornell CS6787: Advanced Machine Learning Systems by Chris De Sa
  • Carnegie Mellon 15-411: Compiler Design by Frank Pfenning
  • Carnegie Mellon 15-745: Optimizing Compilers by Phil Gibbons
  • Rice COMP412: Compiler Construction by Keith Cooper
  • Rice COMP512: Advanced Compiler Construction by Keith Cooper