Singularity Systems: Zero to Hero

Dragon Book 2.0: The Hacker's Guide to Tensor Programs and Compilers

Singularity Systems: Zero to Hero is a SystemsML course designed for the age of heterogenous compute that powers software 2.0. This textbook spans the entire semantic chasm from the transistor to the tensor by building 4 compilers (yes four) from scratch starting from the bottom up and ending with a stack of programming languages capable of compiling the llama family of models.

Roadmap

v1.0: pretraining llama
v1.5: distributed pretraining llama
v2.0: post-training (r1) and inference systems (vllm/sglang)

Contributing

The textbook and its 4 compilers are all open source. For discussion come join us at the #singularity-systems work group in the GPU Mode Discord.

Citation

@article{j4orz2025singsys,
  author  = "j4orz",
  title   = "Singularity Systems: Zero to Hero",
  year    = "2025",
  url     = "https://j4orz.ai/zero-to-hero/"
}