2. Deep Neural Networks

Contents

• 2.1 Sequence Learning with FFN
• 2.2 Differentiation
• 2.3 Symbolic Differentiation
• 2.4 Numeric Differentiation
• 2.5 Automatic Differentiation
• 2.6 Stochastic Gradient Descent
• 2.7 Adam
• 2.8 Feedforward Neural Network
• 2.9 Recurrent Neural Network
• 2.10 Transformers
• 2.11 Mixture of Experts

4.1 Learning Sequences

4.2 Feedforward Neural Networks

4.3 Recurrent Neural Networks

4.4 Transformers

4.5 Mixture of Experts