Neural Networks: Tricks of the Trade

Chapter 1 Introduction

Chapter 2 Efficient BackProp

Chapter 3 Early Stopping - But When?

Chapter 4 A Simple Trick for Estimating the Weight Decay Parameter

Chapter 5 Controlling the hyperparameter search in MacKay’s Bayesian neural network framework

Chapter 6 Adaptive Regularization in Neural Network Modeling

Chapter 7 Large Ensemble Averaging

Chapter 8 Square Unit Augmented Radially Extended Multilayer Perceptrons

Chapter 9 A Dozen Tricks with Multitask Learning

Chapter 10 Solving the Ill-Conditioning in Neural Network Learning

Chapter 11 Centering Neural Network Gradient Factors

Chapter 12 Avoiding roundoff error in backpropagating derivatives

Chapter 13 Transformation Invariance in Pattern Recognition — Tangent Distance and Tangent Propagation

Chapter 14 Combining Neural Networks and Context-Driven Search for Online, Printed Handwriting Recognition in the Newton

Chapter 15 Neural Network Classification and Prior Class Probabilities

Chapter 16 Applying Divide and Conquer to Large Scale Pattern Recognition Tasks

Chapter 18 How to Train Neural Networks