Neural Networks: Tricks of the Trade

Chapter 1 Introduction

Chapter 2 Efficient BackProp

Chapter 3 Early Stopping - But When?

Chapter 4 A Simple Trick for Estimating the Weight Decay Parameter

Chapter 5 Controlling the hyperparameter search in MacKay’s Bayesian neural network framework

Chapter 6 Adaptive Regularization in Neural Network Modeling

Chapter 7 Large Ensemble Averaging

Chapter 8 Square Unit Augmented Radially Extended Multilayer Perceptrons

Chapter 9 A Dozen Tricks with Multitask Learning

Chapter 10 Solving the Ill-Conditioning in Neural Network Learning

Chapter 11 Centering Neural Network Gradient Factors

Chapter 12 Avoiding roundoff error in backpropagating derivatives

Chapter 13 Transformation Invariance in Pattern Recognition — Tangent Distance and Tangent Propagation

Chapter 14 Combining Neural Networks and Context-Driven Search for Online, Printed Handwriting Recognition in the Newton

Chapter 15 Neural Network Classification and Prior Class Probabilities

Chapter 17 Forecasting the Economy with Neural Nets: A Survey of Challenges and Solutions

Chapter 18 How to Train Neural Networks