Backpropagation and the Adjoint Method

aka, Backprop through time, the adjoint method, the reverse AD method

Class notes:
DeepOOBackprop

Papers to read:

Papers that are interesting:

Objected Oriented Neural Networks
Due to L. Bottou
Reference implementations include LUSH (LeCun) and Torch (R. Collobert, S. Bengio)
Deep Networks/Local Training
Y. Bengio, G. Hinton work
(Google: Deep Autoassociator)
“Energy Based Models” Paper

Big Modular Networks in Action
NEC/J. Weston work on NLP
G. Hinton’s group on
Y. LeCunn on image processing and robotics
“Gradient-based learning applied to document recognition “
“Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis”