Conjugate gradient learning algorithms for multilayer perceptrons

Learning complex tasks in a multilayer perceptron is a nonlinear optimization problem that is often very difficult and painstakingly slow. The use of conjugate gradient methods to speed up convergence rates is proposed. These methods result in a very moderate increase in storage and computational complexity compared to the commonly used backpropagation algorithm. The algorithm used is a modified conjugate gradient method that uses inexact line searches. This reduces the number of function evaluations necessary in the line search part of the algorithm. Simulation results that show the improved convergence rate compared to the backpropagation algorithm are presented.