TY - JOUR
T1 - Effect of initial weights on back-propagation and its variations
AU - Lari-Najafi, Hossein
AU - Nasiruddin, Mohammed
AU - Samad, Tariq
PY - 1989/12/1
Y1 - 1989/12/1
N2 - The effect is studied on the convergence properties of the backpropagation learning rule of the range from which the initial weight values are randomly selected. In addition to the standard backpropagation rule, two variations are also considered, namely symmetric backpropagation and expected-value backpropagation. In most applications of backpropagation, the range of initial weights is small. It is shown that significantly higher initial weights can substantially improve learning rates. If the initial weight range is increased beyond a problem-dependent limit, however, performance degrades. Symmetric backpropagation is most sensitive to the initial weight range, while expected value backpropagation is least sensitive. The authors describe an improvement on the symmetric variation that produces faster learning rates with low initial weights.
AB - The effect is studied on the convergence properties of the backpropagation learning rule of the range from which the initial weight values are randomly selected. In addition to the standard backpropagation rule, two variations are also considered, namely symmetric backpropagation and expected-value backpropagation. In most applications of backpropagation, the range of initial weights is small. It is shown that significantly higher initial weights can substantially improve learning rates. If the initial weight range is increased beyond a problem-dependent limit, however, performance degrades. Symmetric backpropagation is most sensitive to the initial weight range, while expected value backpropagation is least sensitive. The authors describe an improvement on the symmetric variation that produces faster learning rates with low initial weights.
UR - http://www.scopus.com/inward/record.url?scp=0024908306&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=0024908306&partnerID=8YFLogxK
M3 - Conference article
AN - SCOPUS:0024908306
SN - 0884-3627
VL - 1
SP - 218
EP - 219
JO - Proceedings of the IEEE International Conference on Systems, Man and Cybernetics
JF - Proceedings of the IEEE International Conference on Systems, Man and Cybernetics
T2 - 1989 IEEE International Conference on Systems, Man, and Cybernetics. Part 1 (of 3)
Y2 - 14 November 1989 through 17 November 1989
ER -