首页 > 代码库 > BP, Gradient descent and Generalisation

BP, Gradient descent and Generalisation

For each training pattern presented to a multilayer neural network, we can computer the error:

yd(p)-y(p)

Sum-Squared Error squaring and summing across all n patterns, SSE give a good measure of the overall performance of the network. 

SSE depends on weight and threshholds.

技术分享


Back-propagation

Back-propagation is a "gradient descent" training algorithm

技术分享

技术分享

Step:

1. Calculate error for a single patternm

2. Compute weight changes that will make the greatest change in error with error gradient(steepest slope)

only possible with differentiable activation functions(e.g. sigmoid)

技术分享技术分享

gradient descent only approximate training proceeds pattern-by pattern.

gradient descent may not always reach true global error minimum, otherwise it may get stuck in "local" minimum.

技术分享

solution: momentum term

技术分享

 

BP, Gradient descent and Generalisation