Adaline神经网络

Adaline利用了最小二乘法的思想，相较于感知机神经网络，对于数据的线性可分的要求更低一些，可以允许一些异常数据。


上面描述了迭代求解的过程，但是在x0(k+1)<script type="math/tex" id="MathJax-Element-1484">x_0(k+1)</script>这里没看懂，如果是更新θ<script type="math/tex" id="MathJax-Element-1485">\theta</script>应该是w0(k+1)<script type="math/tex" id="MathJax-Element-1486">w_0(k+1)</script>才对。应该是PPT有错。
我们来说明下迭代求解需要注意的地方。
首先，将θ<script type="math/tex" id="MathJax-Element-1487">\theta</script>并入到w<script type="math/tex" id="MathJax-Element-1488">w</script>中去，因为θ<script type="math/tex" id="MathJax-Element-1489">\theta</script>前面的系数始终是1，所以x<script type="math/tex" id="MathJax-Element-1490">x</script>也多了一列，这一列都是1。

<script type="math/tex; mode=display" id="MathJax-Element-1491"> \bf x=[1, x_1, x_2, \cdots, x_n]^T \\bf w = [\theta, w_1, \cdots, w_n] </script>
然后是迭代的结束条件。在代码实现中，是计算w(k?1)<script type="math/tex" id="MathJax-Element-1492">w(k-1)</script>与w(k)<script type="math/tex" id="MathJax-Element-1493">w(k)</script>之间的差来判断的，如果差很小很小，说明梯度下降很缓慢，也就是快到达极值点了。

MATLAB实现迭代求解

Adaline.m

function [ w, t ] = Adaline( X,step, init_w, init_t )
%ADALINE Summary of this function goes here
%   Detailed explanation goes here
%   X: data set with label
%   f: active function
%   step: step size
%   init_w:
%   init_t:
    if nargin < 4
       init_t = 0;
    end

    if nargin < 3
        init_w = [];
        init_t = 0;
    end

    if nargin < 2
        step = 0.1;
        init_w = [];
        init_t = 0;
    end
    label = X(:,end);

    data = http://www.mamicode.com/X(:,1:end-1);
    [n_data,n_fea] = size(data);
    data = http://www.mamicode.com/[ones(n_data,1) data];
    n_fea = n_fea + 1;

    n_w = size(init_w);
    if n_w ~= n_fea
       init_w = ones(n_fea,1);
       n_w = n_fea;
    end

    w = init_w;
    w(1) = init_t;
    eps = 1e-6;

    while true
        for i=1:n_w
           dw = (label - data*w)‘*data(:,i);
           new_w(i) = w(i) + step*dw;
        end
        if sum((new_w‘-w).^2) < eps
            break;
        else
            w = new_w‘;
        end

    end
    t = w(1,1);
    w = w(2:end);
end

demo.m 用于简单测试

clc;

%% create random points for train
c1 = [1 1];
c2 = [3 3];

n_L1 = 50; % number of item with label 1 
n_L2 = 20; % number of item with label 2

A = zeros(n_L1,3);
A(:,3) = 1;
B = zeros(n_L2,3);
B(:,3) = -1;

% create random point
for i=1:n_L1
    A(i,1:2) = c1 + randn(1,2);
end
for i=1:n_L2
    B(i,1:2) = c2 + randn(1,2);
end
% show points
scatter(A(:,1), A(:,2),[],‘r‘);
hold on
scatter(B(:,1), B(:,2),[],‘g‘);

%% training with above points
%AA = [ones(n_L1,1) A];
%BB = [ones(n_L2,1) B];
X = [A;B];
%X = [1 1 1 0 1;1 1 0 1 1;1 1 1 1 -1];
[w, t] = Adaline(X,0.0001);
%[w,t] = Adaline2(X);


%%
% plot the result
A = w(1);
B = w(2);
C = t;
if B==0
    %生成100个-C/A放在向量x中.
    x=linspace(-C/A,-C/A,100);
    %从-A)-(|A|+|B|+|C|)到|A|+|B|+|C|等距离生成100个值放在向量y中.?
    y=linspace(-abs(A)-abs(B)-abs(C),abs(A)+abs(B)+abs(C),100);
else
    x = linspace(0, 10, 100);
    y = -w(1)/w(2) * x;
    y = y - 1/w(2)*t;
end
hold on
plot(x,y)

正规化求解

除了迭代求解外，有一种正规化的的方法。这种方法对于不是很大数据集求解是很快的，主要是因为要求矩阵的逆，如果矩阵太大了，求逆将会变得很慢很慢。

Adaline2.m

function [ w, t ] = Adaline2( X )
%ADALINE2 Summary of this function goes here
%   Detailed explanation goes here

    data = http://www.mamicode.com/X(:,1:end-1);
    label = X(:,end);

    [n_data, n_fea] = size(X);
    data = http://www.mamicode.com/[ones(n_data, 1) data];
    n_fea = n_fea + 1;

    w = pinv(data‘*data)*(data‘*label);

    t = w(1,1);
    w = w(2:end);
end